Can we have data as flat files instead of JSON?

Feature requests Genetics feature requests
1

According to data access docs, data is available at EBI FTP and GoogleCloud as multiple JSON files.

Is there anyway we can have them as flat text files - CSV, TSV?

2

Hi @zx8754!

We don’t provide the data downloads as flat text files, as the kind of structured data they contain is not compatible with flat text file formats.

However, tables on individual pages are available as CSV and TSV files. For example, you can download the assigned genes table for 1_154453788_C_T as a JSON, TSV or CSV file by selecting that option in the top right hand corner of the table.

I hope this helps!

3

Thanks, I need to lookup about 1000 variants. Thought having data locally as flat file would make it easier. Currently using Google Big Query option.

4

I think it maybe helpful to provide some links for how to handle json with R or python, for example:

https://cran.r-project.org/web/packages/jsonlite/vignettes/json-aaquickstart.html

5

Hi Guys,

Due to the size of these datasets, I would discourage you from using the base JSON handling machinery of Python or R. The process would not be performant. I would strongly suggest to use pyspark or an equivalent R library. Then join your dataset. If you give us some details on your use-case, we might be able to advise.