Impossible Slowness And Download Cancellation When Accessing Ensembl Biomart

Question

[User 9996]

I am trying to download a simple gene table from Ensembl's biomart interface. Whenever I do so, the download is extremely slow even though I am on a T1 connection (~50-300 kb per sec, though it oscillates) and in virtually all cases, the file I am trying to download gets prematurely cancelled by the server. This happens if I try to download a .gz or a .txt. Any idea what might be happening here? When I use the feature of having ensembl email me the results, it never does.

How can this problem be solved? thanks.

Answer 1

[Joachim]

It might be better to ask support questions on the BioMart mailing list: [e-mail]

Or, you could contact the Ensembl helpdesk: [e-mail]

In either case, please be more specific about the query that you are running. Especially, say which mart you are querying, state the attributes you query and make clear which filters you have possibly set. Any listing of only the genes, may it be by Ensembl Gene ID, Entrez Gene ID or Associated Gene Name, is so tiny that it should even open in your browser window.

Answer 2

[Giulietta - Ensembl Helpdesk]

Hi (from the Ensembl helpdesk),

We do get this complaint a lot. If the query is too large, BioMart times out, and you never get the full result table. To get around this, I would advise applying a filter (for example, filter by chromosome) and do multiple queries. Also, 'compressed web file, notify by email' is the safest option for getting a full result set. (If you don't get anything back within an hour, the result set is too large).

Giulietta ([e-mail])

Answer 3

[biobot 0.0.77.a.1099]

Are you choosing to export the results as 'compressed file' or 'compressed web download'? The latter is best for large queries because it creates the results on the server and sends an email notification on completion, without holding an HTTP connection open.

I'm guessing that it is an HTTP timeout you are seeing, rather than a query timeout from the MySQL backend. However, query timeouts are possible too. Without knowing what you did, or what the error message was, I can't help any further.