Ensembl FTP Mirror in USA? Slow Downloads
0
0
Entering edit mode
13 months ago

Hello I work on a fairly good cluster at an R1 university in the USA. I am downloading the VEP 101 indexed cache files and it's slow as Christmas. I mean AOL speeds slow.

If I download a similar size file from AWS or NCBI it's extremely fast.

Is there a mirror for the USA containing the VEP cache files. I would only need hg38 merged VEP 101.

10 hours to download a 17GB file at 268KB/s

Give me a break, I get you Europeans don't like the USA but come on guys ;)

ensembl vep • 509 views
ADD COMMENT
1
Entering edit mode

Perhaps the problem is local? Is your firewall stepping in? I just tried the download.

$ curl -O ftp://ftp.ensembl.org/pub/release-101/variation/indexed_vep_cache/homo_sapiens_vep_101_GRCh38.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 12 14.2G   12 1882M    0     0  32.4M      0  0:07:30  0:00:58  0:06:32 33.9M
ADD REPLY
0
Entering edit mode
$ curl -O ftp://ftp.ensembl.org/pub/release-101/variation/indexed_vep_cache/homo_sapiens_merged_vep_101_GRCh38.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0 16.4G    0 13.8M    0     0   733k      0  6:31:51  0:00:19  6:31:32  690k

Not sure, I am on a dedicated data mover node and can obtain 10x speeds on NCBI FTPs and AWS buckets. So I am not sure why it's painfully slow for Ensembl. Thanks!

ADD REPLY
1
Entering edit mode

AFAIK Ensembl ftp site has no mirrors (unlike main Ensembl site which does). May want to check with your local networking folks to see if they can shed some light. On east coast no problems.

ADD REPLY
0
Entering edit mode

Thanks I submitted a ticket. Hopefully they will clear it up.

ADD REPLY

Login before adding your answer.

Traffic: 3011 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6