Question: Error while downloading data from database[EGA]
2
gravatar for banerjeeshayantan
2.3 years ago by
banerjeeshayantan170 wrote:

I have access to a particular dataset in EGA. I am trying to download it. But I always get cut off from the server while downloading. The file size is 121.7 GB. Below is the error I am getting. (The error occurs after downloading 35GB)

Starting transfer loop for 08760ef0-82f9-4431-ac96-37f72843acf2 (true)

Download Error occurred at Stage 4
Error: Read timed out
Downloaded :
Error requesting this!


Can someone please point out the problem?

sequencing next-gen • 2.4k views
ADD COMMENTlink modified 22 months ago by miaowzai230 • written 2.3 years ago by banerjeeshayantan170

We are having similar issue with downloads failing after a prolonged wait time in downloading data from EGA servers. Using EGA Download Client and following command to request and download data#### Command

#Request
java -jar /apps/ega/download/default.jar -pf ega_token.txt -rf EGAF00000406021 -re passkey -label req_EGAF00000406021

#Download
java -jar /apps/ega/download/default.jar -pf ega_token.txt -dr req_EGAF00000406021 -nt 8

Error log

Files to download in this request: 5
Start Download Process: 4 (max:15) parallel threads
Iteration 1: 5 files.
Starting download: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg  (0/5)
Starting download: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg  (1/5)
Starting download: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg  (2/5)
Starting download: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg  (3/5)
Download Active: 5 jobs submitted, 0 completed.

Output: Ticket: b1d744f6-a180-4892-948b-a7c2a8de9f49
URL: http://ega.ebi.ac.uk/ega/rest/ds/v2/downloads/b1d744f6-a180-4892-948b-a7c2a8de9f49
Establishing Data Stream for: b1d744f6-a180-4892-948b-a7c2a8de9f49
Data Stream for b1d744f6-a180-4892-948b-a7c2a8de9f49 is established
Path: /gpfs/ctgs0/fastscratch/kimh/tmp/Costello/EGAD00001000714/_EGAZ00001016120_D0LA3ACXX_2.bam.cip
Starting transfer loop for b1d744f6-a180-4892-948b-a7c2a8de9f49 (true)

Download Error occurred at Stage 4
Error: Read timed out
Completed download: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg: false  (1/5)
Download Failed: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg  b1d744f6-a180-4892-948b-a7c2a8de9f49
Download Failed: /EGAZ00001016120/D0LA3ACXX_2.bam.gpg  2c54b30d-369c-4738-b18b-560ecd3e1373
ADD REPLYlink written 2.2 years ago by Samir180

Both issues may have been better directed to the EGA HelpDesk, who, from my experience, respond rapidly. Be wary, however, that in the UK, next Monday (Monday 7th May 2018) is a public holiday.

ADD REPLYlink written 2.2 years ago by Kevin Blighe61k

I'm having the same problem. Any solutions? I did contact EGA help desk and they said that their client had some issues which has been fixed and is up and running. But I still seem to have this problem.

ADD REPLYlink written 2.1 years ago by 29aprilkrutika10

If EGA support are sure that no problems exist on their end then you may want to talk with your local IT to see if there are any firewall restrictions that are preventing you from downloading.

ADD REPLYlink modified 2.1 years ago • written 2.1 years ago by genomax85k

I'm using nohup in an ec2 instance on AWS to download the data. Can there be any issue with that?

ADD REPLYlink written 2.1 years ago by 29aprilkrutika10

I am having the exact same problem so I still haven't finished downloading the files. A few weeks ago, I had been contacting with the EGA staff for at least 20 emails and they were saying that they had a server issue.

Need some solutions too!

ADD REPLYlink modified 23 months ago • written 23 months ago by miaowzai230
1
gravatar for 29aprilkrutika
23 months ago by
29aprilkrutika10 wrote:

I requested EGA to create a aspera account for me with the dataset, it was easy enough to download the data using their download.sh script. You will need to ask for a decryption key for the data as well.

ADD COMMENTlink written 23 months ago by 29aprilkrutika10
1
gravatar for miaowzai
22 months ago by
miaowzai230
United States
miaowzai230 wrote:

I have been having this problem for a long time and now it is solved by using their python3 client. Try it here: https://ega-archive.org/download/downloader-quickguide-v3#downloadClient

I am not entirely sure why the same downloading works with the python3 client but not with the other methods though... (the other methods always show a server error on their side) I believe requesting an aspera downloading works too!

ADD COMMENTlink modified 9 months ago • written 22 months ago by miaowzai230
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1371 users visited in the last hour