Tool:sra-explorer : find SRA and FastQ download URLs in a couple of clicks
0
34
Entering edit mode
2.7 years ago
Phil Ewels ▴ 740

Hi all,

As a fun little side project I've made a web tool to find runs on the NCBI Sequence Read Archive (SRA) and fetch the download URLs for these. You can do all of this in a couple of clicks, hopefully quite a bit easier than navigating the main SRA pages.

In addition to getting the SRA links the tool now queries the European Nucleotide Archive (ENA) for direct FastQ downloads and also provides these URLs.

All links are given as a simple list and also as a bash script which uses curl to download the data and gives nicer filenames that include the sample name in addition to its accession number.

You can find the tool here: https://ewels.github.io/sra-explorer/

It's open source and in fact just a single HTML file, built using AngularJS and Bootstrap. If you're curious, you can find the code here: https://github.com/ewels/sra-explorer

Phil

Edit ( 6.5.2020 by ATpoint ) sra-explorer also (for quite some time now) supports creation of Aspera download links. This enables downloads often faster than ftp. Setting up Aspera is simple, for a tutorial on Aspera installation and some background also see Fast download of FASTQ files from the European Nucleotide Archive (ENA)

3
Entering edit mode

URLs

bash script

If you prefer .sra downloads

3
Entering edit mode

Amazing! Works like a charm! Thanks so much!

0
Entering edit mode

Works great, very useful utility, thank you!

1
Entering edit mode

Very convenient, thank you, also for multiqc! You might consider adding an option to provide links for downloads driven by Aspera as in my tutorial Fast download of FASTQ files from the European Nucleotide Archive (ENA). That speeds up downloads notably.

1
Entering edit mode

Nice idea and great tutorial! I've just added this functionality - it would be great if you could have a look and make sure that I haven't made any errors.. Note that I added an option to append a mv command to rename the files. I've never used Aspera myself, so I hope this actually works!

1
Entering edit mode

Cool, works fine!

1
Entering edit mode

Is it possible that this tool is currently not working? Neither my own SRAs, nor just clicking on the examples provided is working. 😢 Thanks!

1
Entering edit mode

Yes - I’ve had a bunch of reports about problems over the past few days. I guess that an API endpoint changed somewhere. I’m on holiday at the moment but will take a look when I’m back. Apologies for the inconvenience!

1
Entering edit mode

Phil Ewels : Would it be possible add a note to sra-explorer main page that the tool is not currently working and you are going to fix it?

Edit: Main esearch call in sra-explorer seems to be working.

0
Entering edit mode

Not easily - I don’t have a computer with me. If I can edit code to add a notice on the web page then I can probably fix whatever the issue is :) I’ll be back home in a few days (though technically still on holiday) so will try to take a look then. If I can’t figure it out quickly then I’ll add a notice.

0
Entering edit mode

To follow - I managed to take a quick look in an evening over my holiday and it was not as trivial as I was hoping. I did manage to put up a notice though. I'll try to find some time to sit down with it in the next couple of weeks to do some code refactoring / rewriting and get it up and running again.

0
Entering edit mode

Much appreciated. Please let us know when you find a fix. This is the interface NCBI should have provided :-)

0
Entering edit mode

Nice tool...i used and works fine......👍🙏

0
Entering edit mode

Very convenient tool, much needed! Thanks a lot!

0
Entering edit mode

Is there a way to retrieve download links for an entire list of SRR numbers all at once?

0
Entering edit mode

You can download links for entire SRR numbers if you have searched for them already. It is not possible to search for multiple SRR# in the initial search box. You could use a wildcard like SRP0435* expands out to

srp043500[All Fields] OR srp043501[All Fields] OR srp043502[All Fields] OR srp043503[All Fields] ..... srp043591[All Fields] OR srp043592[All Fields] OR srp043593[All Fields] OR srp043594[All Fields] OR srp043597[All Fields] OR srp043598[All Fields] OR srp043599[All Fields]


But srp043500[All Fields] AND srp043503[All Fields] this search does NOT work. Phil Ewels can comment.

0
Entering edit mode

It just uses the SRA search - that style of search term doesn't return any hits there either, you can try it: https://www.ncbi.nlm.nih.gov/sra/?term=srp043500%5BAll+Fields%5D+AND+srp043503%5BAll+Fields%5D

Was it intentional to use AND instead of OR? This works in both the SRA and SRA-explorer:

srp043500[All Fields] OR srp043503[All Fields]


Also to clarify braun_tube - you can check multiple search results and add them to a collection and you'll get the entire list at once. If that's what you meant in your question.

0
Entering edit mode

Ah yes. Multiple searches could be added to the cart and then the cart downloaded at the end.

Phil Ewels I was just randomly trying two accessions and used AND. Using AND requires both to return results so OR would indeed be the way to go.

0
Entering edit mode

Using OR instead of AND between each SRR number in the search box fixed my problem and I got results for all my SRRs at once 👍

If neither OR or AND is specified in the search box then it does AND by default which is why I could not get multiple results at first. Thanks for the great tool.

0
Entering edit mode

ok, nice - good to know! :)