How to download genome assemblies from NCBI with a list of GCA identifiers?
1
0
Entering edit mode
2.5 years ago
O.rka ▴ 290

I went to https://www.ncbi.nlm.nih.gov/assembly, typed in my organism, and now I want to download all of the assemblies that pop up. If I click [Download Assemblies] then it only downloads 1/22 of them and it's been saying "calculating size..." for about 30 minutes now. I tried using https://github.com/kblin/ncbi-genome-download but not all of the records were downloaded

Assembly data download ncbi • 2.9k views
ADD COMMENT
1
Entering edit mode

Are you sure you set the right filters on ngd, such as assembly level etc? It should download anything that’s present in the asssembly summary file.

ADD REPLY
0
Entering edit mode

Apparently the organism I wanted had all of its records in GenBank and not RefSeq

ADD REPLY
1
Entering edit mode

Yep, that is what I expected.

RefSeq is a subset of the total data in Genbank, that has been curated to a high degree manually. They are "reference sequences".

ADD REPLY
1
Entering edit mode

I'm not sure if you can rely on this all the time, but IIRC the accessions starting with "GCA_" are from GenBank. Accessions from RefSeq tend to start with "GCF_".

ADD REPLY
4
Entering edit mode
2.5 years ago
vkkodali ★ 2.5k

UPDATE 2020-11-21

If you have a list of NCBI assembly accessions with GCA or GCF prefixes, the easiest way to download data is to use the new tool NCBI Datasets. There is a web-interface to do this, if you don't want to bother with command line. If you do want to use the command line, you can use the datasets CLI tool as follows:

datasets download genome accession --inputfile assm_accs.txt --exclude-gff3 --exclude-protein --exclude-rna

where assm_accs.txt has NCBI assembly accessions, one per each line.

Note Currently, only the latest assembly accessions for a taxon are in the scope of NCBI Datasets. If you want to download older assemblies, you will have to use Entrez Direct as follows:

esearch -db assembly -query 'Bos taurus[organism] AND latest[filter]' \
    | esummary \
    | xtract -pattern DocumentSummary -element FtpPath_GenBank \
    | while read -r line ; 
    do
        fname=$(echo $line | grep -o 'GCA_.*' | sed 's/$/_genomic.fna.gz/') ;
        wget "$line/$fname" ;
    done

Here, I am first fetching the FTP path for the GenBank assembly using edirect tools and then use standard linux commands to download the genomic fasta file.

If your starting point is a file with a list of NCBI assembly accessions, you can wrap the command shown above in a bash loop like this:

cat assm_accs.txt | while read -r acc ; do
    esearch -db assembly -query $acc </dev/null \
        | esummary \
        | xtract -pattern DocumentSummary -element FtpPath_GenBank \
        | while read -r url ; do
            fname=$(echo $url | grep -o 'GCA_.*' | sed 's/$/_genomic.fna.gz/') ;
            wget "$url/$fname" ;
        done ;
    done
ADD COMMENT
0
Entering edit mode

Hi! Great solution!

Do you think that would be possible to extract the list of downloaded GCAs, ie to the GCA_list.txt file?

ADD REPLY
0
Entering edit mode

There are any way to read the query from a list.txt?

ADD REPLY
1
Entering edit mode

Yes, I updated my answer to include additional options.

ADD REPLY
0
Entering edit mode

thank you. It works very well

ADD REPLY

Login before adding your answer.

Traffic: 1889 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6