I am trying to download ~100 public amplicon sets from MG RAST I am using a slightly modified script from a previous entry here on BioStars (Downloading Data From Mg-Rast) that looks like this
cat names.csv | while read line; do curl http://api.metagenomics.anl.gov/1/download/"$line"?file=100.2 >$line.fna; done
names.csv is a list of the numbers of the files in which I am interested.
This will write all the files for the entries names.csv like it is supposed to, but each of those files only contain an error message:
<html> <head><title>400 Bad Request</title></head> <body bgcolor="white"> <center><h1>400 Bad Request</h1></center> <hr><center>nginx</center> </body> </html>
I can download the files if I replace "$line" with the actual number in my browser or if I run the command
curl http://api.metagenomics.anl.gov/1/download/metagenomenumber?file=100.2 > metagenomenumber.fna
from my terminal window, but I would much prefer to not have to do each one manually.
Any ideas on why my shell script may be failing?