Downloading All Cshl Long Rna Seq Data From Ucsc Ftp Server

Question

[disco]

Hello there,

I'm trying to download all the bam & bam.bai files of ENCODE's CSHL long RNA seq data using Perl from the UCSC FTP server here

I tried downloading one cell line at a time initially, but its too painstaking.. If anyone could please suggest a way or if anybody already has got a Perl script for this, could you please share? I'd be umpty grateful!

Best, Vaish

Answer 1

[biorepine]

Firefox download all is a great tip if you want to download them to your desktop. But if you want to download them in a server you need something else. There are many ways to do this and most of them are not very straight forward. I always use this simple trick to handle UCSC data downloads.

1. go to the folder
2. copy all the file names not the files
3. paste them in a text file and do this ..
4. ls -l | grep bam | awk '{print "wget "$9" ;"}' >> download.sh
5. now run sh download.sh
6. replace ";" with "&" if you want to do simultaneous downloads.

Simple isn't it ;)

[disco]

Thanks for the response. Please pardon my naivety, but what exactly do you mean by downloading in a server?

[Steve Lianoglou]

You might want to download these BAM files to a different computer than you are sitting at. For instance, you might be on a laptop, but you have some "big iron" machine you wan to use to process the data sitting somewhere else that you might be ssh'd into.

[disco]

Thanks for clearing that up. I would go for the more straightforward download them and all sundry option. I'm just stepping into the informatics world by learning one tech word a time.

[disco]

Is this fast? Because I tried the download all option, It was going to take a long time to download. Its a great way to do it, but if this method is faster, I would like to switch this..

Answer 2

[Steve Lianoglou]

If you're not married to the idea of scripting this yourself, and easy way to do this is to use Firefox with the DownThemAll plugin.

1. Navigate to the ENCDOE CSHL Long RNA Seq page
2. Go to Tools > DownThemAll Tools> DownThemAll and enter *.bam,*.bai in the Fast Filtering text box at the bottom of the window that pops up.
3. Pick the appropriate folder you wan to save these fils in locally using the Save files in text box
4. Click the Start! button
5. ...
6. Publish!

[disco]

This is so straightforward! Wonder why I didn't come across this solution anywhere before! But please hold on, I have another question, is it possible to run all the bam files together on samtools? followed by cuffdiff on all of them?

[Steve Lianoglou]

Please submit this as a new question to biostar and add some more detail to give us a better idea of what you are asking. For instane: I have no idea what you want to do when you say "run all the bam files together on samtools" -- doesn't make sense to me

[disco]

I'm sorry, pardon my ignorance. Now I understand that the input for Cuffdiff is a SAM file and not a gtf. I will post a more proper question..

[disco]

Here's the one- Samtools And Cuffdiff