How to bulk download from ENCODE data with an inner query which is not present in the options

Question

[alexmondaini]

Hello everyone,

I am trying to bulk download from ENCODE using this specific query: ENCODE query

The problem is that if I follow the instructions and bulk download using their recommended command after saving locally file.txt : xargs -L 1 curl -O -J -L < files.txt I don't get the desired files I want. If I click into any of the results like for example this one: https://www.encodeproject.org/experiments/ENCSR018WPY/ and click again in file details I see 3 types of bed narrowPeak files. one type is isogenic replicate 1 the other is isogenic replicate 2 and the other is isogenic replicate 1,2.

I really would like to obtain in bulk isogenic replicate 1,2 but unfortunately the bulk query downloads only 1 or 2.

Answer 1

[jerry]

Hi Alex, I struggled a lot before trying to find and download files from ENCODE. xargs + file.txt is one way, but yeah, it sometimes doesn't work.

I ended up creating a tool for this as part of a bioinformatics suite called BioShed (https://www.bioshed.io).

Reach out to me - I'd be happy to help. Thanks!