Gatk Indel Realignment Io Timeout?

Question

[Zev.Kronenberg]

Greetings,

I am running GATK on a cluster. I am using '--num_threads 4' (although I don't even know if this works for indel realignment). It looks like I running into an I/O bottleneck. Has anyone ever experienced this problem?

The obvious answer is to run using less threads and launching only a couple jobs at a time. Is there another solution? I would rather batch everything at once.

error:

INFO  12:59:56,053 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Connection timed out 
INFO  12:59:56,054 HttpMethodDirector - Retrying request

Answer 1

[DG]

The IndelRealigner doesn't support -nt for multi-threading IIRC (I don't have that option in my command-line in my exome alignment pipeline) so I am guessing that is your problem. There may be others. I would try running your intended command not through a submission queue so you can see if it throws any other potential errors.

[Zev.Kronenberg]

i am pretty sure it is ignoring --num_threads. I will remove it anyway. I don't want to try it outside the submission queue as this is the world I am confined to. Interactive mode won't give me a more robust error message.

Answer 2

[Zev.Kronenberg]

GATK calls home to report to the borg collective. My jobs are finishing, but my data isn't being sent off to GATK. This is generating the errors.

[matted]

Interesting, I didn't know about this "feature." Just curious, was it a firewall on your side blocking this?

For reference, you can read more here:

http://gatkforums.broadinstitute.org/discussion/1250/what-is-phone-home-and-how-does-it-affect-me

In later versions, you have to request a key to disable this (!!).

[Zev.Kronenberg]

You have to request the key using an academic email address as well. It wasn't a firewall issue. On the cluster when a job is submitted it does not have access to the internet.

[Zev.Kronenberg]

I though I was allowed to accept my own answer?