Nextflow ERROR : Timeout waiting for connection from pool
0
0
Entering edit mode
3 months ago
dzisis1986 ▴ 70

Hi it is my first time trying to use nf-core sarek pipeline in nextflow . I want to run only the annotation part with input vcf. I am getting the following error and I have no clue or idea how to solve it Do you know any solution or to what this ERROR is related ?

$ nextflow run nf-core/sarek -r 3.4.0 profile test --input AC1/vcf_single_AC1.csv --outdir testResutsAC1_new --step annotate --tools snpeff

ERROR ~ Error executing process > 'NFCORE_SAREK:SAREK:PREPARE_GENOME:UNZIP_RT'

Caused by:
  Timeout waiting for connection from pool


 -- Check '.nextflow.log' file for details
nextflow • 868 views
ADD COMMENT
0
Entering edit mode
ADD REPLY
0
Entering edit mode

Yes sorry for that but they didnt reply to that post

ADD REPLY
0
Entering edit mode

could be a temporary problem with AWS: https://github.com/nextflow-io/nextflow/issues/856

ADD REPLY
0
Entering edit mode

Have you looked in .nexfflow.log files for additional details as the suggestion above notes?

ADD REPLY
0
Entering edit mode

Yes this is the part with the error :

Mar-21 16:21:57.696 [S3TransferManager-49] WARN  c.a.s.s.i.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.
Mar-21 16:21:57.704 [Actor Thread 442] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=NFCORE_SAREK:SAREK:PREPARE_GENOME:UNZIP_RT; work-dir=null
  error [com.amazonaws.SdkClientException]: Unable to execute HTTP request: Timeout waiting for connection from pool
Mar-21 16:21:57.705 [S3TransferManager-39] WARN  c.a.s.s.i.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.
Mar-21 16:21:57.718 [S3TransferManager-31] WARN  c.a.s.s.i.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.
Mar-21 16:21:57.724 [Actor Thread 442] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_SAREK:SAREK:PREPARE_GENOME:UNZIP_RT'

Caused by:
  Timeout waiting for connection from pool

com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1219)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1165)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
ADD REPLY
0
Entering edit mode

Looks to be similar to the thread @pierre posted. Can you try the option indicated here: https://github.com/nextflow-io/nextflow/issues/856#issuecomment-1933259446

ADD REPLY
0
Entering edit mode

i will try it and let you know it can be something like that

ADD REPLY
0
Entering edit mode

It doesn't make sense how to change the threads and in my case it is not working! What kind of problem is that ? is it related to Amazon to the memory of my computer, to a number of threads, or to the internet connection? It is completely strange to me and I cant figure out how to solve it !

ADD REPLY
0
Entering edit mode

As best as I can tell it is a combination of things. Did you try setting export NXF_ENABLE_VIRTUAL_THREADS="true" in your shell? You also need to upgrade to Java 21.

nextflow has a slack channel where all their developers are active. You can join and post this problem there and/or link to this thread to see if they have a solution: https://nextflow.slack.com/signup#/domain-signup

ADD REPLY
0
Entering edit mode

Yes, I tried the export but itdoesn't work and I have Java 21 on my Mac! I think it's a combination of problems and whatever I tried doesn't work!

ADD REPLY

Login before adding your answer.

Traffic: 1823 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6