Question: samtools: writing to standard output failed: Broken pipe
1
gravatar for mcff23
5.0 years ago by
mcff2360
mcff2360 wrote:

Hi everyone!

I'm getting this error from tophat:

samtools: writing to standard output failed: Broken pipe

Log file:
[2014-09-15 16:31:23] Beginning TopHat run (v2.0.12)
-----------------------------------------------
[2014-09-15 16:31:23] Checking for Bowtie
                  Bowtie version:        2.2.3.0
[2014-09-15 16:31:23] Checking for Samtools
[2014-09-15 16:31:23] Checking for Bowtie index files (genome)..
[2014-09-15 16:31:23] Checking for reference FASTA file
[2014-09-15 16:31:23] Generating SAM header for 
[2014-09-15 16:31:23] Reading known junctions from GTF file
[2014-09-15 16:31:27] Preparing reads
         left reads: min. length=101, max. length=101, 99994 kept reads (6 discarded)
        right reads: min. length=101, max. length=101, 99455 kept reads (545 discarded)
[2014-09-15 16:31:32] Building transcriptome data files test_LC1.THout/tmp/Lj2.5_gene_models
[2014-09-15 16:31:38] Building Bowtie index from Lj2.5_gene_models.fa
[2014-09-15 16:32:07] Mapping left_kept_reads to transcriptome Lj2.5_gene_models with Bowtie2 
[2014-09-15 16:32:13] Mapping right_kept_reads to transcriptome Lj2.5_gene_models with Bowtie2 
[2014-09-15 16:32:18] Resuming TopHat pipeline with unmapped reads
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:18] Mapping left_kept_reads.m2g_um to genome Bowtie2Index with Bowtie2 
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:28] Mapping left_kept_reads.m2g_um_seg1 to genome Bowtie2Index with Bowtie2 (1/4)
[2014-09-15 16:32:31] Mapping left_kept_reads.m2g_um_seg2 to genome Bowtie2Index with Bowtie2 (2/4)
[2014-09-15 16:32:33] Mapping left_kept_reads.m2g_um_seg3 to genome Bowtie2Index with Bowtie2 (3/4)
[2014-09-15 16:32:36] Mapping left_kept_reads.m2g_um_seg4 to genome Bowtie2Index with Bowtie2 (4/4)
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:41] Mapping right_kept_reads.m2g_um to genome Bowtie2Index with Bowtie2 
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:50] Mapping right_kept_reads.m2g_um_seg1 to genome Bowtie2Index with Bowtie2 (1/4)
[2014-09-15 16:32:53] Mapping right_kept_reads.m2g_um_seg2 to genome Bowtie2Index with Bowtie2 (2/4)
[2014-09-15 16:32:56] Mapping right_kept_reads.m2g_um_seg3 to genome Bowtie2Index with Bowtie2 (3/4)
[2014-09-15 16:32:59] Mapping right_kept_reads.m2g_um_seg4 to genome Bowtie2Index with Bowtie2 (4/4)
[2014-09-15 16:33:03] Searching for junctions via segment mapping
[2014-09-15 16:33:18] Retrieving sequences for splices
[2014-09-15 16:33:31] Indexing splices
Building a SMALL index
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:33:36] Mapping left_kept_reads.m2g_um_seg1 to genome segment_juncs with Bowtie2 (1/4)
[2014-09-15 16:33:38] Mapping left_kept_reads.m2g_um_seg2 to genome segment_juncs with Bowtie2 (2/4)
[2014-09-15 16:33:39] Mapping left_kept_reads.m2g_um_seg3 to genome segment_juncs with Bowtie2 (3/4)
[2014-09-15 16:33:41] Mapping left_kept_reads.m2g_um_seg4 to genome segment_juncs with Bowtie2 (4/4)
[2014-09-15 16:33:42] Joining segment hits
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
samtools: writing to standard output failed: Broken pipe

 

and so on..

I couldn't find any specific solution...

Thanks in advance!

MF

rna-seq • 9.6k views
ADD COMMENTlink modified 3.5 years ago by Shicheng Guo7.7k • written 5.0 years ago by mcff2360
3

I think that I have tracked the issue down to the following code in tophat.py:

samtools_view_cmd = ["samtools", "view", filepath]
samtools_view = subprocess.Popen(samtools_view_cmd, stdout=subprocess.PIPE)
head_cmd = ["head", "-1"]
head = subprocess.Popen(head_cmd, stdin=samtools_view.stdout, stdout=subprocess.PIPE)
samtools_view.stdout.close() # as per http://bugs.python.org/issue7678
output = head.communicate()[0][:-1]

The issue appears to be that python's handling of SIGPIPEs in this kind of pipeline is nonstandard - see http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html - and that the newer version of samtools cares about this, while the old version didn't.

Installing the backport of python3k's subprocess at http://code.google.com/p/python-subprocess32/ and changing the above code to use this version of subprocess gets rid of the error messages.

 

ADD REPLYlink modified 4.9 years ago • written 4.9 years ago by Jeff Hussmann120

could you put your tophat command you used ? Do you have the write permission in the tophat output directory ? And also samtools version? thanks

ADD REPLYlink modified 5.0 years ago • written 5.0 years ago by Nicolas Rosewick8.1k

HI NicoBxl!

command:

tophat -p 8 -G annotation -o sample.THout bowtie2_index R1.fastq.gz R2.fastq.gz

Yes I have write permissions in the output directory. The samtools version is the latest 1.0

Thanks!

MF

 

ADD REPLYlink modified 4.9 years ago by RamRS24k • written 5.0 years ago by mcff2360

How much memory do you have on the computer running this? Also, have you tried an older version of samtools? Tophat2 doesn't natively support the most recent version.

ADD REPLYlink written 5.0 years ago by Devon Ryan92k

I have 32 Gb of RAM and the 80% is free.

I patched (https://groups.google.com/forum/#!topic/tuxedo-tools-users/BVNDDmInU0A) tophat for samtools 1.0 version support. 

ADD REPLYlink written 5.0 years ago by mcff2360
2

With that much RAM you'd be better off using STAR. Tophat is painfully slow and mildly annoying to debug.

ADD REPLYlink written 5.0 years ago by Devon Ryan92k
1

i would not muck around with unofficial patches, so many things can also break, just use the supported samtools until the main trunk supports it natively 

invoke the new samtools separately in your own work

ADD REPLYlink modified 5.0 years ago • written 5.0 years ago by Istvan Albert ♦♦ 81k

Check that your disk-space isn't close to full

ADD REPLYlink written 5.0 years ago by spiderdijon20

When I submit Bismark jobs in PBS system, I did get some similar error report. However, every result have been successfully obtained. Even that the error log file will show you:

bismark_methylation_extractor --single-end --bedGraph --cutoff 5 --multicore 3 --ignore 3 --ignore_3prime 3 --gzip --buffer_size 4G --zero_based --comprehensive --output ../methyfreq/ CTR154_trimmed.fq.gz_bismark_bt2.bam
Finished BedGraph conversion ...
gzip: stdout: Broken pipe

Don't care about them. I think it is the problem of PBS system.

ADD REPLYlink modified 3.5 years ago • written 3.5 years ago by Shicheng Guo7.7k
1

the same thing happened to me as well. This is the end of fhe PBS output file. but I got the coverage file. Finished generating genome-wide cytosine report

samtools view: writing to standard output failed: Broken pipe samtools view: error closing standard output: -1

gzip: stdout: Broken pipe

ADD REPLYlink written 22 months ago by cjgunase30
3
gravatar for mcff23
5.0 years ago by
mcff2360
mcff2360 wrote:

The solution for this error was using an older version of samtools like Devon Ryan said. I though the patch was working fine and it wasn't.

With Samtools 0.1.19.0 it worked just fine!!

Thanks a lot guys!

 

 

ADD COMMENTlink written 5.0 years ago by mcff2360
2

This error may also happen when using bam2cfg.pl in BreakDancer. 

The solution was the same: use an older version.

ADD REPLYlink written 5.0 years ago by xieshaojun0621130
1

Thanks for reporting back and also mentioning this on the tophat forum.

ADD REPLYlink written 5.0 years ago by Devon Ryan92k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 976 users visited in the last hour