Question

Tophat Error : Oserror: [Errno 2] No Such File Or Directory

3

Entering edit mode

12.0 years ago

Sukhi Singh 11k

Hi,

I am running tophat and getting a lot of different errors, I could understand some of them, but this one, I don't know the reason. So, the error is

OSError: [Errno 2] No such file or directory: 'test_yo/tmp/left_kept_reads.m2g_um.candidates_and_unspl.bam'

This appears, after mapping when the reporting of output tracks is done. Command used :

tophat --bowtie1 -G /projects/grub/archive/Mus_musculus/UCSC/mm9/Annotation/Genes/genes.gtf -o test_yo /biodata/biodb/ABG/genomes/bowtie/mm9 test/test_data/reads_1.fq

Please let me know, if you encountered this or know a solution. Same question appears here but no answer yet.

My tophat.log :

[2012-04-16 18:49:10] Beginning TopHat run (v2.0.0)
-----------------------------------------------
[2012-04-16 18:49:10] Checking for Bowtie
          Bowtie version:     0.12.7.0
[2012-04-16 18:49:10] Checking for Samtools
        Samtools version:     0.1.17.0
[2012-04-16 18:49:10] Checking for Bowtie index files
[2012-04-16 18:49:10] Checking for reference FASTA file
    Warning: Could not find FASTA file /biodata/biodb/ABG/genomes/bowtie/biodata/biodb/ABG/genomes/bowtie/mm9.fa
[2012-04-16 18:49:10] Reconstituting reference FASTA file from Bowtie index
  Executing: /state/partition1/apps/bin/bowtie-inspect /biodata/biodb/ABG/genomes/bowtie/mm9 > test_yo/tmp/mm9.fa

[2012-04-16 18:51:12] Generating SAM header for /biodata/biodb/ABG/genomes/bowtie/mm9
    format:         fastq
    quality scale:     phred33 (default)
[2012-04-16 18:51:32] Reading known junctions from GTF file
[2012-04-16 18:51:35] Preparing reads
     left reads: min. length=75, count=100
[2012-04-16 18:51:35] Creating transcriptome data files..
[2012-04-16 18:51:50] Building Bowtie index from genes.fa
[2012-04-16 18:56:57] Mapping left_kept_reads against transcriptome genes with Bowtie 
[2012-04-16 18:56:58] Converting left_kept_reads.m2g to genomic coordinates (map2gtf)
[2012-04-16 18:57:00] Resuming TopHat pipeline with unmapped reads
[2012-04-16 18:57:00] Mapping left_kept_reads.m2g_um against mm9 with Bowtie 
[2012-04-16 18:57:01] Mapping left_kept_reads.m2g_um_seg1 against mm9 with Bowtie (1/3)
[2012-04-16 18:57:02] Mapping left_kept_reads.m2g_um_seg2 against mm9 with Bowtie (2/3)
[2012-04-16 18:57:03] Mapping left_kept_reads.m2g_um_seg3 against mm9 with Bowtie (3/3)
[2012-04-16 18:57:04] Searching for junctions via segment mapping
[2012-04-16 18:58:38] Retrieving sequences for splices
[2012-04-16 19:00:14] Indexing splices
[2012-04-16 19:00:33] Mapping left_kept_reads.m2g_um_seg1 against segment_juncs with Bowtie (1/3)
[2012-04-16 19:00:41] Mapping left_kept_reads.m2g_um_seg2 against segment_juncs with Bowtie (2/3)
[2012-04-16 19:00:49] Mapping left_kept_reads.m2g_um_seg3 against segment_juncs with Bowtie (3/3)
[2012-04-16 19:00:57] Joining segment hits
[2012-04-16 19:02:31] Reporting output tracks
Traceback (most recent call last):
  File "/share/apps/bin/tophat", line 3778, in ?
    sys.exit(main())
  File "/share/apps/bin/tophat", line 3754, in main
    os.remove(m)
OSError: [Errno 2] No such file or directory: 'test_yo/tmp/left_kept_reads.m2g_um.candidates_and_unspl.bam'

If you want run.log, I can also provide that.

I can reproduce the error using bowtie2 (default in tophat2) as well and the test data used is from the tophat website.

Cheers

tophat error rna-seq • 20k views

ADD COMMENT • link updated 23 months ago by Ram 43k • written 12.0 years ago by Sukhi Singh 11k

0

Entering edit mode

My run with tophat (v2.0.0 with bowtie2) coincides exactly with what your output shows, except that after reporting output tracks, its says run complete ... seconds elapsed. So, I don't think there is any issue with tophat run until there. Which OS are you working in? Did you try copying your fasta to bowtie-index folder (with same name as bowtie-index)...?

ADD REPLY • link 12.0 years ago by Arun 2.4k

2

Entering edit mode

Also try running with the --keep-tmp option (just a thought as your error seems to be from the tmp directory created by tophat).

ADD REPLY • link 12.0 years ago by Arun 2.4k

1

Entering edit mode

Amazing Arun --keep-tmp solves it :)

ADD REPLY • link 12.0 years ago by Sukhi Singh 11k

0

Entering edit mode

I'm glad that fixed it! :)

ADD REPLY • link 12.0 years ago by Arun 2.4k

0

Entering edit mode

Had the same issue and tophat didn't generate the accepted_hits.bam.. So I am running now with the --keep-tmp option! fingers crossed!

ADD REPLY • link 8.9 years ago by rodd ▴ 230

0

Entering edit mode

I am experiencing pretty much exactly this error. However, adding --keep-tmp did not solve the problem.

I am running tophat on a campus cluster machine with shared filesystem, and I wonder if the tmp files are getting removed for some reason related to that.

My command line:

tophat2 -p12 -o tophat-N3-ensembl -N3 --read-edit-dist=3 -G cf3ensembl.1.79.gtf --keep-tmp --transcriptome-index=cf3ensembl-transcriptome filtered.fastq 2> tophat-N3-err.txt

Output:

...
[2015-11-03 14:56:49] Joining segment hits
[2015-11-03 15:01:00] Reporting output tracks
    [FAILED]
Error: [Errno 2] No such file or directory: '/home/hekman2/scratch/transcriptome
/amygdala/aligned/tophat-N3-ensembl-530/tmp/accepted_hits0_sorted.bam'
Found 151163 junctions from happy spliced reads

Version: tophat2.1.0.

Interestingly, most of the files that I process complete just fine. About 1/6 of them have the above error. When I rerun them, they generally get the error again, although once or twice on the third try they succeeded. So maybe something to do with running out of memory, somehow?

Any suggestions would be appreciated!

Jessica

ADD REPLY • link updated 4.4 years ago by Ram 43k • written 8.5 years ago by jessica.hekman • 0

0

Entering edit mode

Yeah, its possible. Can you try to run it on a local machine.

I am unable to help in this regard as I am not sure what is the exact problem.

ADD REPLY • link updated 4.4 years ago by Ram 43k • written 8.5 years ago by Sukhi Singh 11k

0

Entering edit mode

I can't run it on a local machine until probably next week, but I'll try it and report back.

I did run it on the cluster with 8 instead of 12 processors. Result: all the files that had caused errors succeeded (yay). I concluded that I had solved the problem and moved on to the next set of files, using 8 processors each. Result: new error (though clearly similar):

[2015-11-04 14:15:09] Indexing splices
Building a SMALL index
Traceback (most recent call last):
  File "/scratch/users/hekman2/bin/tophat", line 4095, in <module>
  File "/scratch/users/hekman2/bin/tophat", line 4061, in main
  File "/scratch/users/hekman2/bin/tophat", line 3706, in spliced_alignment
  File "/scratch/users/hekman2/bin/tophat", line 1161, in nonzeroFile
  File "/usr/lib64/python2.6/subprocess.py", line 642, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1238, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

So, again, an error about a missing file.

I'll report back when I have a chance to try these on a local machine.

ADD REPLY • link updated 4.4 years ago by Ram 43k • written 8.5 years ago by jessica.hekman • 0

Ram · Answer 1 · 2012-04-17

So, to conclude the tophat errors for future reference from my experience

Error 1

Above mentioned error, can be solved using a parameter --keep-tmp. Thanks to Arun

Error 2

Searching for junctions via segment mapping

[FAILED] Error: segment-based junction search failed with err =127 The exact error cause can be seen in the segment.juncs file present in the tmp dir under your output dir but for me the probable cause was multithread problem. Tophat/Cufflinks are not MPI (Message Parsing protocol) programs, so they can't be distributed to more than one node. So, submitting to a single node with whatever cpu's it has max limit is the max it has solves it. Another reason, is exporting the LD lib path using `export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/apps/lib/` in your qsub or cluster execution script.

Also, there are logs stored in your tmp dir, so check for the last command in the run.log which is big and try running it on the console and you'll see where the error is. Same error happens with err=-9

Error 3

Searching for junctions via segment mapping
[FAILED]
Error: segment-based junction search failed with err =-6

Most likely reason is you run out of memory

Error 4

Bowtie problems, If you update Tophat to latest version i.e.2, be sure to update bowtie 1 to bowtie 2 which is called by default by tophat. If its not there, you'll get error and for using bowtie2, you should also fetch the bowtie2 indexes which are different than bowtie1.

Another error, occurs if your annotation file (genes.gtf) and the your bowtie index are from different sources (NCBI vs UCSC vs Ensembl), because they have a difference in how the chromosomes are named. Also, be sure your fastq is proper fastq else you get an execution error

Error 5

As the user RT reported,

[2014-01-14 00:24:01] Searching for junctions via segment mapping
[FAILED]
Error: segment-based junction search failed with err =127
/shared/apps/tophat/2.0.10/bin/segment_juncs: error while loading shared libraries: libboost_thread.so.1.51.0: cannot open shared object file: No such file or directory

This is because of absence or old version of Boost C++ libraries.

Copying steps from the website

••Building TopHat from source**

In order to build TopHat2 you must have the following installed on your system:

the Boost C++ libraries (we recommend version 1.47 or higher so you can use it for building Cufflinks as well)

Note: Starting with version 2.0.13 TopHat no longer requires the user to have the SAMtools library (libbam.a), headers or even the samtools program installed, as TopHat2 comes pre-packaged with a stable version of SAMtools which is known to work well with TopHat. Also, TopHat uses the SeqAn library that comes with the TopHat2 source code distribution. It is not necessary for the SeqAn library to be installed separately in order to run TopHat.

Building Boost

Download a recent Boost source tarball, unpack it and cd to the newly unpacked Boost source directory.
Prepare the build: ./bootstrap.sh
Build Boost. Note that you can specify where to install Boost with the --prefix option, which specifies the base path (prefix) where sub-directories ./include/ and ./lib/ will host the Boost headers and library files respectively. The default Boost installation directory prefix is /usr/local. Take note of this installation directory (if you specify your own) because you will need to provide it to the --with-boost option of TopHat's ./configure script. Run the build and install command: ./bjam --prefix=<YOUR_BOOST_INSTALL_DIRECTORY> link=static runtime-link=static stage install

I hope it helps someone.

Cheers