Error of SAM file

Question

[Pish]

I can not view the SAM file or convert SAM to BAM using samtools. As below;

user@computer:$ samtools view aln_orn5v2_4marker.sam | head
[W::sam_read1] Parse error at line 1
[main_samview] truncated file.

user@computer:$ samtools view -@ 32 -S -b aln_orn5v2_4marker.sam -t ./bwa_ref/4_marker_bwa.fasta > aln_orn5v2_4marker.bam
[W::sam_read1] Parse error at line 1
[main_samview] truncated file.

---

Then, I used sed to view error at line 1, as below:

user@computer:$ sed -n 1,19p aln_orn5v2_4marker.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 2135566 sequences (320000153 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 235, 1, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (25, 37, 88)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 214)
[M::mem_pestat] mean and std.dev: (54.88, 34.84)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 277)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 2135566 reads in 170.878 CPU sec, 8.467 real sec
@SQ SN:BettaMS10_1 LN:548 @SQ SN:BettaMS14_1 LN:504
@SQ SN:BettaMS2_2 LN:317
@SQ SN:BettaMS14_2 LN:350
@PG ID:bwa PN:bwa VN:0.7.17-r1188 CL:bwa mem -t 32 ./bwa_ref/4_marker_bwa.fasta ./../../ill_other_betta/orn5_1_pairend_trimmed.fastq ./../../ill_other_betta/orn5_2_pairend_trimmed.fastq
SRR18231396.1 77 0 0 0 0
ACGAGATCAAAGCCTTTCTAATTATTCCATTACTTGGTGAAAGACTACGCTTGTGACCCAACGGTGGGGGAGGGATGGAATACAGATCTGCCTGCAGCATTTACCACTTTAAGAGTTTTAGGAAAAGACAACCGGGTGTGTTGTGGTGGA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFF AS:i:0 XS:i:0 SRR18231396.1 141 0 0 0 0 GGTCTCCACCACAACACACCCGGTTGTCTTTTCCTAAAACTCTTAAAGTGGTAAATGCTGCAGGCAGATCTGTATTCCATCCCTCCCCCACCGTTGGGTCACAAGCGTAGTCTTTCACCAAGTAATGGAATAATTAGAAAGGCTTTGATC FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFF,FFFFFFFFFFFFFFF,FFFF:FFFFFFF:FFFF:FFFFFFFFFFFF AS:i:0 XS:i:0

Could you please suggest to me how to solve this problem?

Answer 1

[ATpoint]

The SAM seems to include some of the bwa stderr logs (this stuff starting in square brackets) please show the exact command that was used for it. Anyway, you have to repeat alignment.

[Pish]

This is a command that I used to align.

bwa mem -t 32 ./bwa_ref/4_marker_bwa.fasta ./../../ill_other_betta/orn5_1_pairend_trimmed.fastq ./../../ill_other_betta/orn5_2_pairend_trimmed.fastq > aln_orn5v2_4marker.sam

Anyway, this command was the second time of this process (the first time was successful), I used fastq files, reference file, and command same as the first time, but the result (sam file) can not view or convert to bam file.

The result of the first time;

user@computer:$ sed -n 1,19p aln_orn5_4marker.sam @SQ
SN:BettaMS10_1 LN:548
@SQ SN:BettaMS14_1 LN:504
@SQ SN:BettaMS2_2 LN:317
@SQ SN:BettaMS14_2 LN:350
@PG ID:bwa PN:bwa VN:0.7.17-r1188 CL:bwa mem -t 32 4_marker.fasta ./../../ill_other_betta/orn5_1_pairend_trimmed.fastq ./../../ill_other_betta/orn5_2_pairend_trimmed.fastq
SRR18231396.1 77 0 0 0 0
ACGAGATCAAAGCCTTTCTAATTATTCCATTACTTGGTGAAAGACTACGCTTGTGACCCAACGGTGGGGGAGGGATGGAATACAGATCTGCCTGCAGCATTTACCACTTTAAGAGTTTTAGGAAAAGACAACCGGGTGTGTTGTGGTGGA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFF AS:i:0 XS:i:0
SRR18231396.1 141 0 0 0 0 >GGTCTCCACCACAACACACCCGGTTGTCTTTTCCTAAAACTCTTAAAGTGGTAAATGCTGCAGGCAGATCTGTATTCCATCCCTCCCCCACCGTTGGGTCACAAGCGTAGTCTTTCACCAAGTAATGGAATAATTAGAAAGGCTTTGATC FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFF,FFFFFFFFFFFFFFF,FFFF:FFFFFFF:FFFF:FFFFFFFFFFFF AS:i:0 XS:i:0
SRR18231396.2 77 0 0 0 0 >GTGATGAGGTTTGTGCTGGTTCTCTGGTCCTCAAAGGCCACAGGTCTCCACCACAACACACCCGGTTGTCTTTTCCTAAAACTCTTAAAGTGGTAAATGCTGCAGGCAGATCTGTATTCCATCCCTCCCCCACCGTTGGGTCACAAGCG AAAAAEEEEEEEEEEAEEEEEEEEEEEEEAEE/EEEEEE6EEEEEEEEE/EE/EEAE/EAEEEEEEEEE/6EEE/AE/EE6EE/AE/EAEEEEEAEAEE/EE//EEAAE/EAEEE/AE///E/EEE<EEEE/AEA/EEEE<//A/<E/E AS:i:0 XS:i:0
SRR18231396.2 141 0 0 0 0 >AAGCCTTTCTAATTATTCCATTACTTGGTGAAAGACTACGCTTGTGACCCAACGGTGGGGGAGGGATGGAATACAGATCTGCCTGCAGCATTTACCACTTTAAGAGTTTTAGGAAAAGACAACCGGGTGTGTTGTGGTGGAGACCTGTGG A/AAAEEEEEEEEEEEEEEAEEAEEEEEEEEEEEAEEEEEEEEEEE/EEE/AEEEEEEEEEEEEEEEEEE6EEEEEEEEEEEEEEE6EEEEEEAEEEEEEEEEEEEEEEEEEEEE/EEEEE<EEEEEEEEEEEEEEEEEEAE<EEEEEEE AS:i:0 XS:i:0

[Pierre Lindenbaum]

did you use something like "nohup" to launch the job ?

[Pish]

I am sorry that I did not answer you to the point.

For the second time, yes, I did nohup.

nohup bwa mem -t 32 ./bwa_ref/4_marker_bwa.fasta ./../../ill_other_betta/orn5_1_pairend_trimmed.fastq ./../../ill_other_betta/orn5_2_pairend_trimmed.fastq > aln_orn5v2_4marker.sam &

And, for the first time, I did by

nohup bash file.sh &

the .sh file conian script as below,

!/bin/bash

 while read -r a b c; do                                                                                                                       
   bwa mem -t 32 4_marker.fasta ./../../ill_other_betta/${a} ./../../ill_other_betta/${b} > ${c}                                   
   done < bwa.txt

[Pierre Lindenbaum]

don't use nohup, it mixes the standard error stream and the stdout error stream.

use a Makefile, or a worflow manager like snakemake or nextflow.

[Pierre Lindenbaum]

in your script bwa.txt should be empty,

EDIT : oh, no I thought it was done > bwa.txt not done < bwa.txt. I call this "Yoda Programming", we usually see cat bwa.txt | read ...

[Pish]

Your suggestion was really useful. I'm really grateful for your help.