Question: Error in SAM to BAM conversion
0
gravatar for vivekr
5 weeks ago by
vivekr10
vivekr10 wrote:

I have used Bowtie2 to get alignment output as .SAM files. After getting sam files, I am getting the following errors:

[W::sam_read1] Parse error at line 459
[main_samview] truncated file.

I have used the following command to produce sam file:

./bowtie2 -p 5 -x /ref/grch38 --no-unal -U fastq/SRR7541164.fastq >  /fastq/test.sam

And for samtools :

samtools view -S -b /fastq/test.sam > /fastq/test.bam

I have checked .SAM file and fastq file to get the reason of error : few lines of both files are as follows: Command:

head -n 10 /home/vivekr/Mirpipe/moRNA_data/trimmed_fastq/SRR7541168_trimmed.fq

Output:

@SRR7541168.1 1 length=51
NCAGTGCACTACAGAACTTTGT
+
#0<FFFFFFFFFFIIIIIIIFI
@SRR7541168.2 2 length=51
NCCCTGTGGTCTAGTGGTTAGGATT
+
#0<BFFFFBFFFFIFFBFFFF<BFF
@SRR7541168.3 3 length=51
NATTGCACTTGTCCCGGCCTGT

And for SAM files :

Command:

head -n 460 /home/vivekr/Mirpipe/moRNA_data/sorted_bam/SRR7541164_trimmed.sam | tail -n 20

Response:

@SQ SN:chrUn_KI270749v1 LN:158759
@SQ SN:chrUn_KI270750v1 LN:148850
@SQ SN:chrUn_KI270751v1 LN:150742
@SQ SN:chrUn_KI270752v1 LN:27745
@SQ SN:chrUn_KI270753v1 LN:62944
@SQ SN:chrUn_KI270754v1 LN:40191
@SQ SN:chrUn_KI270755v1 LN:36723
@SQ SN:chrUn_KI270756v1 LN:79590
@SQ SN:chrUn_KI270757v1 LN:71251
@SQ SN:chrUn_GL000214v1 LN:137718
@SQ SN:chrUn_KI270742v1 LN:186739
@SQ SN:chrUn_GL000216v2 LN:176608
@SQ SN:chrUn_GL000218v1 LN:161147
@SQ SN:chrX LN:156040895
@SQ SN:chrY LN:57227415
@SQ SN:chrY_KI270740v1_random   LN:37240
@PG ID:bowtie2  PN:bowtie2  VN:2.3.5.1  CL:"/home/vivekr/Mirpipe/Packages1
/bowtie2-2.3.5.1-sra-linux-x86_64/bowtie2-align-s --wrapper basic-0 -p 8 -x /home/vivekr/Mirpipe
/ref/grch38/hg38 /home/vivekr/Mirpipe/moRNA_data/sorted_bam/SRR7541164_trimmed.sam --passthrough -U /home/vivekr/Mirpipe/moRNA_data/trimmed_fastq/SRR7541164_trimmed.fq"
SRR7541164.1    16  chr7    25949921    42  23M *   0   0   GACAAAGTTCTGTAGTGCACTGN IIIIIIIIIIFFFFFFFFFF<0# AS:i:-1 XN:i:0  XM:i:1  
XO:i:0  XG:i:0  NM:i:1  MD:Z:22A0   YT:Z:UU
@SRR7541164.1 1 length=51%0ANCAGTGCACTACAGAACTTTGTC%0A+%0A#0<FFFFFFFFFFIIIIIIIIII%0A
SRR7541164.2    16  chr7    25949923    42  21M *   0   0   CAAAGTTCTGTAGTGCACTGN   IIIIIIIFFFFFFFFFFF<0#   AS:i:-1 XN:i:0  XM:i:1  XO:i:0  XG:i:0  NM:i:1  MD:Z:20A0   YT:Z:UU

Version details are as follows:

samtools 1.7

Using htslib 1.7-2

Copyright (C) 2018 Genome Research Ltd.

I am not able to check what is causing sam file curruption or error while conversion from sam to bam. Anu sugesstions please....

Thanks.

alignment • 159 views
ADD COMMENTlink modified 5 weeks ago • written 5 weeks ago by vivekr10
1

Hello ,

./bowtie2 -p 5 -x /ref/grch38 --no-unal -U fastq/SRR7541164.fastq > -S /fastq/test.sam

Just a typo here or do you realy redirecte bowtie output to -S?

What's the output of tail /fastq/test.sam?

fin swimmer

ADD REPLYlink written 5 weeks ago by finswimmer11k

No. I use this -S to get output as sam file. The output of tail /fastq/test.sam is as follows:

@SRR7541164.20452273 20452273 length=51%0ACAAAGAATTCTCCTTTTGGGCTGGAATTCTCGGGTGCCAAGGAACTCCAGT%0A+SRR7541164.20452273 20452273 length=51%0ABBBFFFFFFFFFFFFIIIIIIIIIBFFIIIIIIIIIIIIIIIIIIIIIFII%0A SRR7541164.20452274 4 * 0 0 * * 0 0 TCAGTGCACTACAGAACTTTGTCTGGAATTCTCGGGTGCCAAGGAACTCCA BBBFFFFFFFFFFIIIIIIIIIIIIIIFIIIIIIIIIIIIIIIIIIIIFII YT:Z:UU

@SRR7541164.20452274 20452274 length=51%0ATCAGTGCACTACAGAACTTTGTCTGGAATTCTCGGGTGCCAAGGAACTCCA%0A+SRR7541164.20452274 20452274 length=51%0ABBBFFFFFFFFFFIIIIIIIIIIIIIIFIIIIIIIIIIIIIIIIIIIIFII%0A SRR7541164.20452275 4 * 0 0 * * 0 0 GTTTCCGTAGTGTAGTGGTTATCACGTTCGCCTTGGAATTCTCGGGTGCCA BBBFFFFFFFFFFFIFIIIIIIIIFIIIIIIIIIIIIIIIIIIIIIFFBFF YT:Z:UU

@SRR7541164.20452275 20452275 length=51%0AGTTTCCGTAGTGTAGTGGTTATCACGTTCGCCTTGGAATTCTCGGGTGCCA%0A+SRR7541164.20452275 20452275 length=51%0ABBBFFFFFFFFFFFIFIIIIIIIIFIIIIIIIIIIIIIIIIIIIIIFFBFF%0A SRR7541164.20452276 4 * 0 0 * * 0 0 CTGTCTGAGCGTCGCTTGGAATTCTCGGGTGCCAAGGAACTCCAGTCACAT BBBFFFFFFFFFFIIIIIIIIIIIIIIIIFFFIIFFFIIIIIIIIIIBFFI YT:Z:UU

@SRR7541164.20452276 20452276 length=51%0ACTGTCTGAGCGTCGCTTGGAATTCTCGGGTGCCAAGGAACTCCAGTCACAT%0A+SRR7541164.20452276 20452276 length=51%0ABBBFFFFFFFFFFIIIIIIIIIIIIIIIIFFFIIFFFIIIIIIIIIIBFFI%0A SRR7541164.20452277 4 * 0 0 * * 0 0 AGCTACATTGTCTGCTGGGTTTTGGAATTCTCGGGTGCCAAGGAACTCCAG BBBFFFFFFFFFFIIIIIIFFFIIIIIIIIIIIIIIIIIIIIIIIIIIIII YT:Z:UU

@SRR7541164.20452277 20452277 length=51%0AAGCTACATTGTCTGCTGGGTTTTGGAATTCTCGGGTGCCAAGGAACTCCAG%0A+SRR7541164.20452277 20452277 length=51%0ABBBFFFFFFFFFFIIIIIIFFFIIIIIIIIIIIIIIIIIIIIIIIIIIIII%0A

ADD REPLYlink written 5 weeks ago by vivekr10
2

That last line is truncated, suggesting that your alignment got aborted prematurely.

Note that do sam-to-bam conversion on the fly without using a sam intermediate:

bowtie2 -x genome -U reads.fastq.gz | samtools sort -o alignment.bam
ADD REPLYlink written 5 weeks ago by WouterDeCoster40k

You cannot use -S and > at the same time, so either you made a mistake or you gave us the wrong command.

ADD REPLYlink written 5 weeks ago by WouterDeCoster40k
1

I made a mistake. Definitely -S and > can't used together. Thanks for pointing it out. I have also checked and confirmed this with botiew2 manuals. I have edited the question also.

ADD REPLYlink modified 5 weeks ago • written 5 weeks ago by vivekr10

Or sometimes check md5sum, if fastq files are downloaded correctly..

ADD REPLYlink written 5 weeks ago by Korsocius110
1
gravatar for ATpoint
5 weeks ago by
ATpoint19k
Germany
ATpoint19k wrote:

As this is a common problem, here a quick summary:

  • Check that the output path is correct: /fastq/test.sam indicates writing to a root directory
  • do not store SAM files, use samtools view directly to get BAM: bowtie2 (...) | samtools view -o out.bam
  • output seems corrupted, rerun the entire alignment with the given suggestions.
ADD COMMENTlink written 5 weeks ago by ATpoint19k
0
gravatar for vivekr
5 weeks ago by
vivekr10
vivekr10 wrote:

It seems my issue is solved. Let me explain how I was getting fastq file and why I was getting error:

I need to do custom filtration in original fastq file. For that I have used HTSeq package available in bioconda. After that I need to generate bam file for each sample. So there can be two possible reason for getting surrupted sam file:

  1. Re-generated fastq file may not be written as proper/ desired manner which leads to currupted sam file and cause error in sam to bam conversion.

  2. Thanks to @WouterDeCoster to point out error in my command. I don't know why many times I used -S and > in same command and never cause erro but this time I got error form which I got to know about this.

How I solved this:

Instead to using bioconda, I used bbduk.sh and get correct fastq file and also with right command, I am able to get BAM file for all samples. Thanks to all for support.

Although, I need one more custom filtration (remove reads having at most 2 bases under quality score 20 and remove reads with unique sequence having read count less than 10 and I did not find any tool for those task till now. Please give any suggestions if I missed anything....) and I am looking for proper way to do this. For time being, I am moving forward with fastq obtained after bbduk.sh.

ADD COMMENTlink modified 5 weeks ago • written 5 weeks ago by vivekr10
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1261 users visited in the last hour