Question: Problem Converting BAM to SAM on to Amazon-S3 Using Samtools
gravatar for gundalav
3.6 years ago by
La La Land
gundalav300 wrote:

In my Amazon EC2 instance I have the following:

ubuntu@ip-333-31-16-230:~$ df -h
Filesystem       Size  Used Avail Use% Mounted on
udev              79G     0   79G   0% /dev
tmpfs             16G  8.8M   16G   1% /run
/dev/xvda1       985G  136G  809G  15% /
tmpfs             79G     0   79G   0% /dev/shm
tmpfs            5.0M     0  5.0M   0% /run/lock
tmpfs             79G     0   79G   0% /sys/fs/cgroup
tmpfs             16G     0   16G   0% /run/user/1000
bar-something  1.0P     0  1.0P   0% /home/ubuntu/bar-something-data
bar-foolishbrat  1.0P     0  1.0P   0% /home/ubuntu/bar-foolisbrat-data

The last two file system bar-bi and bar-foolishbrat are Amazon S3 bucket, mounted with the following command:

/home/ubuntu/go/bin/goofys bar-foolishbrat bar-foolishbrat-data
/home/ubuntu/go/bin/goofys bar-somehting bar-something-data
# Here we use Goofys. It can be achieved also with S3FS.

Now I have no problem doing this, in my instance

samtools view -h ~/bar-something-data/myfile.bam > test.sam

That is writing the SAM file in my instance file system. But when I do this, namely writing to the mounted S3:

samtools view -h ~/bar-something-data/myfile.bam > ~/bar-something-data/test.sam

It gave the following error:

samtools: writing to standard output failed: Invalid argument
samtools: error closing standard output: -1

How can I resolve this issue? Is there any other tool for converting BAM to SAM other than samtools?

ADD COMMENTlink modified 3.6 years ago by Devon Ryan96k • written 3.6 years ago by gundalav300

Have you tried to use some other means of mounting the file system rather than goofys?

ADD REPLYlink written 3.6 years ago by genomax89k

goofys author here, this shouldn't happen, could you file a bug at with the debug logs?

ADD REPLYlink written 3.6 years ago by github10

fyi this is fixed:

ADD REPLYlink written 3.5 years ago by github10
gravatar for Devon Ryan
3.6 years ago by
Devon Ryan96k
Freiburg, Germany
Devon Ryan96k wrote:

I've never seen documentation for it, but as of version 1.3 samtools can directly interact with S3. Have a look at this post from one of the developers for an example. I would suggest posting to the samtools-help mailing list or posting an issue on github for further help regarding this, since it's mostly only the developer team that seems to be using this feature (at the moment).

ADD COMMENTlink written 3.6 years ago by Devon Ryan96k

For help using this, please post to the samtools-help mailing list. Filing a GitHub issue is more for when you've run into an actual bug, and is less useful for usage problems like this because the conversation will be seen by fewer of your fellow samtools users.

ADD REPLYlink written 3.6 years ago by John Marshall2.1k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 797 users visited in the last hour