Question: Problem Converting BAM to SAM on to Amazon-S3 Using Samtools
0
gravatar for gundalav
2.2 years ago by
gundalav260
La La Land
gundalav260 wrote:

In my Amazon EC2 instance I have the following:

ubuntu@ip-333-31-16-230:~$ df -h
Filesystem       Size  Used Avail Use% Mounted on
udev              79G     0   79G   0% /dev
tmpfs             16G  8.8M   16G   1% /run
/dev/xvda1       985G  136G  809G  15% /
tmpfs             79G     0   79G   0% /dev/shm
tmpfs            5.0M     0  5.0M   0% /run/lock
tmpfs             79G     0   79G   0% /sys/fs/cgroup
tmpfs             16G     0   16G   0% /run/user/1000
bar-something  1.0P     0  1.0P   0% /home/ubuntu/bar-something-data
bar-foolishbrat  1.0P     0  1.0P   0% /home/ubuntu/bar-foolisbrat-data

The last two file system bar-bi and bar-foolishbrat are Amazon S3 bucket, mounted with the following command:

/home/ubuntu/go/bin/goofys bar-foolishbrat bar-foolishbrat-data
/home/ubuntu/go/bin/goofys bar-somehting bar-something-data
# Here we use Goofys. It can be achieved also with S3FS.

Now I have no problem doing this, in my instance

samtools view -h ~/bar-something-data/myfile.bam > test.sam

That is writing the SAM file in my instance file system. But when I do this, namely writing to the mounted S3:

samtools view -h ~/bar-something-data/myfile.bam > ~/bar-something-data/test.sam

It gave the following error:

samtools: writing to standard output failed: Invalid argument
samtools: error closing standard output: -1

How can I resolve this issue? Is there any other tool for converting BAM to SAM other than samtools?

ADD COMMENTlink modified 2.2 years ago by Devon Ryan89k • written 2.2 years ago by gundalav260
1

Have you tried to use some other means of mounting the file system rather than goofys?

ADD REPLYlink written 2.2 years ago by genomax65k
1

goofys author here, this shouldn't happen, could you file a bug at https://github.com/kahing/goofys/issues with the debug logs?

ADD REPLYlink written 2.2 years ago by github10

fyi this is fixed: https://github.com/kahing/goofys/issues/154

ADD REPLYlink written 2.1 years ago by github10
0
gravatar for Devon Ryan
2.2 years ago by
Devon Ryan89k
Freiburg, Germany
Devon Ryan89k wrote:

I've never seen documentation for it, but as of version 1.3 samtools can directly interact with S3. Have a look at this post from one of the developers for an example. I would suggest posting to the samtools-help mailing list or posting an issue on github for further help regarding this, since it's mostly only the developer team that seems to be using this feature (at the moment).

ADD COMMENTlink written 2.2 years ago by Devon Ryan89k

For help using this, please post to the samtools-help mailing list. Filing a GitHub issue is more for when you've run into an actual bug, and is less useful for usage problems like this because the conversation will be seen by fewer of your fellow samtools users.

ADD REPLYlink written 2.2 years ago by John Marshall1.5k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1085 users visited in the last hour