Error: SortDetails.cpp, 1111: Unable to create FILE* for temp file: No space left on device. Out of memory.
1
0
Entering edit mode
4.0 years ago
mropri ▴ 160

I am receiving the following error when I try to use bedmap to get the coverage of specific coordinates in a bed file:

Error: SortDetails.cpp, 1111: Unable to create FILE* for temp file: No space left on device. Out of memory.

I understand that I have to free up space somewhere in my files, don't know exactly where. Also, I am on a university network so do not have admin privileges to use commands like sudo -apt clean to remove cache or whatever files are getting stored where there is little memory. The folder I am using is our labs folder and has plenty of space (10 TB), so confused why the error is there. Appreciate any help!

My df -h output is shown here:

https://docs.google.com/presentation/d/1BlnrlnXxbjpK2BsgPqNcmaetmhWhoON1omlv4tm7D9k/edit?usp=sharing

ChIP-Seq • 1.9k views
ADD COMMENT
1
Entering edit mode

Your /tmp is out of space. Unless bedmap allows setting a different temp location (or accepts the value in $TMPDIR, and thus lets you change the temp location), you'll need to clean up your temp location.

On second thought, your /tmp only shows a use of 1%, so my suggestion might not really apply

ADD REPLY
0
Entering edit mode

Is there a way to set a different temp location?

ADD REPLY
1
Entering edit mode

If the manual does not provide an option for this, try setting TMPDIR to a directory under $PWD like so:

OLD_TMPDIR=$TMPDIR
mkdir temp_dir
export TMPDIR=temp_dir
bedmap ....

If the command runs to completion successfully,

export TMPDIR=$OLD_TMPDIR
unset OLD_TMPDIR
ADD REPLY
1
Entering edit mode

This will not solve this particular problem, as sort-bed does not use the POSIX tempnam function (as it is not entirely safe to use on a multiuser system), and so it does not use TMPDIR. It would be a useful feature enhancement, but it is easy enough to specify --tmpdir and --max-mem.

ADD REPLY
1
Entering edit mode

Sorry, I was looking at bedmap, not sort-bed. bedmap doesn't seem to have a tmp-dir option.

ADD REPLY
1
Entering edit mode

I don't think that it would. I don't know why that error would come out of bedmap. If that's the case, I'd love to see what command is being sent.

ADD REPLY
1
Entering edit mode

Just wanted to say thank you guys for your help in this, Alex I know you had initially helped me with getting raw reads of my coordinates in a bed file from my bam file. Always so thankful for the time you guys give for detailed answers!

ADD REPLY
1
Entering edit mode

Thank you Ram for your help in this. Really appreciative of the help you give us all in helping troubleshoot errors we encounter.

ADD REPLY
2
Entering edit mode
4.0 years ago

To sort data with a custom temporary directory, specify use sort-bed --tmpdir <dirname> --max-mem <value> unsorted.bed.

The dirname passed to --tmpdir will be where sorted chunks of the input are stored temporarily, before the final sort.

Note the use of --max-mem. The value passed to --max-mem specifies the maximum system memory that will be used (e.g. 2G for two gigabytes of RAM, if available). This is not disk space. Use free -m to get a sense of how much free system memory you have available.

The temporary directory passed to --tmpdir will need sufficient free space to temporarily store a partitioned duplicate of your input file.

So if /tmp doesn't have enough free space to store a copy of your input BED file (your Google Doc suggests it is 4GB in size, and some of it may be used by other processes and users), you would use a different directory or share, like a subdirectory of /network/rit/home/ed717179, where it looks like you have plenty of free disk space.

ADD COMMENT
2
Entering edit mode

Thank you so much, going to try this out. Truly, appreciate all your help.

ADD REPLY
0
Entering edit mode

If Alex's answer solved your question, please use the green check mark on the left to mark the post as resolved.

ADD REPLY
0
Entering edit mode

Sorry, but trying Alex's method, will hopefully soon check the mark.

ADD REPLY
0
Entering edit mode

Alex you had helped me out with another question of mine where I was looking to find coverage of my bed file using the associated bam file. So the command I am using is the following:

bedmap --skip-unmapped --delim '\t' --echo --count --bases-uniq --echo-ref-size --bases-uniq-f CombinedSE.sorted.bed <(bam2bed < AT1_2_H3K.sorted.bam) > AT1_2_H3K.coverage.bed

Where CombinedSE.bed is the coordinates of all the super enhancers defined by the ROSE algorithm in my samples. I am using one of my samples bam files to get raw counts for those specific coordinates, but I keep getting the error that is originally listed in the question. The solution you provided above, would I be able to tack that on to the command I am using already to get coverage to allocate a subdirectory to temporarily store the inputs? I feel really bad about bothering you over and over, but appreciate your help.

ADD REPLY
0
Entering edit mode

I suspect it is this part that is causing the disk space error:

<(bam2bed < AT1_2_H3K.sorted.bam)

The program underneath bam2bed uses sort-bed under the hood. It offers two options that configure sort-bed in the way I describe in my first answer:

bam2bed --max-mem=[value] --sort-tmpdir=[dirname] < reads.bam

So, basically, you would add those options to the bam2bed portion of <(bam2bed < AT1_2_H3K.sorted.bam):

... <(bam2bed --max-mem=[value] --sort-tmpdir=[dirname] < AT1_2_H3K.sorted.bam) ...

Replace value and dirname, as needed.

Depending on the particulars of your BAM file, other parameters may be useful. Take a look at bam2bed --help or the online documentation, to look at the BAM-specific conversion options. For instance, using --reduced with bam2bed can speed things up quite a bit for coverage measurements.

ADD REPLY
1
Entering edit mode

Thank you so much Alex, this command worked and I was able to get coverage of the bed file using the corresponding bam file. Appreciate your time and help.

ADD REPLY
0
Entering edit mode

Ok, will do, thank you so much for your time in helping me.

ADD REPLY

Login before adding your answer.

Traffic: 2134 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6