While I of course never have stupid mistakes...ahem...I have many "friends" who:

1. forget to check both strands
2. generate random genomic sites without avoiding masked (NNN) gaps
3. confuse genome freezes and even species

but I'm sure there are some other very common pitfalls that are unique to bioinformatics programming. What are your favorites?

Invent a new weakly defined, internally redundant, ambiguous, bulky fruit salad of a data format. Again.

I truncated many fasta files this way when trying to see which headers it contained:

grep > some.fasta

I also see a lot of off-by-one errors due to switching between formats

• Bed is 0 based

• GFF/GTF are 1-based

and switching between languages:

• Python and nearly every other modern language are 0-based indexing

• R is 1-based (as is Lua)

Gene annotation stored in an excel file and find out that some HUGO gene names have been hacked by Excel. SEPT9 become sept-9. Conclusion Do not use the .xls format to store your data.

Listen people saying this eternal mistake "Hey these two sequences are 50% homologs"

Reminds me of that old joke...

There are only two hard things in computer science: naming things, cache invalidation and off-by-one errors

I feel like a lot of "stupid mistakes" revolve around betrayed trust and false assumptions

For example:

1. Trusting that a downloaded file is actually fully downloaded

2. Trusting that an aligner will accept a list of query files instead of just taking the first and ignoring the rest (quiz: which ones am i talking about?)

3. Assuming that the quality scores in a FASTQ file are from a great Sanger-encoded run instead of a very poor Illumina-1.3 run

4. Assuming chr1 is followed by chr2 not chr10

• off-by-one errors
• regex errors
• parsing a complex alignment/file format incorrectly (e.g. BLAST or GenBank, probably the original rationale for developing BioPerl)
• failing to account for strand
• failing to revcomp sequences
• failing to account for the last element in a file (because of a improper loop condition or no EOL character on last line)
• failing to account for OS dependent line breaks
• using the wrong assembly/annotation/release
• using the wrong genome coordinate system
• using the wrong file (multiple versions, version skew)
• failing to account for nested/intercalated annotation features (e.g. genes)
• assuming all jobs have completed on a cluster
• deleting files
• not randomizing your data properly
• improper use of statistical tests
• not documenting methods fully (to check and correct all of the above)

If you forgive an attempt to be somewhat provocative, my two favorite mistakes are:

1 Letting academics build software

Academics are in the need to publish papers, and one easy way to do that is to implement an algorithm, demonstrate that it works (more or less), and type it up in a manuscript. BT,DT. But robust and useful software requires a bit more than that, as evidenced by the sad state of affairs in typical bioinformatics software (I think I've managed to crash every de novo assembler I've tried, for instance. Not to mention countless hours spent trying - often in vain - to get software to compile and run). Unfortunately, you don't get a lot of academic credit for improved installation proceedures, testing, software manuals, or especially, debugging of complicated errors. Much better and productive to move on to the next publishable implementation.

2 Letting academics build infrastructure

Same argument as above, really. Academics are eager to apply to construct research infrastructures, but of course they aren't all that interested in doing old and boring stuff. So although today's needs might be satisfied by a $300 FTP server, they will usually start conjecturing about tomorrow's needs instead, and embark on ambitious, blue sky stuff that might result in papers, but not in actually useful tools. And even if you get a useful database or web application up and running (and published), there is little incentive to update or improve it, and it is usually left to bitrot, while the authors go off in search of the next publication.

Using grep to find sequence (or other) IDs without using the -w switch: "grep 'seq12'" will also find seq121, seq122 and so on.

One mistake not unique to bioinformatics is: while editing one source file, compile and run another file.

trying to solve any problem with BioPerl :-)

but the

• '+1' error
• and the grep > some.fasta

are my favorite mistakes.

Try to open microarray or, worse, NGS datafiles with excel or word...

Forget to do 'dos2unix' and then spend a lot of time trying to figure out why there is no OUTPUT

Well i have couple:

1) Run a batch BLAST job and forgetting to put the "-o something.out" option. Then switching off the monitor and coming the next day to see a bunch of characters in my terminal

2) "tar -zxvf" without checking the tar file before, I have decompressed thousands of files in my current directory assuming they came in their own folder.

Re-inventing the wheel. So often did I have to debug (or just replace) a bad implementation of a fasta-parser when BioPython/BioPerl have perfect implementations, I don't understand why no-one bothers to use them. 10 minutes in Google can save you 2 days of work and other people a week of work (you save 2 days of programming, they save a week of understanding your program to find the bug)

I'll offer this one, which is a bit on the general side: Deletion of data that appear to serve no relevance from the computational side, but which have importance to the biology/biologist. Often, this arises from a lack of clear communication between the two individuals/teams as to what everything means, what it exactly means and why it is relevant to the process being developed.

• having manual components to an analysis pipeline (editing data sets running scripts manually)
• Not dealing with error conditions at all. This is one thing that I really noticed when I started with bioinformatics; code that would just merrily continue when it hit incorrect data and output jibberish or fail far away from the bad data. A debugging nightmare.
• Not testing edge and corner cases for input data
• Assuming that your input data is sane; I've run into all sorts of inconsistency issues with public data sets (i.e. protein domains at positions off the end of the protein, etc). Usually fixed promptly if you complain but you've got to find them first.

Not keeping an adequate notebook.

I gave my Amazon EC2 password to someone in my group who wanted to run something quickly (estimated cost, $2). I received the bill 2 months later: $156. This person forgot to close the instance. This is a 8 months story and I'm still waiting for my reimbursement... Conclusion: don't trust colleagues!

I often encounter problems related to the fact the computer scientists index their arrays starting with 0, while biologists index their sequences starting with 1. Simple concept that drives the noobs mad and even trips up more experienced scientists every once in a while.

Masking out sequence in a FASTA file (e.g. s/TAAT/NNNN/ig) where the sequence is formated, i.e. split onto multiple lines.

This will miss TAAT that is split over the end of one line and the start of the next!

The classic mistake (also mentioned above by Casey) is not being aware the genome assembly effect coordinates.

Do pathways statistics or gene set enrichment statistics and then represent the list of gene sets as a valuable result, instead using that statistics just as a means to decide which pathways need to be evaluated.

(This is bad for many reasons for instance because the statistical contribution of a key regulatory gene in a pathway is equal to that of 1 out 7 iso-enzymes that catalyze a non-relevant side reaction, and because the significance of a pathway changes when you add a few non-relevant genes, and also because we have many overlapping pathways).

Another typical mistake is to solve problems that nobody has.

Having separate files for each sequence.

Of a 454 run.

One mistake: not looking to see that the 0x4 bit in the bitflag column of a SAM (or BAM) file indicates the entry is mapped. RNAME, CIGAR, and POS may be set to something non-null (an actual string!) but these are not meaningful if the 0x4 flag says the read is unmapped.

using "rm -rf * .fasta " in unintended directory; especially if within the home directory...

Generate Multiple Sequence Alignment direct from fasta or other file and ask: why there are no gaps & deletion in the MSA viewer?

What kills me the most is the hand editing of data sets.

If you're reading this and do it, please stop -- and start using automated builds -- with clear documentation.

1. Using excel to sort or manage your csv records.
2. Using multiple alignments or highly diverse sequences or worst recombining sequences and inferring evolutionary history based on the resulting tree.

developing algorithms and software around one single piece of low quality data with no prior knowledge while being ignorant about the entire problem.

tacking on another command line argument without looking through the rest of them

novoalign -a ATCTCGTATGCCGTCTTCTGCTTG -d genome.ndx -F ILMFQ -f query.fq -a -m -l 17 -h 60 -t 65 -o sam -o FullNW

the first adapter argument (-a ATCTCGTATGCCGTCTTCTGCTTG) is negated by the empty second one

Easy one: you wait for 4 hours downloading a big DNA file (e.g. bam file) and you mistakenly delete it when trying to move it with a good old rm.

Scripting an hour to do something you could have done in half an hour manually, and then never needing to repeat it again.

I've just made one, which cost me a good headache trying to figure out the biology underlying my strange results!

• expecting SAM's POS field to be the leftmost position of my mapped read on the '+' strand, and the rightmost position on the '-' strand

Note to self : "Read the manual..."

I made one a few months ago. I launched a heavy process in a pay-per-use cluster, it was running for one week. I thought, 6 pennies/hr cannot be too much money. I received a bill for $832 usd. I'm not using this cluster again unless I estimate the total cost of the process.

edit: the price is per core

Loosing an hour to learn that some files saved on a Mac have a strange 'begining of file' like character...

Normalize file standards already! x_o

My common one cat aVeryBigFile.whatever &

Running the bwa/GATK pipeline with a corrupt/incompletely generated bwa index of hg19. Everything still aligned, but one of 2 mates would have its strand set incorrectly. Other than the insert size distribution, everything seemed normal, until the TableRecalibration step downshifted all quality scores significantly and then UnifiedGenotyper called 0 SNPs. 1st time I've seen a problem with step 1 of a pipeline not become obvious until step 5+.

Creating a new set of (public) identifiers for your database for entities that already have identifiers in a widely-user public db.

Having unstable identifiers that are publicly available.

Possibly: implementing methods that magically generate p-values from non-replicated RNA-seq experiments, possibly as a result to pressure from 'experimentalists'. I really would like to know the history behind their implementation (where they forced by reviewers, or by other groups?). Now we have to explain why those p-values are bogus, and why there are so little significantly differentially expressed genes detected in a non-replicated analysis.

I had another good one recently. I was executing an untested bash script for generation of output from two input files on our cluster. I just let it run over night. When I checked in the morning, it had  generated about 40 TB worth of output (expectation was about 20 MB). There was a tiny spelling mistake that led to an infinite loop. Oops. I was lucky to check it when I did because there was still a few TB space left so at least other jobs didn't get killed because of it..

I spent hours to implement R parallel script to fully exploit 64 cores and 250 GB RAM of the server lab. Thus, really proud of myself, I run it on my 4 cores and 8 GB RAM desktop pc. Boom.

You have a fasta file, termed as hg19.fa and the chrosome are list as

> chr1

> chr2

> chr3

Now, You want to check how many chrosome are there? right?

therefore, you run the following command:

grep >  hg19.fa

Congratulations! You hg19.fa is NULL now.

What about building an interface not aimed at a community of (fellow) Biologists?

s/foo/bar/g without the g at the end as I just proved in the field

Making claims without experimental validation. Especially involving studies utilizing multiplexed technologies such as microarrays and high-throughput sequencing.

I wouldn't say it's stupid , but I think a very common mistake is to not correct for batch effects in high-throughput data.

Batch effects can (best-case) hide the real effect that you're looking for, or (worst-case) make it look like your variable of interest is contributing to your findings when it's actually an artifact.

Leek + Irizarry et al. have a sobering review on this here.

Just read this article: "How Not To Be A Bioinformatician" Thought it would be interesting to post here....

Some really great comments here, nice to know that such things happen to all genii ;). I have to say my most painful moments relate to my assumption that data obtained elsewhere is correct in every way. I also remember early in my career, using PDB files and realising that sometimes, chains are represented more than once, thus when manually checking calculations involving atomic coordinates, being utterly perplexed and wanting to break my computer. Oh the joys of Bioinformatics.

Using a statistical test on data that does not satisfy the assumptions of that test.

For example, finding differentially-expressed genes by doing ANOVA on log2(FPKM).

Assuming that the gene IDs in "knownGenes.gtf" from UCSC are actually gene IDs. Instead they just put the transcript ID as the gene ID.

This just caused me a bit of pain when doing read counting at the gene level. Basically, any consittutive exon in a gene with multiple splice forms was ignored because all the reads in that exon were treated as ambiguous.

Using the same output (> oh.shit) for multiple commands.

I found myself guilty iterating through the loop and storing data let's say every 100 iterations... but not storing the very last bit of the data (ie lines 10001 to 10026) at the very end.

These are just a few, but thought worth sharing:

• Not having a copy of VERY IMPORTANT file as backup.
• Forgetting to save parameters used, while running a tool and again start hunting them after few days.
• Irrelevant folder/file names, especially not having extensions like fasta or fastq at the end of file.
• Not having a consolidated folder/place, where all scripts and programs lie.
• Upgrading OS without IT support or prior knowledge, and completely losing the GUI, which needs reinstall of OS again.
• using UNIQ command in linux, without SORTING the data.
• Not removing trailing and leading space of variable, before matching it.

When using scp, I did this:

scp -r somename@somehost:$home ~/

instead of $home, should use ~/

this created a file named $home in my other account.

used rm $home 

and kabum!

should pay attention and look how remove this kind of file before.

I am not jocking,

how about forgetting to have a full loaded battery for your wireless keyboard and/or mouse near you, and the current one inside is running out of power ?

How about writing a tool and being convinced it works perfectly, so you start testing it on a complete dataset instead of testing it first on a subset and finding out after it ran for an hour or so that you made a tiny mistake somewhere. Sooo much time wasted that i'll never get back :P

I have spent hours, in repeated occasions, looking for a mysterious error in a perl script that at the end was simply a "=" instead of a "==" within an IF statement.

Another recurrent mistake: not documenting what I did and what those scripts do in the belief that everything is so intuitive, organised, simple and natural that it won't be necessary. Then, sometime after, I have to spend hours trying to guess what all that mess was.

Spending a few months to find an interesting correlation in data and then presenting to the lab that sequenced to find out they changed coverage on the last few samples!

This one was really good, embarking on sudo yum update when there was lots of stuff to update and swap space was very low. Ended up with a situation much like this. Took me good four hours before I saw my desktop again.

Another fun one is when you develop a pipeline with a small test set thinking speed over all and then you increase your test set size and realize that you're creating TBs of temp data and utilizing hundreds of GBs of RAM :)

Chromosome Identifiers - UCSC "chr1" - Ensembl "1"

Produces very confusing outputs ...

A double mistake combo, 1 - use tar to compress a single file and, 2 - inverting the command arguments

tar cvfz file file.tgz

instead of

tar cvfz file.tgz file

Bye bye file!

It happened to me so many times, that I was considering doing an imagery brain check up.

• Forgetting that human chromosomes are named differently by UCSC and Ensembl (UCSC names have the 'chr' prefix)
• Assuming the alphabetical order of human chromosome names is “chr1”, “chr2”, “chr3”, … when in fact it is “chr1”, “chr10”, “chr11”, …
• Forgetting that there is a fifth possible character in a DNA sequence: N (for unknown)

Mixing between hg18, hg19 and yes new one GrCh 38 too !! :) 

1) Forgot to use -n in numerical sorting of positions in a bed file.

2) Not using -k flag in sort for a specific column, instead sorting all possible stuff in it.