Forum:When To Use Sed Awk Over Perl Or Python
5
2
Entering edit mode
8.1 years ago
Medhat 9.0k

Hi,

This is not debate or geeky question, It is simple as it is
sed program is a stream editor,AWK is an interpreted programming language designed for text processing and typically used as a data extraction and reporting tool wikipedia , perl and python are programming languages and here in biostars a lot of talks cover perl vs python topic When To Choose Python Over Perl (And Vice Versa)?

My question If you already knows Perl or python is it worth to learn Sed and Awk?

From your experience which tool of the four helped you best in solving bioinformatics problem and in what situation?

Conclusion: Which to use when?

Thanks

awk perl python Forum • 33k views
5
Entering edit mode
8.1 years ago

For me it's about efficiency and not interrupting workflow.

I only use sed/awk for simple tasks when I know it will be faster to type out and execute than a perl/python script. I try to use sed/awk over scripts because they don't interrupt my workflow. When you are on the command line, it's nice to stay on the command line. Even having to open up vi, type out your script, and then saving can be annoying and interrupts your flow of thought.

However, if you spend 2-3 minutes perfecting a sed/awk one-liner when you could have written a script in 30 seconds, then that's probably not a very good use of your time and you have also interrupted your train of thought.

I think the important mentality to have when doing data analysis is that your goal is to work with the data, not to work with the tools.

0
Entering edit mode

you don't need to interrupt your CLI work - check out Perl one-liners (e.g. this great collection)

5
Entering edit mode
8.1 years ago
Pavel Senin ★ 1.9k

I walk to the office, use a bicycle to get to boulangerie, and a car to get to the ski station. While I can drive up to work and stop by a store on the way back, I really like walking and biking. Java can do all of that stuff perl an python do, but I still use awk/sed very often for no particular reason :).

0
Entering edit mode

Nice and cheerful answer :)

0
Entering edit mode

Really sorry if you didn't like it. Personally, I think that it is awesome that there are so many tools around, so many ways to get things done, so many data sources... One can always find a tool or a way which makes more sense from particular personal (biased) perspective. I am trying to communicate that skills (such as in in sed and awk VS perl or pyton) are really driven by personal preferences and habits.

0
Entering edit mode

I liked it by the way +1 :)

4
Entering edit mode
8.1 years ago

Sure, sometimes awk/sed prove simpler than perl/python (there are a HUGE number of awk examples on this site for that very reason). I use the former two for simple file processing all the time. If things are a bit more complicated, then you need perl/python. One caveat is that sed/awk lend themselves to one-off uses, so I generally just write something in python if I expect to use it numerous times.

0
Entering edit mode

So sed and Awk are interchangeable but yet awk is more powerful?

1
Entering edit mode

They're not really interchangeable. sed is mostly about changing the text of a line using regular expressions. Awk is about filtering, extracting lines and doing some maths on particular columns.

0
Entering edit mode

To reiterate what cts said, no, they're not interchangeable. I often use the two together since, they have different strengths.

3
Entering edit mode
8.1 years ago
cts ★ 1.7k

Perl can do everything that sed and awk can do on the command line, however I still use awk in many occasions because typing out an awk one-liner is usually shorter than the corresponding perl one-liner. I almost never use sed at all because I'm so used to perl's regular expression syntax. Awk has pretty much replaced my need for perl as it can do most of the basic data filtering that I used to do in perl scripts, but it does not support more complex data structures like having a hash or arrays etc. For that I usually use python, particularly if it is something that I'll use alot. I still use perl occasionally for quick scripts that I'm not likely to use many times as autovivification makes creating a complicated data structure very quick when I'm too lazy to write the same thing in python.

0
Entering edit mode
6 months ago
alanh ▴ 110

These days, almost anything that's more complex than a one-liner should probably be a Python script, but perl can still be useful.

For whatever reason, I learned Perl first back in the day and I've had little reason to learn awk or sed. Perl has command line options that are very nice for stream editing:

Extracted from the perl 5.32 command line help:

  -i[extension]     edit <> files in place (makes backup if extension supplied)
-n                assume "while (<>) { ... }" loop around program
-p                assume loop like -n but print line also, like sed


E.g. to replace all instance of foo with bar in file.txt and copy the original file to file.txt.bak:

perl -pi.bak -e 's/foo/bar/g' file.txt


or, say, if I want to extract only the lines that contain sample2 into a file, and correct a path:

cat file.txt | perl -n -e ' s{/wrong/path}[/path/right]g ; print if /sample2/' > file.only_sample2.corrected.txt