how to unique a file row-wise?
2
0
Entering edit mode
9 months ago

Hello all I have a input file like this:

3 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  786 0  0
19 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  787 0
1 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  786 0  0  0  0
0 9 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  787 0  0  0  
11 0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  786 0  0  0  0

I want to make it unique row-wise. The output , i want to get should look like

3 786 0
19 787 0
1 786 0
9 787 0
11 786 0

Can anybody help me out?

awk perl sed linux • 410 views
ADD COMMENT
0
Entering edit mode

Hello sharmatina189059,

how is this question related to bioinformatics? Is the order of the numbers per row in the output important? If so, how is the relationship to the input?

fin swimmer

ADD REPLY
0
Entering edit mode

It is actually a frequency table I got it using Prophecy (Emboss module). I need to check the number of isolates in which these variations exist. That's why I want to make it sort so that I can read it easily. Can you please help? I could not understand how to make it unique row-wise?

ADD REPLY
4
Entering edit mode
9 months ago
dare_devil ★ 1.5k

use awk

awk '{ n=split($0,a,FS); $0=""; j=1; delete u; for (i=1; i<=n; i++) if (!u[a[i]]++) $(j++) = a[i]; print }' < file.txt

Using while loop

while read -r line; do echo "$line" | grep -o '[^ ]*' | sort -h | uniq | paste -s; done < file.txt
ADD COMMENT
1
Entering edit mode
9 months ago

Using GNU datamash and parallel.

Sending every input line to parallel, which calls sed for converting space-delimited values to tab-delimited, and datamash for transposting column-wise data to row-wise, sort -u is for sorting and saving unique values.

cat t.txt \
    | parallel --pipe --recend "\n" -N 1 -k  \
        'sed -r "s/\s+/\t/g" | datamash transpose | sort -u  | datamash transpose '
0       3       786
0       19      787
0       1       786
0       787     9
0       11      786
ADD COMMENT
0
Entering edit mode

Thanks for the reply

perl -MList::Util=uniq -alne 'print join " ", sort { $a <=> $b } uniq @F' out_new_prophecy > perl_out_prophecy

This also works!!

ADD REPLY

Login before adding your answer.

Traffic: 2801 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6