script to organize files
1
0
Entering edit mode
4 weeks ago
Galileo ▴ 10

Hello,

I have thousands of files that I would like to organize into project directories automatically. I have a "file metadata table" as a tabulated text file with two columns: (1) file name, (2) project the file belongs to. How can I easily execute:

• make a directory per project name
• mv files to corresponding directory according to metadata

I believe this is something super easy to script for people that do this routinely. I've tried writing something in bash and perl but without success.

Following this operation my goal is to have another script navigate each directory and run a bioinformatic pipeline.

Thank you very much!

organization file scripting bash • 253 views
1
Entering edit mode
4 weeks ago
jv ▴ 10

A bash example

while read file project
do
# check if project dir exists, if not create it
if [[ ! -d $project ]] then mkdir$project
fi
# move file to project dir
mv $file$project


Revise accordingly for your file and project paths

0
Entering edit mode

Thanks jv, this seems to work but read added '$'\r' at the end of filenames of the second column and thus mv cannot find the file. Any ideas how to remove '$'\r' so the file name is recognized?

1
Entering edit mode

I found a solution: used dos2unix on the files that have '\$'\r' at the end :)