script to organize files
1
0
Entering edit mode
4 weeks ago
Galileo ▴ 10

Hello,

I have thousands of files that I would like to organize into project directories automatically. I have a "file metadata table" as a tabulated text file with two columns: (1) file name, (2) project the file belongs to. How can I easily execute:

  • make a directory per project name
  • mv files to corresponding directory according to metadata

I believe this is something super easy to script for people that do this routinely. I've tried writing something in bash and perl but without success.

Following this operation my goal is to have another script navigate each directory and run a bioinformatic pipeline.

Thank you very much!

organization file scripting bash • 253 views
ADD COMMENT
1
Entering edit mode
4 weeks ago
jv ▴ 10

A bash example

while read file project
do
  # check if project dir exists, if not create it
  if [[ ! -d $project ]]
  then
    mkdir $project
  fi
  # move file to project dir
  mv $file $project
done < metadata.txt

Revise accordingly for your file and project paths

ADD COMMENT
0
Entering edit mode

Thanks jv, this seems to work but read added '$'\r' at the end of filenames of the second column and thus mv cannot find the file. Any ideas how to remove '$'\r' so the file name is recognized?

ADD REPLY
1
Entering edit mode

I found a solution: used dos2unix on the files that have '$'\r' at the end :)

ADD REPLY

Login before adding your answer.

Traffic: 1219 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6