Home Merge multiple files in multiple directories - bash
Reply: 3

Merge multiple files in multiple directories - bash

Annie Jeba
1#
Annie Jeba Published in 2018-02-09 07:54:25Z

We have a requirement to loop multiple directiories and in each directory there will be, multiple text files with Pattern File"n".txt which needs to merged to one, File.txt

We are using Bourne Shell scripting.

Example:

/staging/dusk/inbound/ --> Main Directory

Dir1
File1.txt,File2.txt,..sample1.doc,sample2.pdf,File*.txt  --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir2
File1.txt,File2.txt,..attach1.txt,sample1.doc, File*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir3
File1.txt,File2.txt,File*.txt,..sample1,sample2*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir4
File1.txt,File2.txt,File*.txt,..temp.doc,attach.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Di5
File1.txt,File2.txt,File*.txt,..sample1,sample2*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir"n"
File1.txt,File2.txt,File3.txt,File*.txt..attach1,attach*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

The files from each directory can be looped using cat *.txt > all.txt command.

But how we loop the directiories?

abhishek phukan
2#
abhishek phukan Reply to 2018-02-09 08:42:57Z

you can probably try using the find command

find /staging/dusk/inbound/ -name "*.txt" -type f -exec cat {} \;>>MergedFile.txt
Kernelv5
3#
Kernelv5 Reply to 2018-02-09 08:31:06Z

You can do it in a ad-hok way

find /staging/dusk/inbound -type f -name *.txt | sort -n | awk '{ print "cat " $1 " >> all.txt"}' > marge.sh
chmod +x marge.sh
sh merge.sh
tripleee
4#
tripleee Reply to 2018-02-09 08:23:48Z

To catenate Fil*.txt from all immediate subdirectories of the current directory,

cat */Fil*.txt >Final.txt

To access arbitrarily deeply nested subdirectories, some shells offer ** as an extension, but this is not POSIX sh -compatible.

cat **/Fil*.txt >Final.txt

If you actually want to loop over your directories and do something more complex with each, that's

for d in */; do
    : something with "$d"
done

Or similarly, for shells which support **, you can loop over all directories within directories;

for d in **/; do
    : something with "$d"
done

For example, : something with "$d" could be cat "$d"/Fil*.txt >"$d"/Final.txt to create a Final.txt in each directory, which contains only the Fil*.txt files in that directory.

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.319033 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO