How to cat in for loop
Solution 1
for i in {1..2} ; do cat *_$i.fastq.gz >>Combined_$i.fastq.gz ; done
This probably won't work well, as concatenated gzip files ; I would imagine you would prefer to use zcat *_$i.fastq.gz >>Combined_$i.fastq
& then gzip; or perhaps better, simply scrap the idea of cat & do this:
for i in {1..2} ; do tar -c *_$i.fastq.gz >Combined_$i.fastq.gz.tar; done
per folder - then to recurse the folders, simply enclose the line above in a further loop, and run this from the top-level folder:
for f in *; do
pushd . ; cd $f
for i in {1..2} ; do tar -c *_$i.fastq.gz >Combined_$i.fastq.gz.tar; done
popd
done
So here, the loop index $f picks every folder; saves your place; cd's to the folder; executes the loop; returns to original directory & loops again.
The pushd / popd are there to ensure an error doesn't have you roaming the filesystem! Not strictly neccesary, but useful to learn.
Solution 2
This isn't a great example for learning bash, but the simplest way is:
zcat *_1.fastq.gz | gzip > Combined_1.fastq.gz &
zcat *_2.fastq.gz | gzip > Combined_2.fastq.gz
Using a loop:
for f in *_1.fastq.gz; do
zcat "$f"
done | gzip > Combined_1.fastq.gz
Notes
- you iterate over the results of a glob pattern -- do not be tempted to parse the output of
ls
(http://mywiki.wooledge.org/ParsingLs) - quote your
"$variables"
when you want the value - you can redirect or pipe the output of an entire for or while loop.
Related videos on Youtube
zazga
Updated on September 18, 2022Comments
-
zazga over 1 year
I have 500 folders containing many *_1.fastq.gz and *_2.fastq.gz files per folder.
I want to:
cat *_1.fastq.gz > Combined *_1.fastq.gz & cat *_2.fastq.gz > Combined_2.fastq.gz
per folder.
How do I achieve this? I would like to learn some bash, I am comfortable with python
-
zazga about 9 yearswith some slight tweaks I got it to do what I wanted. I changed back the tar -c to cat, since cat does exactly what I need. Thanks for solving my problem