In [1]:
!date
Mon Apr 11 07:47:32 PDT 2016
In [2]:
%%bash
system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.9.5 (13F34)
      Kernel Version: Darwin 13.4.0
      Boot Volume: Hummingbird
      Boot Mode: Normal
      Computer Name: hummingbird
      User Name: Sam (Sam)
      Secure Virtual Memory: Enabled
      Time since boot: 142 days 16:38

In [128]:
cd /Volumes/nightingales/O_lurida/20160203_mbdseq/
/Volumes/nightingales/O_lurida/20160203_mbdseq

Concatenate entire set of FASTQ files for each individual

The commands below use a for loop to:
  1. process each FASTQ corresponding to an individual (zr1394n*)
  2. concatenate each file (cat \$file) to the designated output file (\${file/_s*_R1/})

The output file is named using bash parameter expansion. It takes the file name ($file) and replaces the designated matching text (_s*_R1) with whatever follows the last slash - in this case, it's replaced with an empty string (i.e. deleted).

In [129]:
%%bash

for file in zr1394_1_*; do
  cat $file >> ${file/_s*_R1/}
done
In [130]:
%%bash

for file in zr1394_2_*; do
  cat $file >> ${file/_s*_R1/}
done
In [131]:
%%bash

for file in zr1394_3_*; do
  cat $file >> ${file/_s*_R1/}
done
In [132]:
%%bash

for file in zr1394_4_*; do
  cat $file >> ${file/_s*_R1/}
done
In [133]:
%%bash

for file in zr1394_5_*; do
  cat $file >> ${file/_s*_R1/}
done
In [134]:
%%bash

for file in zr1394_6_*; do
  cat $file >> ${file/_s*_R1/}
done
In [135]:
%%bash

for file in zr1394_7_*; do
  cat $file >> ${file/_s*_R1/}
done
In [136]:
%%bash

for file in zr1394_8_*; do
  cat $file >> ${file/_s*_R1/}
done
In [137]:
%%bash

for file in zr1394_9_*; do
  cat $file >> ${file/_s*_R1/}
done
In [138]:
%%bash

for file in zr1394_10_*; do
  cat $file >> ${file/_s*_R1/}
done
In [139]:
%%bash

for file in zr1394_11_*; do
  cat $file >> ${file/_s*_R1/}
done
In [140]:
%%bash

for file in zr1394_12_*; do
  cat $file >> ${file/_s*_R1/}
done
In [141]:
%%bash

for file in zr1394_13_*; do
  cat $file >> ${file/_s*_R1/}
done
In [142]:
%%bash

for file in zr1394_14_*; do
  cat $file >> ${file/_s*_R1/}
done
In [143]:
%%bash

for file in zr1394_15_*; do
  cat $file >> ${file/_s*_R1/}
done
In [144]:
%%bash

for file in zr1394_16_*; do
  cat $file >> ${file/_s*_R1/}
done
In [145]:
%%bash

for file in zr1394_17_*; do
  cat $file >> ${file/_s*_R1/}
done
In [146]:
%%bash

for file in zr1394_18_*; do
  cat $file >> ${file/_s*_R1/}
done
In [147]:
%%bash

ls -lh zr1394_{1..18}.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 14:33 zr1394_1.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.7G Apr 11 14:50 zr1394_10.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.8G Apr 11 14:52 zr1394_11.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.8G Apr 11 14:55 zr1394_12.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.0G Apr 11 14:59 zr1394_13.fastq.gz
-rw-rw-rw-  1 Sam  staff   4.6G Apr 11 15:03 zr1394_14.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.2G Apr 11 15:06 zr1394_15.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.5G Apr 11 15:09 zr1394_16.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.2G Apr 11 15:11 zr1394_17.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.5G Apr 11  2016 zr1394_18.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 14:35 zr1394_2.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.5G Apr 11 14:37 zr1394_3.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.1G Apr 11 14:38 zr1394_4.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.0G Apr 11 14:40 zr1394_5.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.3G Apr 11 14:42 zr1394_6.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.6G Apr 11 14:43 zr1394_7.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.0G Apr 11 14:45 zr1394_8.fastq.gz
-rw-rw-rw-  1 Sam  staff   3.9G Apr 11 14:47 zr1394_9.fastq.gz

Concatenate sets of s4, s5, & s6 of FASTQ files for each individual

The commands below use a for loop to:
  1. process the s4, s5, & s6 FASTQ corresponding to an individual (zr1394_1_s{4..6}*)
  2. concatenate each file (cat \$file) to the designated output file (\${file/_s*_R1/_s456})

The output file is named using bash parameter expansion. It takes the file name ($file) and replaces the designated matching text (_s*_R1) with whatever follows the last slash - in this case, it's replaced with "_s456".

In [148]:
%%bash

for file in zr1394_1_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [149]:
%%bash

for file in zr1394_2_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [150]:
%%bash

for file in zr1394_3_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [151]:
%%bash

for file in zr1394_4_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [152]:
%%bash

for file in zr1394_5_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [153]:
%%bash

for file in zr1394_6_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [154]:
%%bash

for file in zr1394_7_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [155]:
%%bash

for file in zr1394_8_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [156]:
%%bash

for file in zr1394_9_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [157]:
%%bash

for file in zr1394_10_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [158]:
%%bash

for file in zr1394_11_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [159]:
%%bash

for file in zr1394_12_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [160]:
%%bash

for file in zr1394_13_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [161]:
%%bash

for file in zr1394_14_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [162]:
%%bash

for file in zr1394_15_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [163]:
%%bash

for file in zr1394_16_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [164]:
%%bash

for file in zr1394_17_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [165]:
%%bash

for file in zr1394_18_s{4..6}*; do
  cat $file >> ${file/_s*_R1/_s456}
done
In [166]:
ls -lh zr1394_*_s456.fastq.gz | sort
-rw-rw-rw-  1 Sam  staff   1.2G Apr 11 15:22 zr1394_8_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.4G Apr 11  2016 zr1394_17_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.4G Apr 11 15:18 zr1394_4_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:14 zr1394_1_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:16 zr1394_2_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:17 zr1394_3_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:19 zr1394_5_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.5G Apr 11 15:20 zr1394_6_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.6G Apr 11 15:22 zr1394_7_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.7G Apr 11  2016 zr1394_18_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.7G Apr 11 15:27 zr1394_11_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   1.9G Apr 11 15:29 zr1394_12_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.0G Apr 11 15:30 zr1394_13_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.2G Apr 11 15:35 zr1394_15_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 15:24 zr1394_9_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.4G Apr 11 15:37 zr1394_16_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.5G Apr 11 15:26 zr1394_10_s456.fastq.gz
-rw-rw-rw-  1 Sam  staff   2.8G Apr 11 15:32 zr1394_14_s456.fastq.gz