Why "syntax error near unexpected token"?

Question

I have about 10k (approx 180x50x2) CSV files which I want to join together as following, but the inner for loop fails because of some syntax error; I cannot see the error in lastFile

#!/bin/bash
dir='/home/masi/CSV/'
targetDir='/tmp/'
ids=(118 119 120)
channels=(1 2)
for id in ids;

        do
        for channel in channels;

                # example filename P209C1T720-T730.csv
                lastFile="$dir'P'$id'C'$channel'T1790-T1800.csv'"
                # show warning if last file does not exist
                if [[ -f $lastFile ]]; then
                    echo "Last file "$lastFile" is missing" 
                    exit 1
                fi

                filenameTarget="$targetDir'P'$id'C'$channel'.csv'"
                cat $dir'P'$id'C'$channel'T'*'.csv' > $filenameTarget

        done;
done

Error

makeCSV.sh: line 12: syntax error near unexpected token `lastFile="$dir'P'$id'C'$channel'T1790-T1800.csv'"'
makeCSV.sh: line 12: `      lastFile="$dir'P'$id'C'$channel'T1790-T1800.csv'"'

OS: Debian 8.5
Linux kernel: 4.6 backports

Solution 1

There is a missing do in your second for loop:

for id in ids;

        do
        for channel in channels; do # <----- here ----

                # example filename P209C1T720-T730.csv
                lastFile="$dir'P'$id'C'$channel'T1790-T1800.csv'"
                # show warning if last file does not exist
                if [[ -f $lastFile ]]; then
                    echo "Last file "$lastFile" is missing" 
                    exit 1
                fi

                filenameTarget="$targetDir'P'$id'C'$channel'.csv'"
                cat $dir'P'$id'C'$channel'T'*'.csv' > $filenameTarget

        done;
done

Based on the discussion in the comments I see your confusion with the syntax of the for loop.

This is the rough syntax of the for loop:

for name in list; do commands; done

There always must be a do before commands and a ; (or newline) followed by done after the commands.

Here is a variation with more newlines:

for name in list
do
    commands
done

Solution 2

It's properly working:

#!/bin/bash
dir='/home/masi/CSV/'
targetDir='/tmp/'
ids=(118 119 120)
channels=(1 2)
for id in ids ; do

        # Add do after ';'
        for channel in channels ; do

                # example filename P209C1T720-T730.csv
                lastFile="$dir'P'$id'C'$channel'T1790-T1800.csv'"
                # show warning if last file does not exist
                if [[ -f $lastFile ]] ; then
                    echo "Last file "$lastFile" is missing" 
                    exit 1
                fi

                filenameTarget="$targetDir'P'$id'C'$channel'.csv'"
                cat $dir'P'$id'C'$channel'T'*'.csv' > $filenameTarget

        done
done

For the future use bash debugger: bash -x /path/to/your/script.

Comments

[Lesmana]

there is a missing do in your second for loop

[Lesmana]

I really really don't see a do in the second for loop. Are you really sure that there is a do?

[Lesmana]

The do in the previous line is the do for the first for loop. The second for loop needs another do. Syntax of for loop: for name in list; do commands; done.

[Léo Léopold Hertz 준영]

How can you use the asterisk correctly? I get cat: /home/masi/CSV/PidsCchannelsT*.csv: No such file or directory where variable expansions are not working and also * does not seem to work as expected. Also, all variable expansions are not working.

[Léo Léopold Hertz 준영]

There is still one problem in the lastFile variable. No variables are expanded as expected and * is not used as used as expected. Do you understand why?

[Lesmana]

if bash does not expand * that usually means there are no filenames which match the pattern. If you still have problems with that part then please ask a new question.

[Léo Léopold Hertz 준영]

I expanded the other part here unix.stackexchange.com/q/319689/16920