Unwanted question mark appended to directory name when using mkdir

Question

I am unsure why when I use mkdir -p $fbase an unwanted ? is appended to the name of the directory (27obs87St01Rec1?) . I have followed similar threads and ran cat -A filename to see if there are unwanted characters in the script but there dont appear to be. I have also run dos2unix filename. I am writing the script from scratch on a linux machine so no influence from Windows. Is it because I am using a parameter $filebase. Appreciate any help

#!/bin/csh

# Distribute files into separate directories and run SHEBA on each
# Check what phase to run analysis for

echo "RUN_SHEBA.CSH: CNTRL-C to abort"

unalias sac

mkdir -p filtered
mkdir -p stack
mkdir -p plots
mkdir -p results

# READ STATION LIST

set nev = `cat stationlist1.csv | wc -l`
echo "NUMBER OF EVENTS=" $nev
@ iev = 1
echo $iev

# LOOP OVER EVENTS
while ($iev <= $nev)
echo "CURRENT EVENT=" $iev

# EXTRACT EVENT LINE
set fbase = `head -$iev stationlist1.csv | tail -1`
echo $fbase


# CLEAN UP ANY PRE-EXISTING DIRECTORIES
\rm -rf $fbase

mkdir -p $fbase

# COPY THE FILES
cp ~/SAC/sac/macros/$fbase.* $fbase

cd $fbase

The full script is below...

#!/bin/csh

# Distribute files into separate directories and run SHEBA on each
# Check what phase to run analysis for

echo "RUN_SHEBA.CSH: CNTRL-C to abort"

unalias sac

mkdir -p filtered
mkdir -p stack
mkdir -p plots
mkdir -p results

# READ STATION LIST

set nev = `cat stationlist1.csv | wc -l`
echo "NUMBER OF EVENTS=" $nev
@ iev = 1
echo $iev

# LOOP OVER EVENTS
while ($iev <= $nev)
echo "CURRENT EVENT=" $iev

# EXTRACT EVENT LINE
set fbase = `head -$iev stationlist1.csv | tail -1`
echo $fbase


# CLEAN UP ANY PRE-EXISTING DIRECTORIES
\rm -rf $fbase

mkdir -p $fbase

# COPY THE FILES
cp ~/SAC/sac/macros/$fbase.* $fbase

cd $fbase

## CONFIGURE SHEBA

# BUILD THE ANALYSIS MACRO
cat << END >.temporary_sac_shebanew
setmacro /apps/SAC/sac/macros
window 1 X 0.1 0.6 Y 0.50 1.00
qdp off
gtext hardware
END

echo "m shebanew file $fbase comps C1 C2 C3 pick no plot yes batch yes" >>.temporary_sac_shebanew

# RUN SHEBA

sac .temporary_sac_shebanew

\cp ${fbase}_result.eps ../plots
\cp ${fbase}.lam2 ../stack
\cp *.final_result ../results

cd..

@ iev = $iev + 1

echo "I AM FINISHED"
end

Solution

I see in my crystal ball that the input file was prepared under Windows.

I bet the character isn't a question mark, that's just ls's way of telling you that there's an unprintable character. The unprintable character in question is a carriage return character (CR, commonly represented as \r or \015 or ^M).

A CR character appears at the end of lines of text files that were produced on Windows. Windows and Unix represent text files differently: Unix uses the LF character (\n, \012, ^J) as a line terminator (i.e. every line consists of its printable characters followed by LF), whereas Windows uses the two-character sequence CR-LF as a line separator (i.e. every line consists of its printable characters followed by CR followed by LF, except that the last line usually consists of only its printable characters). So when you copy a Windows text file to a Unix system, it has this spurious CR character at the end of every line except sometimes the last one. As far as Linux or other Unix systems are concerned, the CR character is an ordinary character (albeit an unprintable one), so it just stays there and causes trouble.

You should convert the file to be a Unix text file. On Linux, you can use

sed -i -e 's/\r$//' /path/to/file

Alternatively, to make a clean copy, you can use

</path/to/windows/file tr -d '\r' >/path/to/unix/file

If you prefer to make your script robust, remove the CR when you read the file:

set fbase = `head -$iev stationlist1.csv | tail -1 | tr -d '\r'`

Comments

[jai_s]

Adding "set -x" at the top (for sh/bash - you'll need to check the equivalent for csh) will give you debug output. What does the "echo $fbase" show ?

[Mark Plotnick]

If you change echo $fbase to echo $fbase | cat -vet, do you see any unwanted characters at the end?