Bash programmation (Cygwin): Illegal Character ^M

Question

I have a problem with a character. I think it's a conversion problem between dos and unix.

I have a variable that is a float value. When I print it with the echo command i get:

0.495959

But when I try to make an operation on that value with the bc command (I am not sure how to write the bc command).

echo $mean *1000 |bc

I get:

(standard_in) 1 : illegal character: ^M

I already use the dos2unix command on my .sh file. I think it's because my variable have the ^M character (not printed with the echo command)

How can i eliminate this error?

Solution 1

I don't have Cygwin handy, but in regular Bash, you can use the tr -d command to strip out specified characters, and you can use the $'...' notation to specify weird characters in a command-line argument (it's like a normal single-quoted string, except that it supports C/Java/Perl/etc.-like escape sequences). So, this:

echo "$mean" * 1000 | tr -d $'\r' | bc

will strip out carriage-returns on the way from echo to bc.

You might actually want to run this:

mean=$(echo "$mean" | tr -d $'\r')

which will modify $mean to strip out any carriage-returns inside, and then you won't have to worry about it in later commands that use it.

(Though it's also worth taking a look at the code that sets $mean to begin with. How does $mean end up having a carriage-return in it, anyway? Maybe you can fix that.)

Solution 2

This works:

${mean/^M/}

You can get ^M by typing Ctrl-V followed by Ctrl-M. Or, alternatively:

${mean/$(printf "\r")/}

The benefit of this method compared to @ruakh's is that here you are using bash built-ins only. The first will be faster as the second will run inside a subshell.

If you just want to "unixize" $mean:

mean="${mean/^M/}"

Edit: There's yet another way:

${mean/$'\r'/}

Solution 3

Running Windows stuff in cygwin has one nasty side-effect as you found out - capturing the output of Windows programs in a cygwin bash variable will also capture the CR output by the program.

Judicious use of d2u avoids the issue - for example,

runtime="`mediainfo --Inform='Video;%Duration%' ${movie} | d2u`"

(Without the d2u, ${runtime} would have a CR tacked on the end, which causes the problem you saw when you feed it to 'bc' for example.)

Solution 4

Maybe you should just save your script in UNIX format instead of DOS.

View more solutions

Comments

[ruakh]

That's a good thought, but it won't help. If $mean contains ^M, and Bash recognizes ^M as a word separator (which, by default, it does not), then echo $mean * 1000 will already drop the ^M, because $mean isn't quoted. If it is echo that's adding the ^M, then it obviously won't work (as you've realized), and if the ^M is in the variable and Bash isn't recognizing it as a word separator, then the inner echo will still print the ^M and the outer echo will repeat it -- nothing gained.

[Alfian Nahar]

@ruakh You're right, I hadn't thought it very through!

[Admin]

I already use the dos2unix command to convert it. Is it what you mean?

[loscuropresagio]

Yes, it is. I don't know why using dos2unix didn't solve your problem. The code looks good to me. I tried it and it works fine. But, if I save the script in dos mode, i get the same error: (standard_in) 1: illegal character: ^M. Usually i don't use dos2unix, just because I'm more comfortable with an editor. To replicate your code I used pspad. Its free, maybe you could give it a try.

[Admin]

Thank you I'll try pspad

[Admin]

Thanks, I'll give it a try. btw I was using Wordpad, and that probably causes the problem..

[Payne]

It's actually best practice to convert ANSI-C style string into integer, I think. Only solution that worked for me!