Easiest way to find missing quote in a bash a script?

Question

I have a bash script which is currently 700+ lines long. After a particularly long round of editing, it now errors like so:

./run_me.sh: line 693: unexpected EOF while looking for matching `''
./run_me.sh: line 702: syntax error: unexpected end of file

However, there is nothing wrong I can see with line 693. There isn't even a quote on it.

I've tried running bash -x run_me.sh, and can see the last line run, but there is nothing wrong with that line either (and the code between that line and 693 is, effectively, most of the code).

I can comment out whole swathes of the code, but then I am more likely to see errors due to missing functions, etc, rather than the EOF error I am getting.

So how is someone supposed to find the missing quote, if the line number is reported incorrectly?

(Why is bash's line number reporting so far off, anyway?)

Edit

Incidentally, I found my particular error (by commenting swathes), which was actually a missing } in a variable expansion at an arbitrary line number nowhere near the line indicated by the error message -- quote-based syntax highlighting did not help here, e.g. with the missing brace in "${MY_ARRAY[@]" (should be "${MY_ARRAY[@]}").

Solution 1

A low-tech approach is

tr -cd "'\n" < run_me.sh | awk 'length%2==1 {print NR, $0}'

The tr deletes all characters except for single quotes and newlines, and the awk identifies the lines that have an odd numbers of characters (i.e., non-matching quotes).  Check them out individually; note that valid strings like "That's not a bug!" will be flagged.

If this doesn’t highlight the problem, try again with tr -cd '"\n' (to find non-matching double quotes).  Next, expand the tr to allow (), [], and ultimately {} to go through.  At this point, manual inspection of the output becomes a less fruitful pursuit; while ( and ) are usually matched within the same line, and quotes and square brackets almost always are, this is not true of { and }.  (However, looking for non-matching { and } would find the problem that the OP actually had, namely, typing "${MY_ARRAY[@]" instead of "${MY_ARRAY[@]}").

If you have narrowed the problem down to a curly brace mismatch, and especially if you have narrowed down a range in the file, go to Plan B.  Open the file in vim (or vi).  Go to a { character that you believe is before the error and type %.  If the cursor jumps to the } that you think matches the { you started on, move forward to the next { and repeat.  If it jumps to a } that doesn’t match the { you started on, the error occurs between those two braces.  If it doesn’t move, the error occurs between your current location and the end of the file.  Zoom in.

This approach may be slightly better suited to program files in languages like C, C++, and Java (if their compilers don’t do a good enough job), but it should work fairly well for shell scripts.

Solution 2

An external linting tool like ShellCheck can detect problems and may have better messages and locations than bash itself.

For a program that is mostly echo statements with a "${MY_ARRAY[1]" in the middle, ShellCheck tells you that it couldn't parse the quoted string. It even pins the problem down to the $ character and hints that the parameter expansion was at fault.

ShellCheck works with Vim, Emacs, SublimeText and Atom (among others).

Solution 3

Easiest way? Use an editor with syntax coloring that's aware of the shell you're using, and inspect visually. When you get a big swathe of string color, you know you left out a quote. Just about any decent programming editor has syntax coloring.

Syntax coloring sometimes gets it wrong, but that's usually a sign that your program is too complex and humans are likely to get it wrong too.

For shell programming, make sure to use a font where ", ' and ` are easy to tell apart. Check that you haven't inadvertently used non-ASCII characters that look like ASCII quotes, such as ‘’“”.

Solution 4

I was able to resolve a big problem i had in a 2000+ lines bash script using the low tech solution from G-Man.

I'd just add the following line which does the same for odd number of doublequotes than G-Man's answer for singlequotes.

tr -cd "\"\n" < my_script.sh | awk 'length%2==1 {print NR, $0}'

Btw, shellcheck.net is very nice and every good script should go through it.

View more solutions

Comments

[mikeserv]

You don't have to comment swathes - just the head of each compound command (or function definition). Then uncomment each one-by-one starting from the head of the file. Then just do sh -nv <script as you do. As soon as your error message changes you've found the compound command in which your problem lies. And the line-numbers are fine - each function gets its own.

[jimbobmcgee]

@mikeserv - can you expand on 'the line numbers are fine - each function gets its own'? There are certainly no functions that are 693 lines long (unless you count the script itself as a function). If the error is not at or between the two reported lines, that doesn't seem fine to me. If there is some method to derive/divine the actual failing line (or even the start of the failing compound command) from the line numbers reported by bash, maybe that would be a good start.

[mikeserv]

Ok, you make good points, but if you tried that -nv thing you might get a better idea - the prompts will clue you in. You'll get $PS2 when the shell has at some point begun interpreting a quote or compound layer and has not managed to close that token since the last newline occurred in input. Also try: PS4='LAST EXECLINE: $((LAST+!(LAST=LINENO)))'; set -x because the lineno's can sometimes get rather funny in subshells.

[mikeserv]

Actually, as it turns out, I guess I have explained at least some of it a little better before.

[Emmanuel]

The EOF suggests a missing backquote. In the case of a missing quote bash reports an "unterminated quoted string" .

[jimbobmcgee]

Was actually a missing } inside a quoted string, which Geany's colourising did not spot.

[Gilles 'SO- stop being evil']

@jimbobmcgee Ah. Emacs doesn't detect this, but Vim does.

[jimbobmcgee]

Yes vim does do a reasonable job...eventually. First had to stop it ignoring :syntax on by installing vim-enhanced over vim-minimal (thanks Fedora!), then had to force it into a decent color scheme, since dark blue was actually visible (thanks LXTerminal!). After that, vim did highlight missing braces with a nice red background!

[DisplayName]

Sometimes doing that can be hard. I have an array with over 2000 values now, which appears white in Sublime Text, where all my other arrays are yellow. I would have to pay attention and scroll sideways for like a very long time.

[mike]

Thanks! Sometimes low-tech is better than no-tech! changed the ' to a \" and found my missing double-quote in seconds.

[lampShadesDrifter]

I was using cpdf to add bookmarks and getting a Bad bookmark file (syntax) at line 0 error (and the line number specified is a lie). This post helped me find the mismatched quotes. (Hopefully this comment helps this post get picked up in search for others with this problem).