How to calculate values in a shell script?

Question

I run this command in the terminal:

grep "bla bla blah" blah* | echo "Blah: $(wc -l) / $(ls | wc -l) * 100"

And I get this output:

Blah: 44 / 89 * 100

What I expect to see:

49.4

Is there a way to obtain the desired output using just the bash commands? I don't prefer scripts I am planning to pipe the output.

Solution 1

Your code says to print a string. It doesn't say anywhere that this string is in fact an arithmetic expression that you want evaluated. So you can't reasonably expect your expression to be evaluated.

Your code is suboptimal. $(wc -l) will count the number of matches returned by grep, but there's a simpler way: run grep -c instead. $(ls | wc -l) is an unreliable way of counting the non-dot files in the current directory, because the output of ls isn't reliable; $(set -- *; echo $#) is a reliable way of doing this (assuming there is at least one matching file; if that assumption might not hold, use $(set -- *; if [ -e "$1" ]; then echo $#; else echo 0; fi, but note that this will result in a division by zero below which you should treat as an error condition one way or another). So you can write your code this way:

matches=$(grep -c "bla bla blah" blah*)
files=$(set -- *; echo $#)
echo "Blah: $matches / $files * 100"

or you can inline the computation of the two intermediate values:

echo "Blah: $(grep -c "bla bla blah" blah*) / $(set -- *; echo $#) * 100"

Now, to perform the arithmetic, you can use the shell's built-in arithmetic expansion, but it's limited to integer operations, so the / operator will round down to the nearest integer.

echo "Blah: $(($matches * 100 / $files))"

In ksh93, zsh and yash, but not in other shells, you get floating-point arithmetic if there's something in the expression to force floating-point, such as a floating-point constant. This feature is not present in the Bourne shell, ksh88, pdksh, bash, ash.

echo "Blah: $(($matches * 100.0 / $files))"

The bc utility performs operations on decimal number with arbitrary precision.

echo "Blah: $(echo "scale=2; $matches * 100 / $files" | bc)"

Another standard utility that can perform floating-point computation (with fewer mathematical functions available) is awk.

echo "$matches" "$files" | awk '{print "Blah:", $1 * 100 / $2}'

Solution 2

First of all, you've not specified your shell. I'll presume you're using bash, but please state it in future.

It's also very important that you don't parse the output of ls. There's good documentation on why to not do so here.

Also, what are you attempting to obtain the percentage output of? You don't seem to be attempting to calculate a percentage at the end. For now I just did the exact calculation you listed.

Here is a small script that should be able to do this without the issues mentioned:

#!/bin/bash

_die() {
    printf '%s\n' "${@:2}"
    exit "$1"
}

(( $# )) || _die 1 "Usage: ${0##*/} pattern <dir>"

[[ $2 ]] && _dir=$2 || _dir=.

[[ -d ${_dir} ]] || _die 2 "Directory does not exist: ${_dir}"

for _file in "${_dir}"/*; do
    [[ -f ${_file} ]] && _files+=( "${_file}" )
done

(( ${#_files[@]} )) || _die 3 'No files matched by glob, not attempting to divide by 0.'

# We pass the same files found to grep instead of reglobbing to avoid a race condition.
while IFS= read -r _number_of_matches; do
    (( _total_matches )) && (( _total_matches+=_number_of_matches )) || _total_matches=${_number_of_matches}
done < <(grep -hc "$1" "${_files[@]}")

(( _total_matches )) || _die 4 "Nothing matched by expression: $1"

printf 'Blah: %s\n' "$(bc <<< "${_total_matches}/${#_files[@]}")"

Bear in mind that bc is not portable. If you don't mind using integer arithmetic, you could use the shell to calculate and return it instead of passing it to bc by using $((.

Comments

[clerksx]

This command is dangerous (race condition).

[jokerdino]

If there is a way to obtain the result without the usage of scripts, I would be grateful! And yes, I am using bash.

[jokerdino]

This command does provide me the desired output. Is it really dangerous? Are there any safer alternatives?

[clerksx]

Well, as "dangerous" as a command doing this can be, that is, you are in danger of getting the complete wrong result.

[clerksx]

@jokerdino Put it in a function in your ~/.bashrc then (note, you also will want to source ~/.bashrc from ~/.bash_profile.

[Gilles 'SO- stop being evil']

If you're assuming bash anyway, shopt -s nullglob is a far easier way of handling the case when * doesn't match. Also, bc is standard.

[Gilles 'SO- stop being evil']

@jokerdino If a file bla* is created or removed while you're running this command, then grep and cat may work through different sets of files. I don't know where the cat bla* comes from, though, it doesn't seem to be in the requirements.

[clerksx]

@Gilles Many Unices do not include bc by default (Arch Linux offhand). As for nullglob, I'd still have to do the checks regardless.

[janmoesen]

You can also set the nullglob shell option to avoid the "no matching files returns the glob pattern" problem. num_files=$(shopt -s nullglob; set -- *; echo $#)

[rvdginste]

Where is the race condition? The code inside the backticks should be evaluated before the whole argument is passed to "echo"? There are two pieces of code with backticks, and the order in which those are executed does not matter.

[rvdginste]

Hmm.. you mean that extra files are created or removed while executing the command. Ok, fair enough. That line is just a quick'n dirty way to print some statistics. It is not a full-blown, bulletproof script. Whether it is actually dangerous, depends on the circumstances where you intend to use it.