How to defer variable expansion

Question

I was wanting to initialize some strings at the top of my script with variables that have no yet been set, such as:

str1='I went to ${PLACE} and saw ${EVENT}'
str2='If you do ${ACTION} you will ${RESULT}'

and then later on PLACE, EVENT, ACTION, and RESULT will be set. I want to then be able to print my strings out with the variables expanded. Is my only option eval? This seems to work:

eval "echo ${str1}"

is this standard? is there a better way to do this? It would be nice to not run eval considering the variables could be anything.

Solution 1

With the kind of input you show, the only way to leverage shell expansion to substitute values into a string is to use eval in some form. This is safe as long as you control the value of str1 and can ensure that it only references variables that are known as safe (not containing confidential data) and doesn't contain any other unquoted shell special character. You should expand the string inside double quotes or in a here document, that way only "$\` are special (they need to be preceded by a \ in str1).

eval "substituted=\"$str1\""

It would be a lot more robust to define a function instead of a string.

fill_template () {
  sentence1="I went to ${PLACE} and saw ${EVENT}"
  sentence2="If you do ${ACTION} you will ${RESULT}"
}

Set the variables then call the function fill_template to set the output variables.

PLACE=Sydney; EVENT=fireworks
ACTION='not learn from history'; RESULT='have to relive history'
fill_template
echo "During my holidays, $sentence1."
echo "Cicero said: \"$sentence2\"."

Solution 2

As I take your meaning, I don't believe that any of these answers are correct. eval is not necessary in any way, nor do you have any need even to twice evaluate your variables.

It's true, @Gilles comes very close, but he does not address the problem of possibly overriding values and how they should be used if you need them more than once. After all, a template should be used more than once, right?

I think it's more the order in which you evaluate them that's important. Consider the following:

TOP

Here you'll set some defaults and prepare to print them when called...

#!/bin/sh
    _top_of_script_pr() ( 
        IFS="$nl" ; set -f #only split at newlines and don't expand paths
        printf %s\\n ${strings}
    ) 3<<-TEMPLATES
        ${nl=
}
        ${PLACE:="your mother's house"}
        ${EVENT:="the unspeakable."}
        ${ACTION:="heroin"}
        ${RESULT:="succeed."}
        ${strings:="
            I went to ${PLACE} and saw ${EVENT}
            If you do ${ACTION} you will ${RESULT}
        "}
    #END
    TEMPLATES

MIDDLE

This is where you define other functions to call on your print function based on their results...

    EVENT="Disney on Ice."
    _more_important_function() { #...some logic...
        [ $((1+one)) -ne 2 ] && ACTION="remedial mathematics"
            _top_of_script_pr
    }
    _less_important_function() { #...more logic...
        one=2
        : "${ACTION:="calligraphy"}"
        _top_of_script_pr
    }

BOTTOM

You've got it all setup now, so here's where you'll execute and pull your results.

    _less_important_function
    : "${PLACE:="the cemetery"}" 
    _more_important_function
    : "${RESULT:="regret it."}" 
    _less_important_function    

RESULTS

I'll go into why in a moment, but running the above produces the following results:

_less_important_function()'s first run:

I went to your mother's house and saw Disney on Ice.

If you do calligraphy you will succeed.

then _more_important_function():

I went to the cemetery and saw Disney on Ice.

If you do remedial mathematics you will succeed.

_less_important_function() again:

I went to the cemetery and saw Disney on Ice.

If you do remedial mathematics you will regret it.

HOW IT WORKS:

The key feature here is the concept of conditional ${parameter} expansion. You can set a variable to a value only if it is unset or null using the form:

${var_name:=desired_value}

If instead you wish to set only an unset variable, you would omit the :colon and null values would remain as is.

ON SCOPE:

You might notice that in the above example $PLACE and $RESULT get changed when set via parameter expansion even though _top_of_script_pr() has already been called, presumably setting them when it's run. The reason this works is that _top_of_script_pr() is a ( subshelled ) function - I enclosed it in parens rather than the { curly braces } used for the others. Because it is called in a subshell, every variable it sets is locally scoped and as it returns to its parent shell those values disappear.

But when _more_important_function() sets $ACTION it is globally scoped so it affects _less_important_function()'s second evaluation of $ACTION because _less_important_function() sets $ACTION only via ${parameter:=expansion}.

:NULL

And why do I use the leading :colon? Well, the man page will tell you that : does nothing, gracefully. You see, parameter expansion is exactly what it sounds like - it expands to the value of the ${parameter}. So when we set a variable with ${parameter:=expansion} we're left with its value - which the shell will attempt to execute in-line. If it tried to run the cemetery it would just spit some errors at you. PLACE="${PLACE:="the cemetery"}" would produce the same results, but it's also redundant in this case and I preferred that the shell : ${did:=nothing, gracefully}.

It does allow you to do this:

    echo ${var:=something or other}
    echo $var
something or other
something or other

HERE-DOCUMENTS

And by the way - the in-line definition of a null or unset variable is also why the following works:

    <<HEREDOC echo $yo
        ${yo=yoyo}
    HEREDOC
yoyo

The best way to think of a here-document is as an actual file streamed to an input file-descriptor. More or less that's what they are, but different shells implement them slightly differently.

In any case, if you don't quote the <<LIMITER you get it streamed in and evaluated for expansion. So declaring a variable in a here-document can work, but only via expansion which limits you to setting only variables that are not already set. Still, that perfectly suits your needs as you've described them, as your default values will always be set when you call your template print function.

WHY NOT eval?

Well, the example I've presented provides a safe and effective means of accepting parameters. Because it handles scope, every variable within set via ${parameter:=expansion} is definable from outside. So, if you put all this in a script called template_pr.sh and ran:

 % RESULT=something_else template_pr.sh

You'd get:

I went to your mother's house and saw Disney on Ice

If you do calligraphy you will something_else

I went to the cemetery and saw Disney on Ice

If you do remedial mathematics you will something_else

I went to the cemetery and saw Disney on Ice

If you do remedial mathematics you will something_else

This wouldn't work for those variables that were literally set in the script, such as $EVENT, $ACTION, and $one, but I only defined those in that way to demonstrate the difference.

In any case, the acceptance of unknown input into an evaled statement is inherently unsafe, whereas parameter expansion is specifically designed to do it.

Solution 3

You can use placeholders for string templates instead of unexpanded variables. This will get messy pretty quickly. If what you are doing is very template heavy, you may want to consider a language with a real template library.

format_template() {
    changed_str=$1

    for word in $changed_str; do
        if [[ $word == %*% ]]; then
            var="${word//\%/}"
            changed_str="${changed_str//$word/${!var}}"
        fi
    done
}

str1='I went to %PLACE% and saw %EVENT%'
PLACE="foo"
EVENT="bar"
format_template "$str1"
echo "$changed_str"

The downside to the above is that the template variable must be it's own word (eg you can't do "%prefix%foo"). This could be fixed with some modifications, or simply by hard-coding the template variable instead of it being dynamic.

Comments

[Clayton Stanley]

Nice work using a function to delay evaluation, and to avoid the explicit eval call.