Environment variable substitution in sed
Solution 1
Your two examples look identical, which makes problems hard to diagnose. Potential problems:
You may need double quotes, as in
sed 's/xxx/'"$PWD"'/'
$PWD
may contain a slash, in which case you need to find a character not contained in$PWD
to use as a delimiter.
To nail both issues at once, perhaps
sed 's@xxx@'"$PWD"'@'
Solution 2
In addition to Norman Ramsey's answer, I'd like to add that you can double-quote the entire string (which may make the statement more readable and less error prone).
So if you want to search for 'foo' and replace it with the content of $BAR, you can enclose the sed command in double-quotes.
sed 's/foo/$BAR/g'
sed "s/foo/$BAR/g"
In the first, $BAR will not expand correctly while in the second $BAR will expand correctly.
Solution 3
Another easy alternative:
Since $PWD
will usually contain a slash /
, use |
instead of /
for the sed statement:
sed -e "s|xxx|$PWD|"
Solution 4
You can use other characters besides "/" in substitution:
sed "s#$1#$2#g" -i FILE
Solution 5
一. bad way: change delimiter
sed 's/xxx/'"$PWD"'/'
sed 's:xxx:'"$PWD"':'
sed 's@xxx@'"$PWD"'@'
maybe those not the final answer,
you can not known what character will occur in $PWD
, /
:
OR @
.
if delimiter char in $PWD, they will break the expression
the good way is replace(escape) the special character in $PWD
.
二. good way: escape delimiter
for example:
try to replace URL
as $url (has :
/
in content)
x.com:80/aa/bb/aa.js
in string $tmp
<a href="URL">URL</a>
A. use /
as delimiter
escape /
as \/
in var (before use in sed expression)
## step 1: try escape
echo ${url//\//\\/}
x.com:80\/aa\/bb\/aa.js #escape fine
echo ${url//\//\/}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//\//\/}"
x.com:80\/aa\/bb\/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s/URL/${url//\//\\/}/"
<a href="x.com:80/aa/bb/aa.js">URL</a>
echo $tmp | sed "s/URL/${url//\//\/}/"
<a href="x.com:80/aa/bb/aa.js">URL</a>
OR
B. use :
as delimiter (more readable than /
)
escape :
as \:
in var (before use in sed expression)
## step 1: try escape
echo ${url//:/\:}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//:/\:}"
x.com\:80/aa/bb/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s:URL:${url//:/\:}:g"
<a href="x.com:80/aa/bb/aa.js">x.com:80/aa/bb/aa.js</a>
Related videos on Youtube
Comments
-
RomanM over 2 years
If I run these commands from a script:
#my.sh PWD=bla sed 's/xxx/'$PWD'/' ... $ ./my.sh xxx bla
it is fine.
But, if I run:
#my.sh sed 's/xxx/'$PWD'/' ... $ ./my.sh $ sed: -e expression #1, char 8: Unknown option to `s'
I read in tutorials that to substitute environment variables from shell you need to stop, and 'out quote' the
$varname
part so that it is not substituted directly, which is what I did, and which works only if the variable is defined immediately before.How can I get sed to recognize a
$var
as an environment variable as it is defined in the shell?-
derobert about 15 years$PWD contains a / which is ending the substitute command.
-
RomanM about 15 years@derobert: tnx. One of the solutions addresses this ...
-
bobbogo almost 12 yearsUse
set -x
in the shell to get the shell to echo each command just before it executes them. This can clear up a lot of confusion. (Also, I often useset -u
to make de-referencing unset variables a hard error. (Seeset -e
too.)) -
ThorSummoner over 5 yearsI was hoping to find a way for sed to handle the environment variables as not to leak the values into the process table, seems like sed is the wrong tool for installing secrets according to all the answers in this thread
-
-
RomanM about 15 yearsbut PWD is defined by the shell ... if i go echo $PWD i get the pwd
-
RomanM about 15 yearsbut, then what character can i use i know for sure will not appear in a path name ?
-
Norman Ramsey about 15 yearsYou can have several candidates like @#%! and check with a case expression to find if $PWD has them. E.g., case "$PWD" of @) ;; *) delim="@" ;; esac; repeat until $delim is not empty.
-
einpoklum over 10 yearsSo, how do you escape the environment variables?
-
Eddie over 10 yearsThe selected answer describes a workaround ... don't use slash for the delimiter
-
Thales Ceolin about 10 yearsThere's another alternative instead of using double quotes. See my answer below.
-
Vladislavs Dovgalecs over 9 yearsThis is cleaner than messing with double quotes, single quotes etc.
-
blubberdiblub over 9 yearsWhich will become a problem as soon as your current working directory contains that other character.
-
Dan over 9 yearsYou said "an alternative to using double quotes" and yet your example uses double quotes?
-
Boyd Hemphill over 8 yearsIn my specific case, $2 was a file path, so
sed
was barfing due to interpreting the/
in the contents of $2. This was exactly what I needed to get past it. Thanks fora great tip! -
izikandrw almost 8 yearsthis is what I had to use to get the environment variable to expand correctly in this command: sed -i "s/127.0.0.1/127.0.0.1 localhost $HOSTNAME/" hosts
-
mjp over 6 yearsThis is neat but it is not working when you want to do some more complex substitutions, such as
"2,$s/^/$foo/"
as$s
gets interpreted as a variable too and it should not. -
Christian over 4 yearsI guess the point is it doesn't use double-quotes directly around
$PWD
...? -
andov over 4 yearsI was struggling to get this working in Azure DevOps YAML pipelines. This comment helped me to successfully use this trick to tokenize some configuration files.
-
Guchelkaben almost 4 yearsYou can use another delimiter for sed. You dont need to use / you can use , as well if your environment variable is an url.
-
Max Waterman almost 4 yearsWhat if the string contains a \ followed by an n - how to stop sed from converting that into a single newline character?
-
MoRe over 3 yearsThis alone didn't work as I still needed a different delimiter for paths like the other answers suggest.
-
NameOfTheRose over 3 years@mjp, what you say is correct but I think one may write it as
"2,$ s/^/$foo/"
-
bananaforscale almost 3 yearsNice clean pattern but the -e (--expression) is not needed here, since this example is not stringing several expressions together.