How to check if sed has changed a file

Question

I am trying to find a clever way to figure out if the file passed to sed has been altered successfully or not.

Basically, I want to know if the file has been changed or not without having to look at the file modification date.

The reason why I need this is because I need to do some extra stuff if sed has successfully replaced a pattern.

I currently have:

    grep -q $pattern $filename
    if [ $? -eq 0 ]
    then
        sed -i s:$pattern:$new_pattern: $filename
                # DO SOME OTHER STUFF HERE
    else
        # DO SOME OTHER STUFF HERE
    fi

The above code is a bit expensive and I would love to be able to use some hacks here.

Solution 1

You could use awk instead:

awk '$0 ~ p { gsub(p, r); t=1} 1 END{ exit (!t) }' p="$pattern" r="$repl"

I'm ignoring the -i feature: you can use the shell do do redirections as necessary.

Sigh. Many comments below asking for basic tutorial on the shell. You can use the above command as follows:

if awk '$0 ~ p { gsub(p, r); t=1} 1 END{ exit (!t) }' \
        p="$pattern" r="$repl" "$filename" > "${filename}.new"; then
    cat "${filename}.new" > "${filename}"
    # DO SOME OTHER STUFF HERE
else
    # DO SOME OTHER STUFF HERE
fi

It is not clear to me if "DO SOME OTHER STUFF HERE" is the same in each case. Any similar code in the two blocks should be refactored accordingly.

Solution 2

A bit late to the party but for the benefit of others, I found the 'w' flag to be exactly what I was looking for.

sed -i "s/$pattern/$new_pattern/w changelog.txt" "$filename"
if [ -s changelog.txt ]; then
    # CHANGES MADE, DO SOME STUFF HERE
else
    # NO CHANGES MADE, DO SOME OTHER STUFF HERE
fi

changelog.txt will contain each change (ie the changed text) on it's own line. If there were no changes, changelog.txt will be zero bytes.

A really helpful sed resource (and where I found this info) is http://www.grymoire.com/Unix/Sed.html.

Solution 3

I believe you may find these GNU sed extensions useful

t label

If a s/// has done a successful substitution since the last input line
was read and since the last t or T command, then branch to label; if
label is omitted, branch to end of script.

and

q [exit-code]

Immediately quit the sed script without processing any more input, except 
that if auto-print is not disabled the current pattern space will be printed. 
The exit code argument is a GNU extension.

It seems like exactly what are you looking for.

Solution 4

This might work for you (GNU sed):

sed -i.bak '/'"$old_pattern"'/{s//'"$new_pattern"'/;h};${x;/./{x;q1};x}' file || echo changed

Explanation:

• /'"$old_pattern"'/{s//'"$new_pattern"'/;h} if the pattern space (PS) contains the old pattern, replace it by the new pattern and copy the PS to the hold space (HS).
• ${x;/./{x;q1};x} on encountering the last line, swap to the HS and test it for the presence of any string. If a string is found in the HS (i.e. a substitution has taken place) swap back to the original PS and exit using the exit code of 1, otherwise swap back to the original PS and exit with the exit code of 0 (the default).

Solution 5

You can diff the original file with the sed output to see if it changed:

sed -i.bak s:$pattern:$new_pattern: "$filename"
if ! diff "$filename" "$filename.bak" &> /dev/null; then
  echo "changed"
else
  echo "not changed"
fi
rm "$filename.bak"

View more solutions

Comments

[William Pursell]

Any time you are trying to do something "clever", you probably shouldn't do it.

[Henry Gomersall]

@WilliamPursell because the world was built with stupid inventions.

[AlvaroGMJ]

What about writing the changes to a new file, and then diff-ing the original and the generated? By the way, shouldn't sed always replace the pattern if grep found it before?

[breakdown1986]

@AlvaroGMJ Yeah that is the idea, if grep found a pattern i know for sure that sed will replace it, so i can do my extra stuff right after it.

[Lev Levitsky]

Can SOME OTHER STUFF be done with sed, too?

[breakdown1986]

The SOME OTHER STUFF is mostly creating logs. So i am guessing no.

[AlvaroGMJ]

@breakdown1986 then (unless the "replaced succesfully" bit indicates that you think it might fail) I don't see the point in the question: you are already sure that SED will replace the pattern when you enter the IF block, and you are using GREP, which is the faster filter in the UNIX world :) (maybe you can add the -F flag, but usually the differences are not significant). Just check sed's exit value and that's it. If you think SED might fail and corrupt your file, write the output to another file and replace it only after checking the return value.

[chepner]

sed's exit code does not reflect whether any matches were found.

[Gabriel Staples]

Here's a full function to count replacements with grep then perform them with sed. After much consternation getting to this point, I am very pleased with the result. The screenshot shows the output, which is very nice, as it shows color matching of each replacement, counts the number of lines replaced, and counts the number of replacements: stackoverflow.com/a/61238414/4561887

[chepner]

Is diff going to be less expensive than the grep he's trying to replace?

[AlvaroGMJ]

You can't do in place modifications using only shell redirection. command file > file doesn't work (file is truncated at the moment the redirection is applied, which happens before the command is started)

[breakdown1986]

Hey, thank you for the solution, but I think doing a diff on each file i am gonna search through might be a bit cpu intensive. What do you think ?

[perreal]

I think so too, this is not really efficient

[chepner]

awk '...' $filename > tmp.txt; mv tmp.txt $filename. sed -i just hides the details of the temp file from you.

[William Pursell]

@AlvaroGMJ: you cannot do in-place modifications using sed -i, but you certainly can do it with shell redirections. But you are right, you cannot do it with cmd file > file.

[Robin Winslow]

I don't understand how to use this

[Dan Dascalescu]

This overwrites the changelog.txt file. Any idea how to append to it instead?

[aureliandevel]

Not directly, however some out-of-process file manipulation is certainly doable.

[qodeninja]

Can you give a full example of the solution using awk? It's not clear what you mean about ignoring the -i feature or what sed has to do with this. Your code as it is doesnt appear to work

[Tomáš Zato]

I too do not understand. Hostmaster could you please explain where do we put this options? I'm running sed within an install script and I need to raise warning if sed doesn't manage to change configuration file.

[Tomáš Zato]

I find the overwriting convenient... After every sed I can check without keeping in mind to delete the file.

[Justin Moh]

Sed - An Introduction and Tutorial by Bruce Barnett is a treasure chest but it's way too long. And every time I re-read it I learn new stuffs.

[ceving]

diff shows you what is different. If you just want to know if there is any difference cmp is sufficient. Also [ $? -ne 0] is useless. Just put the command into the if condition: if cmp "$filename" "$filename".bak; then.

[tripleee]

In case the reader is unfamiliar with shell quoting rules, maybe point out that the text inside the single quotes will not have any shell variables expanded; regard this as pseudocode.

[aureliandevel]

Fair call. Changes made.

[Benjamin W.]

That's not always going to work after the latest edit - I think it would better be 's:'"$pattern"':'"$new_pattern"':w changelog.txt' so you still (double) quote shell parameters. Without any quoting at all, the command will break if there is a space or a shell special character in either parameter.

[aureliandevel]

Yeah, I edited again after actually running the code. This revision works for me in bash. I'd actually made the edit before the latest comment from Benjamin W and then found that comment waiting for me. C'est la vie.

[pgr]

How to apply this with /g flag? I mean, how can I combine the /w with the /g?

[orodbhen]

@pgr Look at this post, which covers the same topic, but in more detail.

[Igor Dvorzhak]

How to use t and q options with sed command explained here: askubuntu.com/a/1036918/250399. TLDR: sed -i 's/orig/repl/; t; q1' file.txt

[Nino DELCEY]

Can you expend this answer ? What is "!t" ? What is "exit" ?

[William Pursell]

@NinoLenoska !t is "not t". eg, if t is zero, then !t is 1. If t is non-zero, !t is zero. exit is the function that causes awk to exit.

[quezak]

Important: looks like this works only for one-line patterns. For sed-ing a whole file, where the replacement is done somewhere in the middle, after sed parses the first line and the replacement is not found there, it will just print the first line unchanged and exit.

[jaam]

@DanDascalescu Add cat changelog.txt >> changes.txt

[milahu]

sponge from moreutils can help to avoid the tempfile: awk ... $filename | sponge $filename

[milahu]

also works with s/search/replace/ w /dev/stdout - no need to create temporary files : ) see the solution by acecilia

[milahu]

limitation: can handle only one substitution. so this does not work: sed -i 's,a,b, w /dev/stdout ; s,b,a, w /dev/stdout' input.txt. see my workaround

[capr]

I can't believe Kernighan is proud of this

[Max_Payne]

@pgr sed -i "s/a/b/g w /dev/stdout" file