Find and Replace Inside a Text File from a Bash Command
Solution 1
The easiest way is to use sed (or perl):
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
Which will invoke sed to do an in-place edit due to the -i
option. This can be called from bash.
If you really really want to use just bash, then the following can work:
while IFS='' read -r a; do
echo "${a//abc/XYZ}"
done < /tmp/file.txt > /tmp/file.txt.t
mv /tmp/file.txt{.t,}
This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name. (For robustness and security, the temporary file name should not be static or predictable, but let's not go there.)
For Mac users:
sed -i '' 's/abc/XYZ/g' /tmp/file.txt
(See the comment below why)
Solution 2
File manipulation isn't normally done by Bash, but by programs invoked by Bash, e.g.:
perl -pi -e 's/abc/XYZ/g' /tmp/file.txt
The -i
flag tells it to do an in-place replacement.
See man perlrun
for more details, including how to take a backup of the original file.
Solution 3
I was surprised when I stumbled over this...
There is a replace
command which ships with the "mysql-server"
package, so if you have installed it try it out:
# replace string abc to XYZ in files
replace "abc" "XYZ" -- file.txt file2.txt file3.txt
# or pipe an echo to replace
echo "abcdef" |replace "abc" "XYZ"
See man replace
for more on this.
Solution 4
This is an old post but for anyone wanting to use variables as @centurian said the single quotes mean nothing will be expanded.
A simple way to get variables in is to do string concatenation since this is done by juxtaposition in bash the following should work:
sed -i -e "s/$var1/$var2/g" /tmp/file.txt
Solution 5
Bash, like other shells, is just a tool for coordinating other commands. Typically you would try to use standard UNIX commands, but you can of course use Bash to invoke anything, including your own compiled programs, other shell scripts, Python and Perl scripts etc.
In this case, there are a couple of ways to do it.
If you want to read a file, and write it to another file, doing search/replace as you go, use sed:
sed 's/abc/XYZ/g' <infile >outfile
If you want to edit the file in place (as if opening the file in an editor, editing it, then saving it) supply instructions to the line editor 'ex'
echo "%s/abc/XYZ/g
w
q
" | ex file
Example is like vi
without the fullscreen mode. You can give it the same commands you would at vi
's :
prompt.
Comments
-
Ash almost 2 years
What's the simplest way to do a find and replace for a given input string, say
abc
, and replace with another string, sayXYZ
in file/tmp/file.txt
?I am writting an app and using IronPython to execute commands through SSH — but I don't know Unix that well and don't know what to look for.
I have heard that Bash, apart from being a command line interface, can be a very powerful scripting language. So, if this is true, I assume you can perform actions like these.
Can I do it with bash, and what's the simplest (one line) script to achieve my goal?
-
Tomato over 15 yearsExcept that invoking mv is pretty much as 'non Bash' as using sed. I nearly said the same of echo, but it's a shell builtin.
-
Panky over 12 yearsThe -i argument for sed doesn't exist for Solaris (and I would think some other implementations) however, so keep that in mind. Just spent several minutes figuring that out...
-
checksum over 11 yearsNote to self: about regular expression of
sed
:s/..../..../ - Substitute
and/g - Global
-
Martin L. over 11 yearsPs: For me, this variant is 1/3 faster:
for a in `cat hosts.txt` ; do echo ${a//abc/XYZ} ; done > hosts.txt.t
) -
Austin over 11 yearsNote for Mac users who get an
invalid command code C
error... For in-place replacements, BSDsed
requires a file extension after the-i
flag because it saves a backup file with the given extension. For example:sed -i '.bak' 's/find/replace/' /file.txt
You can skip the backup by using an empty string like so:sed -i '' 's/find/replace/' /file.txt
-
Tomato over 11 yearsIn your (2) -- you can do
sed -e "s/$x/$y/"
, and it will work. Not the double quotes. It can get seriously confusing if the strings in the variables themselves contain characters with special meaning. For example if x="/" or x="\". When you hit these issues, it probably means you should stop trying to use the shell for this job. -
CMCDragonkai over 10 yearsCan you show a more complex example. Something like replacing "chdir /blah" with "chdir /blah2". I tried
perl -pi -e 's/chdir (?:\\/[\\w\\.\\-]+)+/chdir blah/g' text
, but I keep getting an error with Having no space between pattern and following word is deprecated at -e line 1. Unmatched ( in regex; marked by <-- HERE in m/(chdir)( )( <-- HERE ?:\\/ at -e line 1. -
Boris D. Teoharov almost 10 yearsTip: If you want case insensitive repalce use
s/abc/XYZ/gi
-
Philip Whitehouse almost 10 yearsTwo things are possible here: a)
replace
is a useful independent tool and the MySQL folks should release it separately and depend on it b)replace
requires some bit of MySQL o_O Either way, installing mysql-server to get replace would be the wrong thing to do :) -
paul almost 8 yearsonly works for mac? in my ubuntu I centos that command does not exist
-
tripleee over 7 yearsThis is an excellent answer to "how do I accidentally all the files in all subdirectories, too" but that does not seem to be what is asked here.
-
Phius over 7 yearsThat's because you don't have
mysql-server
package installed. As pointed by @rayro,replace
is part of it. -
Brian Hannay almost 7 yearsAny tips for using this with tab characters? For some reason my script doesn't find anything with tabs after changing from sed with lots of escaping to this method.
-
johnraff over 6 yearsIf you want to put a tab in the string you're replacing, you can do so with Bash's "dollared single quotes" syntax, so a tab is represented by $'\t', and you can do $ echo 'tab'$'\t''separated' > testfile; $ file_contents=$(<testfile); $ echo "${file_contents//$'\t'/TAB}"; tabTABseparated `
-
Steven Vachon over 6 years"Warning: replace is deprecated and will be removed in a future version."
-
JLB over 6 yearsAs of 2018, it is included in the MariaDB package for Ubuntu.
-
neuro_sys about 6 yearsNot as powerful, but I have a simple "script" here that does the same in case downloading a MySQL or MariaDB is overkill: gist.github.com/neuro-sys/3bf00b6cf28a93e07e44
-
Maor about 6 yearsBe careful not to run the REPLACE command on Windows! On Windows the REPLACE command is for a fast replication of files. Not relevant to this discussion.
-
vineeshvs about 5 yearsWanted to do the replacements interactively. Hence tried "vim -esnc '%s/foo/bar/gc|:wq' file.txt". But the terminal is stuck now. How shall we make the replacements interactively without the bash shell behaving weirdly.
-
glerYbo about 5 yearsTo debug, add
-V1
, to force quit, usewq!
. -
glerYbo almost 5 yearsThis syntax won't work for BSD version of
sed
, usesed -i''
instead. -
Alfonso Santiago over 4 years@CMCDragonkai Check this answer: stackoverflow.com/a/12061491/2730528
-
Royi about 4 yearsAnyway to disable the
RegEx
functionality and use them as only strings (Replace only the whole word)? -
gvee almost 4 years
-i
isn't "ignore case", it's-i[SUFFIX], --in-place[=SUFFIX]
(edit files in place (makes backup if SUFFIX supplied)) -
Tomato almost 4 years@awattar Just do them one at a time in a
for
loop. -
awattar almost 4 yearsIs there an option to use it together with globbing as a one liner?
-
Tomato almost 4 yearsNot as far as I know.
for f in report*.txt; do echo "%s/abc/XYZ/g \n w \n q \n" | ex file; done
is clean and simple. Why put functionality intoex
that the shell already has? -
Tomato almost 4 years(Or, if your problem outgrows the shell, use Python/Perl/whatever)
-
Timo over 3 yearsI use this in a file:
sed "s/\"$li\"/- [x]\"\${li:5}\"/" $dat
ang get sed unterminated `s' command -
Timo over 3 yearsProblem solved, well, ... $li comes from a file line, so there is e.g. \n and the error is there. So either awk or another language like python comes.
-
GoodJeans almost 3 yearsNote to everyone s/ is for search /g is for global hence forth search globally
-
user4463876 over 2 years@Royi I'm not sure if I understood your question, as regex can replace whole words only. Simply type a word between / / slash characters. You must escape some characters with backslash, such as $, *, ", ', \.