Why am I getting some extra, weird characters when making a file from grep output?
printf "%s\n" *.file_ID > my_list.txt
GREP_OPTIONS environment variable contains
--color=always, which causes the output to be stuffed with control characters, even when piped to a file.
Even better, don't use
ls *.file_ID > my_list.txt
usually it is automatically =>
if it pipes into a csv file that looks like this
you would have to type this in the terminal:
grep --color=auto "your regex" > example.csv
if you want it to be a permanent situation where you do not have to type "
--color=auto" every time, type this in the terminal:
more info: https://linuxcommando.blogspot.com/2007/10/grep-with-color-output.html
This should take care of it (assuming GNU find and no directory traversing):
find . -maxdepth 1 -type f -name "*.file_ID" -printf "%f\n" > my_list.txt
~> ls *file_ID* a.file_ID b.file_ID c.file_ID ~> find . -maxdepth 1 -type f -name "*.file_ID" -printf "%f\n" > my_list.txt ~> cat my_list.txt a.file_ID b.file_ID c.file_ID
As far as the "^[[00m" characters, check your ls options:
~> alias -p | grep "ls="
You may get something like:
alias ls='/bin/ls $LS_OPTIONS'
If so, check env for this:
~> env | grep LS_OP LS_OPTIONS=-N --color=tty -T 0
The character string you're referencing is used to turn off colors, so your shell likely has been set to show colors. Removing and/or changing the ls alias should resolve it.
Related videos on Youtube
jake9115Updated on September 18, 2022
jake9115 8 months
I am doing a very basic command that never gave me trouble in the past, but is inexplicably returning undesired characters now.
I am in BASH on linux, and simply want to search through a directory and make a file containing filenames that match a pattern:
ls | grep "*.file_ID" > my_list.txt
...This works fine, and if I cat the data:
cat my_list.txt seriesA.file_ID seriesB.file_ID seriesC.file_ID
However, when I try to feed this file into downstream processes, I keep getting a weird errors, as if the file isn't properly formatted as a list of file names. When I open the file in vim to reveal any unnecessary characters, I find the file actually looks like this:
vi my_list.txt ^[[00mseriesA.file_ID^[[00m ^[[00mseriesB.file_ID^[[00m ^[[00mseriesC.file_ID^[[00m
For some reason, every line is started and ended with the characters
^[[00m. If I delete these characters, all of the downstream processes work fine. However, I need to have my scripts automatically make such a file list, so I can't keep going in and manually deleting these chars.
Does anyone know what is producing the
^[[00mcharacters? I don't have any idea where they are coming from, and need a to be able to generate files without them.
chepner almost 10 yearsOr in the event of a large number of matching files that could overflow the command line,
find . -name '*.file_ID' > my_list.txt(although, some extra work is needed to remove the leading './' that
findwill produce for each file).
twalberg almost 10 yearsOr, even better...
echo *.file_ID > ..., or
for f in *.file_ID; do ....
Ansel Halliburton almost 10 yearsThis does not answer the question
Ansel Halliburton almost 10 yearsYou're right, the extra characters are not produced by
grep, but by
ls, which is why there is no problem with
grep -o(as per the comments above)
glenn jackman almost 10 years@Pumbaa80, the literal answer to the question is use
command ls, but the sensible answer is "don't do that"
Stuart over 9 years@glenn jackman, The question was "Why...". Can you explain the reason this happens so others can apply the general solution?
glenn jackman over 9 yearsMost likely the
lscommand is actually an alias or function that adds the
--coloroption. So the odd chars are actually terminal color codes.
lsoutput is intended for human eyes, not as input to other programs.'