Bash : Native way to get rid of quotation around each array member

Question

I read an array from another script. This array needs to put " " around all array members since some members are empty.

in_file=./data
vector=($(./readdata.sh 0 $in_file))
for index in ${!vector[@]}
do
    echo ${vector[index]}
done

The problem is that I have quotations around each output line and I want to get rid of them.

"red"
"blue"
"green"
""
"white"
"black"

must change to:

red
blue
green

white
black

I look for a method which does not use awk, tr, sed or any other pipeline based way. I just want to solve with using native ways such as using parenthesis, different punctuations, ....

Solution 1

This might work:

in_file=./data
vector=($(./readdata.sh 0 $in_file))
for index in ${!vector[@]}
do
    echo ${vector[index]//\"/}
done

Ref: http://www.tldp.org/LDP/abs/html/refcards.html#AEN22828

Solution 2

You'll make your program simpler and more robust if readdata.sh produces newline-delimited data, with no extra quotes.

In your current program, the output of readdata.sh is split at whitespace (so e.g. "a b" results in two array elements "a and b") and each resulting word is interpreted as a wildcard pattern (so e.g. "a * b" results in "a, then the file names in the current directory, and finally b"). See Why does my shell script choke on whitespace or other special characters? for more details.

Bash provides a very simple way of reading newline-delimited data: the mapfile builtin. Since bash executes the right-hand side of a pipeline in a subshell, you can't just write ./readdata.sh 0 "$in_file" | mapfile -t vector, you have to put the use of the variable in a command block, or you can use process substitution:

in_file=./data
mapfile -t vector < <(./readdata.sh 0 "$in_file")
for index in "${!vector[@]}"
do
  echo "$index: ${vector[index]}"
done

If you aren't using the indices, only the elements, a simpler way to iterate over the array is

for element in "${vector[@]}"
do
  echo "$element"
done

Solution 3

As you prefer a one liner,

vector=("${vector[@]//\"/}")

Remember that text substitutions could work for the array as a whole.

Solution 4

If you want to avoid a "hacky" search-and-replace solution, using eval seems to do the job:

vector=( $(eval echo ${vector[@]}) )

This causes the shell to interpret them as quoted arguments to echo, which removes the quotes. Of course, the line above as it is has still problems with white space, but the key point here is the idea of using eval.

View more solutions

Comments

[rahul]

This might help - stackoverflow.com/questions/9733338/…

[ar2015]

@rahul i saw it. it has used sed

[rahul]

look for the accepted answer. They have used the native prefix/sufix feature without sed. sed is presented as an alternative. But the accepted answer does not use sed.

[Anthon]

@ar2015 The accepted answer there starts with a version without sed.

[ar2015]

@rahul the accepted answer only removes prefix or suffix but not both. unless you write 3 lines. That is worse than using pipeline.

[ar2015]

@mikeserv The only reason for asking question was finding an inline way with the minimum change in the code. However thanks for your solution.

[ar2015]

@Anthon That accepted solution uses 3 lines. I am looking for a very simple way. an inline way with maximum readability. I have that code repeated many times in my code. it is very hard to change for all files and using as a template for the next times. My problem is not to obtain the result but how to do it.

[ar2015]

@mikeserv some array members are empty strings. Then they vanish this way. Actually I am putting " " around members for this purpose. avoid putting it around members in the original script is much easier.

[mikeserv]

@ar2015 - (my recommendation) is really not a very good one. A good way to do it with minimal change could be ...vector=($(...| sed 's/"//;s/"$//'))... To only affect not-null members do instead: ...sed 's/"\(..*\)"/\1/'...

[Anthon]

@ar2015 if readability counts then an one liner often (but not always) is the wrong solution. But what is certainly seems wrong to me is that you have to update your code in multiple locations, you should your reused code in a function and only update that one location. How else are you going to do sensible unit testing before using your scripts?

[ar2015]

@Anthon I am writing a C++ code and bash files are to generate automatic codes. I am in hurry of finishing my C++ code. So bash is not my main purpose. Still I have convincing reason to keep my code this way. first of all each part do different job and cannot be encapsulated. second is that this code is not large enough to be put in a different bash file.

[ar2015]

I was looking for a clean way. A readable way. This is the best ever solution.

[Scott - Слава Україні]

You should enclose the entire expression in double quotes (echo "${vector[index]//\"/}") in case the value has leading spaces, trailing spaces, or embedded strings of multiple spaces.  Note that this will remove embedded (internal) quotes as well as the ones at the beginning and end.  (And the ; at the end of the command line is unnecessary.)

[ar2015]

@Scott in my case it does not happen because the scrip puts quotations. but thanks for mentioning.

[Scott - Слава Україні]

I don't understand which part of my comment you're responding to, or how your comment makes sense as a response to mine.

[FloHimself]

@Scott the ; at the end of the command was introduced by pasting the snipped into my terminal. It is not in the OP's snippet. I'll remove it from the snippet. There are several other things that could be optimized, but this wasn't the requirement in this question, so I keep it the way it was posted by the OP.

[Martynas Jusevičius]

How would that work for a 2-dimensional array? Rows are already separated by newlines, but elements need to be quoted because some of them are optional.

[Gilles 'SO- stop being evil']

@MartynasJusevičius If you have quoting, simple shell tools such as read and awk won't cut it. You need specialized tools such as csvtool or python -m csv.