Converting a Bash array into a delimited string

Question

I would like to know the following;

1. Why the given non-working example doesn't work.
2. If there are any other cleaner methods than those given in working example.

Non-working example

> ids=(1 2 3 4);echo ${ids[*]// /|}
1 2 3 4
> ids=(1 2 3 4);echo ${${ids[*]}// /|}
-bash: ${${ids[*]}// /|}: bad substitution
> ids=(1 2 3 4);echo ${"${ids[*]}"// /|}
-bash: ${"${ids[*]}"// /|}: bad substitution

Working example

> ids=(1 2 3 4);id="${ids[@]}";echo ${id// /|}
1|2|3|4
> ids=(1 2 3 4); lst=$( IFS='|'; echo "${ids[*]}" ); echo $lst
1|2|3|4

In context, the delimited string to be used in a sed command for further parsing.

Solution 1

Because parentheses are used to delimit an array, not a string:

ids="1 2 3 4";echo ${ids// /|}
1|2|3|4

Some samples: Populating $ids with two strings: a b and c d

ids=("a b" "c d")

echo ${ids[*]// /|}
a|b c|d

IFS='|';echo "${ids[*]}";IFS=$' \t\n'
a b|c d

... and finally:

IFS='|';echo "${ids[*]// /|}";IFS=$' \t\n'
a|b|c|d

Where array is assembled, separated by 1st char of $IFS, but with space replaced by | in each element of array.

When you do:

id="${ids[@]}"

you transfer the string build from the merging of the array ids by a space to a new variable of type string.

Note: when "${ids[@]}" give a space-separated string, "${ids[*]}" (with a star * instead of the at sign @) will render a string separated by the first character of $IFS.

what man bash says:

man -Len -Pcol\ -b bash | sed -ne '/^ *IFS /{N;N;p;q}'
   IFS    The  Internal  Field  Separator  that  is used for word splitting
          after expansion and to split  lines  into  words  with  the  read
          builtin command.  The default value is ``<space><tab><newline>''.

Playing with $IFS:

declare -p IFS
declare -- IFS=" 
"
printf "%q\n" "$IFS"
$' \t\n'

Literally a space, a tabulation and (meaning or) a line-feed. So, while the first character is a space. the use of * will do the same as @.

But:

{
    IFS=: read -a array < <(echo root:x:0:0:root:/root:/bin/bash)
    
    echo 1 "${array[@]}"
    echo 2 "${array[*]}"
    OIFS="$IFS" IFS=:
    echo 3 "${array[@]}"
    echo 4 "${array[*]}"
    IFS="$OIFS"
}
1 root x 0 0 root /root /bin/bash
2 root x 0 0 root /root /bin/bash
3 root x 0 0 root /root /bin/bash
4 root:x:0:0:root:/root:/bin/bash

Note: The line IFS=: read -a array < <(...) will use : as separator, without setting $IFS permanently. This is because output line #2 present spaces as separators.

Solution 2

You can use printf too, without any external commands or the need to manipulate IFS:

ids=(1 2 3 4)                     # create array
printf -v ids_d '|%s' "${ids[@]}" # yields "|1|2|3|4"
ids_d=${ids_d:1}                  # remove the leading '|'

Solution 3

Your first question is already addressed in F. Hauri's answer. Here's canonical way to join the elements of an array:

ids=( 1 2 3 4 )
IFS=\| eval 'lst="${ids[*]}"'

Some people will cry out loud that eval is evil, yet it's perfectly safe here, thanks to the single quotes. This only has advantages: there are no subshells, IFS is not globally modified, it will not trim trailing newlines, and it's very simple.

Solution 4

An utility function to join arguments array into a delimited string:

#!/usr/bin/env bash

# Join arguments with delimiter
# @Params
# $1: The delimiter string
# ${@:2}: The arguments to join
# @Output
# >&1: The arguments separated by the delimiter string
array::join() {
  (($#)) || return 1 # At least delimiter required
  local -- delim="$1" str IFS=
  shift
  str="${*/#/$delim}" # Expands arguments with prefixed delimiter (Empty IFS)
  printf '%s\n' "${str:${#delim}}" # Echo without first delimiter
}

declare -a my_array=( 'Paris' 'Berlin' 'London' 'Brussel' 'Madrid' 'Oslo' )

array::join ', ' "${my_array[@]}"
array::join '*' {1..9} | bc # 1*2*3*4*5*6*7*8*9=362880 Factorial 9

declare -a null_array=()

array::join '== Ultimate separator of nothing ==' "${null_array[@]}"

Output:

Paris, Berlin, London, Brussel, Madrid, Oslo
362880

Now with Bash 4.2+'s nameref variables, using sub-shells output capture is no longer needed.

#!/usr/bin/env bash

if ((BASH_VERSINFO[0] < 4 || (BASH_VERSINFO[0] == 4 && BASH_VERSINFO[0] < 2)))
then
  printf 'Bash version 4.2 or above required for nameref variables\n' >&2
  exit 1
fi

# Join arguments with delimiter
# @Params
# $1: The variable reference to receive the joined output
# $2: The delimiter string
# ${@:3}: The arguments to join
# @Output
array::join_to() {
  (($# > 1)) || return 1 # At least nameref and delimiter required
  local -n out="$1"
  local -- delim="$2" str IFS=
  shift 2
  str="${*/#/$delim}" # Expands arguments with prefixed delimiter (Empty IFS)
  # shellcheck disable=SC2034 # Nameref variable
  out="${str:${#delim}}" # Discards prefixed delimiter
}

declare -g result1 result2 result3
declare -a my_array=( 'Paris' 'Berlin' 'London' 'Brussel' 'Madrid' 'Oslo' )

array::join_to result1 ', ' "${my_array[@]}"
array::join_to result2 '*' {1..9}
result2=$((result2)) # Expands arythmetic expression

declare -a null_array=()

array::join_to result3 '== Ultimate separator of nothing ==' "${null_array[@]}"

printf '%s\n' "$result1" "$result2" "$result3"

View more solutions

Comments

[Gilles Quenot]

${${ids[*]}// /|} is a syntax error, that's all. Dunno what you're trying to achieve here.

[koola]

Trying to achieve variable substitution in 1 hop, it was never going to work...

[codeforester]

Related: How can I join elements of an array in Bash?.

[koola]

So it's both a typeof and variable substitution error given ${ is expecting a var of type string but receives neither. Thank you for the detailed explanation.

[Jonathan Y.]

One can also skip the string assignment and still convert an array into a delimited string with an arbitrary delimiter, not necessarily a single character, using printf '%smyDelim' "${array[@]}". The last instance of the delimiter myDelim can be removed by piping into sed -e 's/myDelim$//', but that's cumbersome. Better ideas?

[F. Hauri - Give Up GitHub]

@JonathanY. If so, use printf -v myVar '%smyDelim' "${array[@]}"; myVar="${myVar%myDelim}" instead of fork to sed

[Jonathan Y.]

Except that doesn't skip a variable assignment, e.g., when using printf to provide input arguments for another executable. I get that assigning a variable is probably quicker and cheaper; it just feels less elegant.

[Cometsong]

This was simplest for me, as I had >1 character string to put in between the array elements, and in my case I didn't need to remove the "extra" either. Works delightfully! printf -v strng "'%s',\n" ${thearray[*]} :)

[TheDudeAbides]

Very handy, thanks! Perhaps join would be a more appropriate name, though, in line with how, e.g., Perl and Python's join functions work?

[Léa Gris]

@TheDudeAbides Could not name it directly join because this conflict with an existing command name that join lines of files.

[TheDudeAbides]

*facepalm* - ah yes, of course. I forgot about that.

[JuroOravec]

Agree, simplest to use. Additionally, it's also the user-friendlier option if you are using IDE with syntax highlighting, which won't pick up eval'd syntax as in gniourf_gniourf's answer.

[Swepter]

@F.Hauri you must have double quotes around myDelim. You also must have the $ symbol . Lite this "%s$myDelim"

[F. Hauri - Give Up GitHub]

@Swepter In my sample, myDelim is not a variable, but a fixed string, in answer to previous comment from @Jonathan.

[F. Hauri - Give Up GitHub]

Nice tool! permitting variable length delimiter! But consider using nameref to pass result as a variable in order to prevent forks...

[Léa Gris]

@F.Hauri now with a nameref version