How to concatenate all lines from a file in Bash?

Question

I have a file csv :

data1,data2,data2
data3,data4,data5
data6,data7,data8

I want to convert it to (Contained in a variable):

variable=data1,data2,data2%0D%0Adata3,data4,data5%0D%0Adata6,data7,data8

My attempt :

data=''
cat csv | while read line
do
data="${data}%0D%0A${line}"
done
echo $data  # Fails, since data remains empty (loop emulates a sub-shell and looses data)

Please help..

Solution 1

In bash,

data=$(
while read line
do
  echo -n "%0D%0A${line}"
done < csv)

In non-bash shells, you can use `...` instead of $(...). Also, echo -n, which suppresses the newline, is unfortunately not completely portable, but again this will work in bash.

Solution 2

Simpler to just strip newlines from the file:

tr '\n' '' < yourfile.txt > concatfile.txt

Solution 3

Some of these answers are incredibly complicated. How about this.

 data="$(xargs printf ',%s' < csv | cut -b 2-)"

or

 data="$(tr '\n' ',' < csv | cut -b 2-)"

Too "external utility" for you?

IFS=$'\n', read -d'\0' -a data < csv

Now you have an array! Output it however you like, perhaps with

data="$(tr ' ' , <<<"${data[@]}")"

Still too "external utility?" Well fine,

data="$(printf "${data[0]}" ; printf ',%s' "${data[@]:1:${#data}}")"

Yes, printf can be a builtin. If it isn't but your echo is and it supports -n, use echo -n instead:

data="$(echo -n "${data[0]}" ; for d in "${data[@]:1:${#data[@]}}" ; do echo -n ,"$d" ; done)"

Okay, now I admit that I am getting a bit silly. Andrew's answer is perfectly correct.

Solution 4

I would much prefer a loop:

for line in $(cat file.txt); do echo -n $line; done

Note: This solution requires the input file to have a new line at the end of the file or it will drop the last line.

Solution 5

Another short bash solution

variable=$(
  RS=""
  while read line; do
    printf "%s%s" "$RS" "$line"
    RS='%0D%0A'
  done < filename
)

View more solutions