How to concatenate two lines with `sed`?

Question

I am trying to concatenate two lines of iwlist output. My grep and sed commands to remove leading and trailing spaces work fine but I can't get the last sed statement to remove the \n to work. What am I missing?

sudo iwlist wlan0 scanning |
   grep -e ESSID -e Signal |
   sed -e 's/^[ \t]*//;s/[ \t]*$//' |
   sed -e 's/dBm\n/dBm /'

The regex works fine in vim.

Solution

sed is a text tool. By default it outputs one line of output per one line of input. While it's easy to inject newlines into the output (making it more lines), it's not that easy to get rid of newlines from the input because they don't really belong to lines being processed, they separate them.

Excerpt from info sed (emphasis mine):

sed operates by performing the following cycle on each lines of input: first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space. Then commands are executed; […].

There is however N:

N
Add a newline to the pattern space, then append the next line of input to the pattern space. If there is no more input then sed exits without processing any more commands.

This allows you to process lines in pairs, this is what you need. Include N in your last sed like this:

sed -e 'N;s/dBm\n/dBm /'

---

Note it will only work if the entire input logically consists of lines you want to process pair by pair, i.e. this logical structure is OK:

line 1
paired 1
line 2
paired 2
…

while this one is not:

header
line 1
paired 1
line 2
paired 2
…

In this case sed will pair header with line 1, paired 1 with line 2 etc., against your logic. Another bad example:

line 1
paired 1
excessive line
line 2
paired 2
…

The excessive line will be paired with line 2. This will break the logic of consecutive pairs.

So you have to be careful. On the other hand, if only the lines are paired right, you don't have to worry which newlines you're replacing, because you cannot alter ones separating pairs anyway. For this reason including dBm in your pattern is now completely unnecessary. Your last sed may as well be:

sed -e 'N;s/\n/ /'

If you ever need more complex logic, see this answer. It uses sed to concatenate lines according to their content.

Comments

[Cyrus]

Please take a look at: What should I do when someone answers my question?