Need some sed magic: Moving marked lines to the beginning of a file

Question

I find myself in dire need of some sed magic (I really need to sit myself down and learn this). I have a file with lots of lines. After reviewing it, I've marked some lines by adding a star (*) to the beginning of the line.

What I would like to do with some sedtrickery (if possible), is to move all the marked-lines to the beginning (or the end – I'm not picky) of the file, so they form a single block. The other lines (unmarked) should be left undisturbed.

How can I do something like this with sed? I know sed has some buffers for moving text...

Solution 1

is sed essential? if you don't mind two passes through the source file, this is easily done with grep.

e.g.

grep '^\*' input > outputfile
grep -v '^\*' input >> outputfile

Solution 2

Depending on the size of the file you can do:

sed '/^*/!H;//p;$!d;g;s/\n//'

That stacks in Hold space lines which do not match /^*/. Those that do match are printed as they occur in input. Then all lines which are !not the $last are deleted from output. On the last line we get hold space by overwriting pattern space, then the first \newline character is s/\n//ubstituted away because the first Held line will result in one extra every time.

That requires a big buffer though as it has store all of those lines in Hold space. This, on the other hand...

sed '/^*/p;$!d;g;r file' <file    |
sed -e '1,/^$/{/./p;d' -e '};/^*/d'

...does not have that requirement.

The first sed prints only /^*/ matching lines until the $last line, at which time it prints a blank line then reads out the entire input file all over again.

The second sed works first on a line range from the first line through the first blank line printing all lines which match at least a single character then deleting the lot. After it encounters the first blank line, it then deletes all lines which match /^*/.

Solution 3

You don't need sed to do this, you can use some basic greps to pull the star (*) lines to the top. Say for example you had this file:

$ cat sample.txt 
1
2
3
4
* 5
* 6
* 7
8
9
10

Now to grep the sample.txt file putting the star (*) lines first:

$ cat <(grep '*' sample.txt) <(grep -v '*' sample.txt)
* 5
* 6
* 7
1
2
3
4
8
9
10

The above will run 2 greps, the 1st pulls all the lines with stars out, while the 2nd pulls all the non-starred lines. The output from these 2 commands is redirected as input to the cat command using the <() notation.

Alternative method

If you don't want to use cat + the 2 subshells you can do as was suggested by @terdon:

$ grep '*' sample.txt; grep -v '*' sample.txt

This will pull out all the lines from sample.txt that include a star (*) followed by all the lines that don't.

References

• grep man page

Solution 4

Here is a solution using ed (and expanding a bit on the vi solution posted by Kaz, which underneath uses ex - a beefed up ed).
Replace w with ,p\n to print the output instead of writing to file.

Move the matching lines to the end of the file:

ed -s file <<< $'g/^\*/m$\nwq\n'

The global subcommand marks every line that matches the pattern. Then, for each marked line, it sets the current line to the marked line and runs the subcommand: move after the line at address $ (after the last line); then write to file and quit the editor.

Move the matching lines to the beginning of the file (or rather move the non matching lines to the end of the file - but it's the end result that matters - keep the line order):

ed -s file <<< $'v/^\*/m$\nwq\n'

The inverse global subcommand marks every line that does NOT match the pattern. Then, for each marked line, it sets the current line to the marked line and runs the subcommand: move after the line at address $ (after the last line); then write to file and quit the editor.

The reason I use inverse match to achieve the desired result instead of g/^\*/m0 (move all matching lines to the beginning) suggested by Kaz, is that (as he pointed out) when you move the matching lines to the beginning they will end up in reverse order.
This - as I emphasized above - is because g and v, for each marked line, set the current line to the marked line and run the specified subcommand(s). The first line marked as matching is moved after the line addressed by the parameter after m: 0 in this case - so it becomes the first line. Then the second matching line is moved to the same address so it becomes the first line and the line that was moved before it becomes the second line... and so on... until the last matching line is moved to the specified address.

Solution 5

If you don't mind the empty line between the two blocks:

sed -n -e '/^* /{H;$!d}' -e '/^* /!p' -e '${g;p}'

or the other way round

sed -n -e '/^* /{p;$!d}' -e '/^* /!H' -e '${g;p}' file    

View more solutions

Comments

[sendmoreinfo]

OP probably didn't want their file sorted...

[slm]

@sendmoreinfo - thanks didn't see that bit in the question, will adjust.

[terdon]

Why do you use cat and subshells? grep '*' sample.txt;grep -v '*' sample.txt will do exactly the same thing. Am I missing something?

[slm]

@terdon - No reason just felt like doing it that way.

[terdon]

Ah, OK, I though it was some kind of arcane trickery I was not aware of :),

[slm]

@terdon - I wrote it at 2AM this morning and just felt like doing it a little differently.

[Admin]

You don't have to use cat. This works as well: tac <(grep '*' sample.txt) <(grep -v '*' sample.txt) > >(tac).

[manatwork]

One minor glitch: gawk 4.1.0 I use requires semicolon between 1 and END. Does that code work for you as is, or the missing semicolon is just a copy-paste issue?

[Jean Paul]

To avoid the empty line between the two blocks you can do: sed -n -e '/^* /{p;$!d}' -e '/^* /!H' -e '${g;s/^\n//;p}' file