How to add lines/text to the beginning of a file

Question

We have the following example file:

tcpmux          1/tcp                           # TCP port service multiplexer
tcpmux          1/udp                           # TCP port service multiplexer
rje             5/tcp                           # Remote Job Entry
rje             5/udp                           # Remote Job Entry
echo            7/tcp
echo            7/udp
discard         9/tcp           sink null
discard         9/udp           sink null
systat          11/tcp          users
systat          11/udp          users
daytime         13/tcp
daytime         13/udp
qotd            17/tcp          quote
qotd            17/udp          quote
msp             18/tcp                          # Message send protocol (historic)
msp             18/udp                          # Message send protocol (historic)
chargen         19/tcp          ttytst source
chargen         19/udp          ttytst source

How may we append the following lines to the beginning of the file?

# The latest IANA port assignments can be gotten from
#       http://www.iana.org/assignments/port-numbers
# The Well Known Ports are those from 0 through 1023.
# The Registered Ports are those from 1024 through 49151
# The Dynamic and/or Private Ports are those from 49152 through 65535
#
# Each line describes one service, and is of the form:
#
# service-name  port/protocol  [aliases ...]   [# comment]

So that the file will look like:

# The latest IANA port assignments can be gotten from
#       http://www.iana.org/assignments/port-numbers
# The Well Known Ports are those from 0 through 1023.
# The Registered Ports are those from 1024 through 49151
# The Dynamic and/or Private Ports are those from 49152 through 65535
#
# Each line describes one service, and is of the form:
#
# service-name  port/protocol  [aliases ...]   [# comment]
tcpmux          1/tcp                           # TCP port service multiplexer
tcpmux          1/udp                           # TCP port service multiplexer
rje             5/tcp                           # Remote Job Entry
rje             5/udp                           # Remote Job Entry
echo            7/tcp
echo            7/udp
discard         9/tcp           sink null
discard         9/udp           sink null
systat          11/tcp          users
systat          11/udp          users
daytime         13/tcp
daytime         13/udp
qotd            17/tcp          quote
qotd            17/udp          quote
msp             18/tcp                          # Message send protocol (historic)
msp             18/udp                          # Message send protocol (historic)
chargen         19/tcp          ttytst source
chargen         19/udp          ttytst source

The simple solution is to copy the original file to file.bck, append the new lines to the file, and append file.bck to the file.

But this isn't an elegant solution.

Solution 1

Relatively elegant solution using POSIX specified file editor ex—at least elegant in the sense that this will handle any arbitrary contents rather than depending on a specific format (trailing backslashes) or a specific absence of format.

printf '0r headerfile\nx\n' | ex file-with-contents

This will open file-with-contents in ex, read in the full contents of the headerfile at the very top, and then save the modified buffer back to file-with-contents.

If performance is a SEVERE concern and the files are huge this may not be the right way for you, but (a) there is no performant general way to prepend data to a file and (b) I don't expect you will be editing your /etc/services file that often.

---

A slightly cleaner syntax (the way I would actually code this):

printf '%s\n' '0r headerfile' x | ex file-with-contents

---

A more complicated, but convergent, bit of code that will check whether the beginning of services EXACTLY matches the entirety of header, byte for byte, and IF NOT will then prepend the entire contents of header to services and save the changes, follows.

This is fully POSIX compliant.

dd if=services bs=1 count="$(wc -c < header)" 2>/dev/null |
  cmp -s - header ||
    printf '%s\n' '0r header' x |
      ex services

A much simpler version, using GNU cmp's "-n" option:

cmp -sn "$(wc -c <header)" header services ||
  printf '%s\n' '0r header' x | ex services

Of course, neither of these is smart enough to check for PARTIAL matches, but that's getting far beyond the ability of a simple one liner, since guesswork would be intrinsically involved.

Solution 2

OK I have decided to write an answer besides a comment.

You can use the i command of sed like:

sed -i '1i \
# The latest IANA port assignments can be gotten from\
#       http://www.iana.org/assignments/port-numbers\
# The Well Known Ports are those from 0 through 1023.\
# The Registered Ports are those from 1024 through 49151\
# The Dynamic and/or Private Ports are those from 49152 through 65535\
#\
# Each line describes one service, and is of the form:\
#\
# service-name  port/protocol  [aliases ...]   [# comment]' file

The is for GNU sed. For sed on Macs you need to use sed -i '' -e ..., and for POSIX sed there is no simple way to do things in place.

Solution 3

Usually, you do just that. Prepending lines to a file is hard, since files are just sequences of bytes, so you'd need to move the existing data ahead to make space for the new data, and there's no direct method for that (at least no standard method). In theory, one might imagine a filesystem based on variable length records, where you could add new records at the start or between existing records, but that's not how it works in practice.

Some filesystems can move blocks of data around, but they're fixed-size blocks, and so not much use for text files, where the lines have variable lengths.

Even if you do something like sed -i or perl -i, they're going to create a temporary file behind the scenes just for that reason.

So, be it elegant or not, I'd go with:

cat prefix data > data.new && mv data.new data

For a few lines, you could use (in GNU sed):

sed -i.bak -e '1i first prefix line' -e '1i second prefix line'  data

But generating the insert commands or adding backslashes for each line to be added isn't elegant either.

Comments

[Alessio]

+1. I was going to comment with your cleaner syntax second version, but it's already there. that's the way I'd do it....in fact, it's the way i have done it but with ed rather than ex. btw, one more advantage of using ed or ex is that (unlike the -i options of sed or perl) it's a real in-place edit - the resulting file has the same inode as the original.

[don_crissti]

This doesn't edit the file in-place. If you just need to output the combined content of files to stdout you can simply do cat file1 file2 there's no need to chain 3 sed invocations (and btw, this can be done with a single sed invocation)

[Jeff Schaller]

This is worse than ilkkachu's very similar answer from 4 hours ago: cat prefix data > data.new && mv data.new data, as yours will clobber file.txt if the initial cat fails (full filesystem?).

[G-Man Says 'Reinstate Monica']

Alternatively, cat prefix data > data.new  &&  cp data.new data  &&  rm data.new will keep data in the same inode (which is important if you have hard links, or if you need to preserve attributes like ownership or ACLs), at the cost of doing twice as much I/O.

[Nubarke]

You have a point there, but I did start the answer by assuming a normal, sane environment. A full filesystem is a rare condition, yet very easily noticed when it happens (and as such you can take special precautions, such as when working with a system out of memory). Otherwise, inserting an explanatory header into a regular text config file is rather unlikely to be the tipping point.

[Wildcard]

The thing is that for scripted solutions, you don't get to assume a sane operating environment. So at the very least an answer like this should have a big bold warning message that it should only be used interactively—and in that case, it really shouldn't be on one line (i.e. the semicolon should be a literal newline). The percentage of Production outages that are caused by lazy scripting that assumes a sane operating environment is probably in excess of 75%.