What is ELF Magic?

Question

I've seen discussion before about ELF magic, most recently the comments in this Security stack exchange question. I've seen it mentioned before, and I've seen it in my own boot logs.. But I'm not sure what it is.

The man page on elf is a bit over my head, as I don't do C or lower level languages.

As someone who uses Linux as an every day operating system, what is ELF?

Solution 1

Right from the man page you reference:

elf - format of Executable and Linking Format (ELF) files

ELF defines the binary format of executable files used by Linux. When you invoke an executable, the OS must know how to load the executable into memory properly, how to resolve dynamic library dependencies and then where to jump into the loaded executable to start executing it. The ELF format provides this information. ELF magic is used to identify ELF files and is merely the very first few bytes of a file:

% od -c -N 16 /bin/ls
0000000 177   E   L   F 002 001 001  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000020

or

% readelf -h /bin/ls | grep Magic
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 

These 16 bytes unambiguously identify a file as an ELF executable. Many file formats have "magic" bytes that accomplish the same task -- identifying a type of file.

Solution 2

"Magic numbers" is the name given to constant sequences of bytes (usually) at the beginning of files, used to mark those files as being of a particular file format. They serve a similar purpose to file extensions.

See the jargon file entry for more information.

For example, PNG images always start with the same eight bytes: 137 80 78 71 13 10 26 10

Hence ELF magic numbers are the bytes at the beginning of elf files that identify them as such.

Comments

[Simon Richter]

The actual magic is just the first four bytes. The following fields describe endianness, CPU architecture, and various other things.

[casey]

@SimonRichter that boils down to semantics. The first 4 bytes are magic for generic identification of many file types, but the asker did specify "ELF magic", which even readelf acknowledges is 16 bytes.

[Michael Mrozek]

If you want to get really technical, the first 16 bytes are the "identification" (e_ident), of which the first 4 bytes are the magic number (EI_MAG0 through EI_MAG3)

[slebetman]

@MichaelMrozek: casey's use of the word "magic" derives from the file program which calls file identifiers "magic". Your use of the word magic derives from programmers referring to mysterious, uncommented, hardcoded numbers in source code as "magic".

[Michael Mrozek]

@slebetman My use of "magic" comes from the ELF spec: "A file's first 4 bytes hold a 'magic number,' identifying the file as an ELF object file"

[slebetman]

@MichaelMrozek: Yes, that use of "magic number" by the ELF spec comes from programmers use of "magic number".

[cHao]

"Unambiguously" is overstating a tiny bit. Unless you know where the file came from, anything is just a guess. A file in /bin, sure, it's almost certainly an ELF binary. Some random file you downloaded, though...there's no telling.

[Ángel]

@cHao, if the file starts by those bytes, you can be quite sure that it attempts to be an ELF binary, even though it could be malformed.