EXT3: If block size is 4K, why does ls -l show file sizes below that?

Question

If you run ls -l on a file that contains one letter, it will list as 2B in size. If your file system is in 4k blocks, I thought it rounded files up to the block size? Is it because ls -l actually reads the byte count from the inode? In what circumstances do you get rounded up to block answers vs actual byte count answers in Linux 2.6 Kernel GNU utils?

Solution 1

I guess you got that one letter into the file with echo a > file or vim file, which means, you'll have that letter and an additional newline in it (two characters, thus two bytes). ls -l shows file size in bytes, not blocks (to be more specific: file length):

$ echo a > testfile
$ ls -l testfile
-rw-r--r-- 1 user user 2 Apr 28 22:08 testfile
$ cat -A testfile
a$

(note that cat -A displays newlines as $ character)

In contrast to ls -l, du will show the real size occupied on disk:

$ du testfile
4

(actually, du shows size in 1kiB units, so here the size is 4×1024 bytes = 4096 bytes = 4 kiB, which is the block size on this file system)

To have ls show this, you'll have to use the -s option instead of/in addition to -l:

$ ls -ls testfile
4 -rw-r--r-- 1 user user 2 Apr 28 22:08 testfile

The first column is the allocated size, again in units of 1kiB. Last can be changed by specifying --block-size, e.g.

$ ls -ls --block-size=1 testfile
4096 -rw-r--r-- 1 aw aw 2 Apr 28 22:08 testfile

Solution 2

I think that the deep answer is the following:

Logical file length and disk space occupied are really different things.

As the other answers show, in principle a file created with two bytes has length two bytes (show by ls -l) and occupy 4 KiB ( show by duor ls -ls).

See:

1& [:~/tmp] % echo -n A > test
1& [:~/tmp] % ls -l test
-rw-rw-r-- 1 romano romano 1 Apr 28 14:31 test
1& [:~/tmp] % du test
4 test

Ok, testhas length 1 and size (on disk) 4 KiB. But:

1& [:~/tmp] % truncate -s +8191 test
1& [:~/tmp] % ls -l test
-rw-rw-r-- 1 romano romano 8192 Apr 28 14:33 test
1& [:~/tmp] % du test
4   test

(the first command add 8191 zero bytes to test), now test has length 8192 but still occupy 4 KiB on disk (it has a "hole") in it(1).

Some filesystem can also compact short files so that they occupy less space by sharing blocks (see for example tail packing) and others like btrfs do copy on write, so the relationship between a file, its logical length, and how much space it occupies on a disk is a complex one.

Footnotes:

(1) It is not really a hole, it is at the end... but still, it works to the end of the example.

Solution 3

ls -l is just a long format. ls -ls is used to display the block size.

Testing

echo "1" > 1.txt

bash-3.2$ ls -l 1.txt
-rw-rw-r-- 1 ramesh ramesh 2 Apr 28 15:15 1.txt

As we can see the size of the file is listed as 2B. However, if you need to check the block size, you need to run the below command.

bash-3.2$ ls -ls 1.txt
4 -rw-rw-r-- 1 ramesh ramesh 2 Apr 28 15:15 1.txt

The 4 above displays the block size used. We can also verify the same using the stat command.

bash-3.2$ stat 1.txt
  File: `1.txt'
  Size: 2               Blocks: 8          IO Block: 4096   regular file
Device: 805h/2053d      Inode: 48267720    Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  505/  ramesh)   Gid: (  508/  ramesh)
Access: 2014-04-28 15:17:31.000000000 -0500
Modify: 2014-04-28 15:15:58.000000000 -0500
Change: 2014-04-28 15:15:58.000000000 -0500

Now the question arises on why ls -ls lists the block size as 4 while stat displays the block size as 8. The reason for this behavior is clearly explained in the answer here.

Many disks have a sector size of 512 bytes, meaning that any read or write on the disk transfers a whole 512-byte sector at a time. It is quite natural to design filesystems where a sector is not split between files (that would complicate the design and hurt performance); therefore filesystems tend to use 512-byte chunks for files. Hence traditional utilities such as ls and du indicate sizes in units of 512-byte chunks.

Comments

[Rmano]

And, moreover, it is nice to have the two informations. Filesystems can do "tail compaction" (sp?)(use one block shared among short files), "copy on write" and "hole punching"... making the file size <-> disk space relationship a complex one.