Ubuntu Linux: Process swap memory and memory usage

Question

My Ubuntu eats more memory than the task manager is showing:

sudo ps -e --format rss  | awk 'BEGIN{c=0} {c+=$1} END{print c/1024}'
2750.29

free -m

             total       used       free     shared    buffers     cached
Mem:          3860       2765       1094          0          3        300
-/+ buffers/cache:       2461       1398
Swap:         2729       2374        354

That's strange. Can someone explain this difference?

But what is more important: I'd like to know how much memory a process is really using. I don't want to know the virtual memory size, but rather the resident memory plus swap of a process.

I have also tried to output the format param "sz" of 'ps', but the sum of this is to high (16000 MB) (param 'size' gives 36700 MB). Are there any other options?

I really want to use this, to determine which programs/processes are eating to much memory (and swap), to kill them, because memory is valuable :-) This just really don't make sense, so I'm asking here.

Output of /proc/meminfo:

MemTotal:        3952812 kB                
MemFree:         1119192 kB
Buffers:            2676 kB
Cached:           290068 kB
SwapCached:       160980 kB
Active:          1805396 kB
Inactive:         731680 kB
Active(anon):    1745820 kB
Inactive(anon):   689184 kB
Active(file):      59576 kB
Inactive(file):    42496 kB
Unevictable:         148 kB
Mlocked:             148 kB
SwapTotal:       2795272 kB
SwapFree:         390900 kB
Dirty:              1984 kB
Writeback:             0 kB
AnonPages:       2085472 kB
Mapped:            67432 kB
Shmem:            190676 kB
Slab:              88012 kB
SReclaimable:      42704 kB
SUnreclaim:        45308 kB
KernelStack:        5496 kB
PageTables:        87860 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     4771676 kB
Committed_AS:    9522364 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      374404 kB
VmallocChunk:   34359330144 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       61440 kB
DirectMap2M:     4030464 kB

Solution 1

I finally got the answer to my question. There's a program called smem (on Ubuntu/Debian apt install smem) that lets you list swap and used memory separately.

A few different ways of listing swap can be found here: https://www.cyberciti.biz/faq/linux-which-process-is-using-swap/.

Solution 2

The linux virtual memory system isn't quite so simple. You can't just add up all the RSS fields and get the value reported used by free. There many reasons for this, but I'll hit a couple of the biggest ones.

• When a process forks, both the parent and the child will show with the same RSS. However linux employs copy-on-write so that both processes are really using the same memory. Only when one of the processes modifies the memory will it actually be duplicated.
  This will cause the free number to be smaller than the top RSS sum.

• The RSS value doesn't include shared memory. Because shared memory isn't owned by any one process, top doesn't include it in RSS.
  This will cause the free number to be larger than the top RSS sum.

Solution 3

I think you're better off trusting the output of "free" as far as your total memory usage goes, and trusting "ps" for a general idea of how much memory a single process is using.

Just because the sum of "ps" RSS values doesn't equal "free" doesn't stop you from sorting your processes by RSS and evaluating the biggest ones for killing.

That being said, if all your effort is only in service of making sure the machine can hibernate, creating more swap (in the form of a file on disk, if necessary) is probably an easier path to take.

Comments

[cjc]

I should point out that "1043.84" isn't very far from "1178", which is the number you're going to get to with that calculation you're doing.

[GoldenNewby]

I had literally the identical question on unix.stackexchange (+1). Patrick blew my mind over there too :P-- unix.stackexchange.com/questions/34795/…

[David Halter]

Oh, thank you! I always thought, that free -m shows the actual shared memory sum of the whole system. But as "man ps" points out: "The shared memory column should be ignored; it is obsolete."

[David Halter]

Sorry, i cannot accept this answer, because it doesn't answer the swap problem, but thank you anyway!

[phemmer]

@DavidHalter what swap problem? The information I provided applies to all memory, including swap.

[David Halter]

shared memory is not really big, normally? At least that's what I saw in the task manager. But my swap is pretty heavily used: 1035 MB; The sum of PS is 1 GB and the sum of swap + rss is > 2 GB. I want to see how much memory a process is really using, not just rss. Even more interessting would be how much swapped memory a process is using.

[phemmer]

@DavidHalter if you want to see how much memory a process is using (including shared), then look at the 'VSZ' column.

[David Halter]

If shm + rss are added, the output of free must be equal or lower, right? They are not: 190 MB swap (/proc/meminfo), 2700 MB rss and 5100 MB used (free -m -> swap + mem).

[phemmer]

@DavidHalter I'm not sure what youre adding up your last comment. If you edit your question and provide details there, I might be able to explain what youre seeing. However right now I can answer with this, youre not going to be able to add up rss or vsz or any combination thereof and get numbers that match total memory used or free. The linux virtual memory system just isnt that simple.