Linux Huge Pages Usage Accounting

Question

I have configured Huge Pages for use with Java and it appears to be working well although I have a question about the accounting in /proc/meminfo. To illustrate

# grep HugePages /proc/meminfo 
AnonHugePages:    274432 kB
HugePages_Total:    1008
HugePages_Free:      596
HugePages_Rsvd:      594
HugePages_Surp:        0

My question concerns the "Free" and "Rsvd" numbers - why do they not add up to "Total" of 1008? They actually add up to 1190. What am I not understanding here?

Solution

This is because HugePages_rsvd is essentially read from HugePages_Free. Meaning, out of 596 huge pages which are free, 594 are already reserved by some application for use. That is kernel has committed that those 594 huge pages are available for the application.

If there is a request for 3 huge pages now, then it will fail as only 2 are available to be reserved. Think of it as a malloc() call, when you reserve memory virtual pages to account for the VSZ for the process but when the process actually uses them, it becomes the RSZ (running set) of the process.

As huge pages are always resident on main memory, when an app requests them kernel decrements it from free pool and increase the Rsvd counter.

This is from the kernel source. https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt

where:
HugePages_Total is the size of the pool of huge pages.
HugePages_Free  is the number of huge pages in the pool that are not yet
                allocated.
HugePages_Rsvd  is short for "reserved," and is the number of huge pages for
                which a commitment to allocate from the pool has been made,
                but no allocation has yet been made.  Reserved huge pages
                guarantee that an application will be able to allocate a
                huge page from the pool of huge pages at fault time.
HugePages_Surp  is short for "surplus," and is the number of huge pages in
                the pool above the value in /proc/sys/vm/nr_hugepages. The
                maximum number of surplus huge pages is controlled by
                /proc/sys/vm/nr_overcommit_hugepages.