Is unreclaimable memory allocated to slab considered used or available cache?

Question

After evaluating /proc/meminfo, I see the following information:

$cat /proc/meminfo
MemTotal:       197852592 kB
MemFree:        64755992 kB
MemAvailable:   65655112 kB
Buffers:            4388 kB
Cached:           759952 kB
SwapCached:            0 kB
Active:           649472 kB
Inactive:         308340 kB
Active(anon):     193840 kB
Inactive(anon):    25316 kB
Active(file):     455632 kB
Inactive(file):   283024 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4095932 kB
SwapFree:        4095932 kB
Dirty:                 8 kB
Writeback:             0 kB
AnonPages:        193560 kB
Mapped:            86208 kB
Shmem:             25684 kB
Slab:           49141432 kB
SReclaimable:     667284 kB
SUnreclaim:     48474148 kB
KernelStack:       25600 kB
PageTables:        15152 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    103022228 kB
Committed_AS:    1097276 kB
VmallocTotal:   34359738367 kB
VmallocUsed:    82484800 kB
VmallocChunk:   34126392952 kB
HardwareCorrupted:     0 kB
AnonHugePages:      4096 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      762368 kB
DirectMap2M:    51664896 kB
DirectMap1G:    148897792 kB

And specifically the slab:

Slab:         46.8651GB
SReclaimable: 0.63637GB
SUnreclaim:   46.2286GB

After evaluating slabtop I see the following as the largest users:

      OBJS    ACTIVE  USE OBJ    SIZE   SLABS  OBJ/SLAB  CACHE SIZE                    NAME
  10855309  10855309     100%   1.07K  374321        29      11.42G         zfs_znode_cache
  10893059  10893059     100%   0.85K  294407        37       8.98G                 dnode_t
    412694    410756      99%  16.00K  206347         2       6.30G           zio_buf_16384
  12502304  12290111      98%   0.50K  390697        32       5.96G             kmalloc-512
  12776610  12744002      99%   0.29K  232302        55       3.54G          dmu_buf_impl_t
  10855309  10855309     100%   0.27K  374321        29       2.86G                sa_cache
    370776    370718      99%   8.00K   92694         4       2.83G            kmalloc-8192
   3269280   3028688      92%   0.32K   66720        49       1.02G               taskstats
  10899924  10899924     100%   0.08K  213724        51       0.82G  selinux_inode_security
  12161408  12150615      99%   0.06K  190022        64       0.72G              kmalloc-64
   3266592   3266072      99%   0.19K   77776        42       0.59G                  dentry
   5577600   5519569      98%   0.09K  132800        42       0.51G              kmalloc-96
     92872     82422      88%   4.00K   11609         8       0.35G            kmalloc-4096
   1962464   1953555      99%   0.12K   61327        32       0.23G             kmalloc-128
   6022528   6022428      99%   0.03K   47051       128       0.18G              kmalloc-32
      8356      8346      99%  12.00K    4178         2       0.13G           zio_buf_12288

What makes slab memory reclaimable versus unreclaimable? What does "unreclaimable" mean when the system needs to allocate and there isn't enough memory available? Is it as flexible as buff/cache?

Solution

Reclaimable SLAB memory can be reused by the kernel for other things, unreclaimable can't. Where any given SLAB allocation ends up accounted depends on the properties of the pool it's allocated in, which in turn means it's a property of the allocation itself.

Slightly OT, but for what it's worth, that huge chunk of unreclaimable SLAB memory is probably the ZFS ARC.

Comments

[Zhro]

As this is a very technical question would you be able to provide citations for your answer?

[Austin Hemmelgarn]

Sure, but it may be a while, since I'm going to have to search through kernel code to find the bits that relate to this (it's unfortunately not documented much of anywhere, and my knowledge of it is largely based on code inspection).

[Gregg Leventhal]

@AustinHemmelgarn its been awhile and im interested as well.