What is the appropriate value of vm.swappiness when using zram?

Question

I'm using zram on my computer as a compressed RAM-backed swap. When the system needs to swap something out, swapping it to a zram-backed swap file is more or less equivalent to compressing that data in-memory to free up space. This makes swapping very fast most of the time, relative to disk-backed swap. Because of this, I wonder if there is some performance to be gained by encouraging the system to swap out unused stuff more aggressively, since it can do so without actually hitting the disk?

So has anyone messed around with, say, setting vm.swappiness to 100 while using zram? Would this be desirable?

sysctl -w vm.swappiness=100

Solution 1

Short answer:vm.swappiness=100is appropriate value for zram(At least on Debian Stretch with Linux 4.9 , I believe that is best value )

I already test vm.swappiness=100 for me.

I think you can do some simple test to sure which value is best for you.

Also I made another simple program for test this question. x On my machine a very low vm.swappiness value(such as vm.swappiness=1 ) will cause obvious responsiveness problem.

About SwapCached in /proc/meminfo:

First,try vm.page-cluster=0,this maybe can reduce some useless SwapCached from swap-in.

SwapCached can speed up zram same as non-zram swap device

SwapCached is can reuse(free) when necessary:

./linux-4.9/mm$ grep -rn delete_from_swap_cache
memory-failure.c:715:   delete_from_swap_cache(p);
shmem.c:1115:       delete_from_swap_cache(*pagep);
shmem.c:1645:            * unaccounting, now delete_from_swap_cache() will do
shmem.c:1652:               delete_from_swap_cache(page);
shmem.c:1668:       delete_from_swap_cache(page);
vmscan.c:673:       __delete_from_swap_cache(page);
swap_state.c:137:void __delete_from_swap_cache(struct page *page)
swap_state.c:218:void delete_from_swap_cache(struct page *page)
swap_state.c:227:   __delete_from_swap_cache(page);
swapfile.c:947:         delete_from_swap_cache(page);
swapfile.c:987: delete_from_swap_cache(page);
swapfile.c:1023:            delete_from_swap_cache(page);
swapfile.c:1571:            delete_from_swap_cache(page);
./linux-4.9/mm$ 

Solution 2

I would really not recommend to put the swappiness higher. A common mechanism in kernel is that it will put pages (chunk of memory) in the swap to free some memory for other running tasks.

First "problem" when the kernel wants n pages to be freed, m (with m < n, m is the number of compressed pages required to hold n) are newly created in RAM, I am not sure if that can disturb the kernel or not.

Then anyway, when you have pages in the swap, it is possible that you use the application later on with some of its pages in the swap. What the kernel do is bring back those pages into physical memory but does not remove them from swap (which with standard swap can be seen as caching, so when the application go back in the background, the kernel does not have to write back those pages into the slow swap). However with zram it is perhaps not a wise trick, because you then have in memory the m pages in zram + the n pages that are back in memory!

The kernel has normaly a "total memory" thatit can use to do its business. When you add zram, it is counting in the "swap" memory only, as it would be with any disk based swap, but it reduced the actual "total memory" and that is not expected/anticipated by the kernel. Sometimes you can have weird and not wanted behaviour because of this!

With zram, it would be good that the kernel does not swap too much to this area when it is under memory pressure. And you should always have a real hard disk swap partition bigger at least than your zram maximum size, so that the system won't get OOM whereas at the same time you would see plenty of free space as reported by free!

Solution 3

Pages needs to be swapped out (to disk) when memory is full. If you're using memory to create the place to swap out pages when memory is full, one would think it beats the purpose, except if the compression makes a difference (and then it would be natural to compress the memory directly instead of going through swap). Guess one would have to benchmark this, as computers are increasingly fast at compressing and decompressing compared to memory speed.

Comments

[Ryan C. Thompson]

I've found zram-backed swap itself to be quite helpful when the system is running out of memory. It's saved me a few times from getting completely stuck in swapping hell and having to restart (I'm analyzing large data sets, so I need the memory, all 24 GB of it). I'm just wondering if the vm.swappiness value is tuned for disk-backed swap and if I should change it if I'll mostly be using zram-backed swap.

[chrishollinworth]

"increasingly fast"? On the fly compression has been performing better than direct disk I/O for more than the past decade (its never going to be faster than memory access - that's not the point)

[Alexander]

symcbean, you forgot "compared to memory speed". Where is the disagreement?

[Daniel Papasian]

I think symcbean's point is that the goal of compressed memory backed pages (zram) is to replace swapping to physical medium. The reason it is not "natural to compress the memory directly" is because it would be fraught with complication for applications to have to determine which portions of its memory could be compressed and when; the VM subsystem is a much easier place to implement this. Workloads that benefit from zram have pages that aren't in the working set and can be easily compressed.

[Elder Geek]

zram provides RAM block devices. Everything written to these block devices gets compressed. If zram block devices are used as swap, when the system tries to move parts of memory to swap it will be effectively moving memory from one part of the RAM to another, except that the data will be compressed before being copied to the destination. This effectively works as a cheap memory compression mechanism to improve responsiveness in systems with limited amounts of memory. ... Zram has been in staging since Linux 2.6.33. In 3.14 zram was moved out of staging to drivers/block/zram.

[Huygens]

@ElderGeek exactly! And I will take an example to illustrate why that is not perfect. The kernel will try to remove 64MB of RAM, it put them into the zram swap. Compressed the 64MB chunk is now 32MB. The result is the memory decreased by 32MB and not 64MB. Now what happen when the application requires the 64MB back in memory, the kernel copy (and not move) the chunk back in memory. You have now 64MB in RAM + 32MB in zram. Why copy? Because the kernel caches the pages as I explained in my answer. Both behaviour are not ideal. And when memory is not well compressable, that's even worse.

[Elder Geek]

Still in the testing phase. I would think that there must be some way to tune the caching algorithm that zram uses to free the compressed page when it's copied back to uncompressed RAM.

[Huygens]

I've tried it several times until 2012. At that time my computer was limited to 1GB of RAM and while browsing internet it was painfully slow (I usually have 10-50 tabs opened). But since then I never used it again. I have more than enough RAM that I don't use swap anymore. When using zram with Firefox back then, I quickly started "swapping" given the little RAM I had. And quickly I had an unresponsive systems with many crashes. Without zram it was painfully slow but stable at least. My problem was that probably it was constantly compressing/uncompressing memory pages.

[Elder Geek]

Yes, my results have been somewhat similar, on a system with 8GB I was able to force an OOM by launching several VM's during an encoding job. Do you have any experience with zswap? It seems a viable alternative based on what I've read.

[Huygens]

Sorry, since then I haven't had a need to try to optimise memory consumption. So I did not try alternatives.