How to reduce buffers\cache

Question

A monitoring system keeps alerting that my machine is reaching\breaking through it's RAM utilization threshold which is 15 GBs.

I've done some reading and understood that the apparent RAM utilization is not actual and that the extra RAM is used for caching\buffering disk I\O operation to improve the performance of the server. I'm running MySQL on that server, that's the only notable service running.

• So how can I reduce the disk I\O caching\buffering RAM as not to break through the threshold ? Could this be a MySQL issue and not Linux's ?

That's the output of free -gt

[root@ipk ~]# free -gt
             total       used       free     shared    buffers     cached
Mem:            15         15          0          0          0          9
-/+ buffers/cache:          5         10
Swap:            5          0          5
Total:          21         15          6

Linux version is:

[root@ipk ~]# uname -rmo
2.6.32-220.el6.x86_64 x86_64 GNU/Linux

Solution 1

Since you don't seem to accept neither our opinions not the various pages we have linked to as 'official', perhaps the official Red Hat documentation will convince you:

In this example the total amount of available memory is 4040360 KB. 264224 KB are used by processes and 3776136 KB are free for other applications. Do not get confused by the first line which shows that 28160KB are free! If you look at the usage figures you can see that most of the memory use is for buffers and cache. Linux always tries to use RAM to speed up disk operations by using available memory for buffers (file system metadata) and cache (pages with actual contents of files or block devices). This helps the system to run faster because disk information is already in memory which saves I/O operations. If space is needed by programs or applications like Oracle, then Linux will free up the buffers and cache to yield memory for the applications. If your system runs for a while you will usually see a small number under the field "free" on the first line.

Solution 2

This is all fine and dandy, but what about the lag time between having to make the request for memory, mapping it out for what the disk cache has to release? Sure, you may say it's minimal, or not even noticeable, but maybe I care about the 0.0000001 seconds added because of this process. Who are you to decide how my RAM is used?

What if I don't want to experience this lag time? Maybe I don't mind my applications loading slower from the drive, but want it to occupy RAM immediately without the lag time to unmap memory from disk caching.

How can I limit how much RAM my disk cache is taking up? Don't tell me this was implemented without a way to do so.

UPDATE: Found a way to do it.

echo 3 | sudo tee /proc/sys/vm/drop_caches

Do a: free -m after running the above to confirm. It works.

Comments

[Admin]

You have 10 giga free RAM. If a monitoring system alerts you about that - fix the monitoring system.

[Admin]

You should fix your alerting software to monitor the actual thing that needs monitoring. Dropping buffers/cache is simply wasting performance.

[Admin]

I don't have enough knowledge to support my argument. Can't I just reduce the caching\buffering service by 1 GB ? Else, is there an official document to support that this problem is in the monitoring system ?

[Admin]

Depends of the monitoring system. But my bet is that it's reading "free" memory instead "Total memory - Used memory that's not cache/buffers".

[Admin]

@Braiam, I don't understand what you are trying to say.

[Admin]

@MuhammadGelbana: your monitoring software should not count cached memory as used. Simple as that.

[Admin]

Reducing the buffer/cache would severely reduce the performance of your MySQL server, so it is not an ideal solution. Your monitoring software is broken, it needs to be fixed.

[Admin]

@jsbillings, you are correct. But I can either reduce the caching\buffering RAM by 1 GB or at least 512 MB which I don't think will heavily affect the performance, or get a strong official document to support my argument.

[Admin]

@jsbillings, that's exactly where I got my info but it's not an official source :)

[Admin]

Tell us what you think an official source is, and I'll try to find it. Here's more documentation on the Linux memory manager: linux-mm.org/Low_On_Memory . Since it appears you are running Red Hat Enterprise Linux 6 (or some clone thereof), here's Red Hat's tips on memory usage: redhat.com/advice/tips/meminfo.html

[Admin]

@MuhammadGelbana see the link at the bottom of the linuxatemyrampage for some checks you can run to convince yourself. Seriously, you're using a broken monitoring system and you are approaching this is the wrong way. A simple check is to launch a memory intensive process and watch how easily it will claim all that cached memory.

[chicks]

I know it is what they asked, but why encourage the OP to do something silly like this?

[Arunas]

It might surprise you that some people are not running stock 'server' or 'desktop' applications. There are definitely situations where we have come across severe performance hits due to caching. We acquire data off an ADC at 600000000Bytes/s and often write large chunks of that to disk, while processing other bits in a GPU. The disk cache can be a blessing and a curse - it handles backlogs when the storage device cannot handle the data quickly enough, but also limits us when the situation changes.

[Pedro Lobito]

Using a daily cronjob like sync; echo 3 > /proc/sys/vm/drop_caches would be a good idea?

[terdon]

@PedroLobito no, of course not. Why would you do that? The whole point is that this doesn't count as used RAM. All the RAM being used by these processes is still available to any program that requests it.

[Pedro Lobito]

That was exactly what I though, tks.

[Muhammad Gelbana]

I'm still getting notifications for this question and it ocurred to me that I might've upset some of who tried hard to help me out since I didn't accept non-official sources. I just felt the urge to appologize for the annoyance I may have caused.