How can i free up memory / manage memory on a linux box?

Question

I am running two Ubuntu 12.04 servers (no GUI / desktop) using canonical's cloud images on Amazon EC2. One is a micro with 613MB of RAM and one is a medium with 3.75GB of RAM.

I only have opscode's chef installed on the micro box, and on the large box I have a java app / SOLR index initialized with 2048MB of ram:

JAVA_OPTIONS="-server -Xms2048m -Xmx2048m -XX:+UseParallelGC -XX:NewRatio=5"

If I have had chef running on the micro box for a while, and then try and install a new program, or start up something like the RabbitMQ or Apache2 I will get an error about not having enough free memory to complete the operation.

On the large box, it seems that the Java application will periodically crash when the memory gets low, or after the box has been on for a while.

Using the free / free -m command I see there is in fact not much memory available, at which point I usually have to reboot the system to "clear out" the memory.

It seems to be that either the memory should be automatically freed or "cleared" after some point of time, or if it must be done manually there might be a better way to free up / clean/ manage the memory more efficiently.

Is there a way I can manually free up / clear the system's memory? Is there a tool that will check / clean the memory automatically?

Solution

Needing to reboot certainly doesn't make much sense -- if your memory is full, it must be full of something, and if you stop that process (eg the java app), then that memory would be freed.

However, if you need to do this just so you can then restart the same app, there is the question of why the process would have grown to consume so much memory this way. Normally a process which just grows and grows and never releases anything might considered one which is leaking memory due to a bug internal to it.

Ie., the first thing you need to do is find out what software specifically is causing the problem; tools like top are commonly used for this. You might also be interested in plog.

Comments

[MelBurslan]

could please update your question with the output from command "watch -n 1 cat /proc/meminfo" ?

[cwd]

it is a possibility that after stopping processes they would not free up all the memory they used?

[goldilocks]

@cwd No. A process which no longer exists does not use any memory. top (or htop) is a pretty basic diagnostic tool that you do need to know how to use if you are working on a linux server. Play around, get the hang of it, find out what the various metrics are (understanding the difference between virtual address space and physical memory is important), then next time you get "not enough memory" you will have some means of determining where it's gone to.

[Gilles 'SO- stop being evil']

Caches and buffers get flushed out if memory is needed. Dropping caches is useful for benchmarks and little else. It cannot help in an insufficient memory situation.

[Gilles 'SO- stop being evil']

@cwd No, barring a kernel bug (which is technically possible, but very unlikely).

[sean]

Best thing to do is restart the process and figure out why it uses so much memory. Example with httpd : service httpd restart