write-through RAM disk, or massive caching of file system?

Question

I have a program that is very heavily hitting the file system, reading and writing randomly to a set of working files. The files total several gigabytes in size, but I can spare the RAM to keep them all mostly in memory. The machines this program runs on are typically Ubuntu Linux boxes.

Is there a way to configure the file system to have a very very large cache, and even to cache writes so they hit the disk later? I understand the issues with power loss or such, and am prepared to accept that. Crashing aside, in normal operation the writes should eventually reach the disk!

Or is there a way to create a RAM disk that writes-through to real disk?

Solution 1

Consider using an ext4 filesystem using fast-and-loose mount options:

noatime,data=writeback,nobh,barrier=0,commit=300

to postpone writing data from cache back to physical disk.

Other than that, you could use aufs to union-mount a tmpfs-filesystem on top of your regular filesystem, do all the writing and then merge the tmpfs back down to the real filesystem afterwards.

Solution 2

Are you seeing high numbers of IO waits, indicating that the read and write requests are not being satisfied via existing buffers? As others have noted, Linux is very good about giving spare RAM to buffers, so you should check this first.

If you're not seeing IO waits, then it's possible that your performance problems (do you even have problems? your question doesn't say) are due to kernel context switches from lots of small program-initiated IO operations. In this case you can gain a significant performance boost by rewriting your application to use memory-mapped files. But that's more of a question for StackOverflow.

Solution 3

Linux by default uses any spare RAM as a file cache, so no configuration is necessary for that.

You may want to consider using ext4 as the filesystem. It uses quite a number of techniques to speed up disk access, including delayed allocation which:

This has the effect of batching together allocations into larger runs. Such delayed processing reduces CPU usage, and tends to reduce disk fragmentation, especially for files which grow slowly. It can also help in keeping allocations contiguous when there are several files growing at the same time.

Data loss is pretty rare due to the use of journaling.

Ext4 is now the default filesystem in recent releases of Linux, though you will probably want to make sure the kernel you use is at least 2.6.30

Solution 4

You can see my answer here: Reserve RAM for cache and buffer. If you want to reserve memory for cache and buffer: echo 10 > /proc/sys/vm/vfs_cache_pressure when 100 is the default value. Then you can limit the max ram used by each app: echo 8192 > /proc/sys/vm/max_map_count.

Solution 5

Use mmap on those files - it uses the kernel's superior caching, making it much faster.

View more solutions

Comments

[Doug Coburn]

I don't think this is about Linux using RAM efficiently but rather about disk caching, like a Linux equivalent to FlashFire: flashfire.org/xe

[Yuriy P]

hmm ... last answer before mine was 3 months ago, OP hasn't been seen since posting the question, no other obvious activity, yet it appeared on the front page ... I guess the system is trying to get answers to questions

[Corey]

That is precisely what user account Community is for - bumping unanswered questions. stackoverflow.com/users/-1/community

[Yuriy P]

:shrug: the OP is long gone, and nobody else seems to care

[Niklas]

I do care, and I'm back; it will take me a while to digest your question and answer it fairly, please be patient

[netvope]

meaning of the options: mjmwired.net/kernel/Documentation/filesystems/ext3.txt

[DylanYoung]

FYI: nobh/bh is deprecated for ext4 as it doesn't do anything.