Is there a command which will force Linux to flush cache of one file on an NFS share?

36,564

Solution 1

Check these items, and see if any work for you:

  1. On the client, if you're not already using the cto option in the options column of the /etc/fstab line for your NFS filesystem, add it. cto tells the nfs client to open files via close-to-open, which makes them refresh the file whenever they open it.

  2. On the server, make sure your filesystem is exported with the sync option, and not async. With synchronous writes, the client will flush to disk when the file is closed. (or with nfs V3, whenever a commit() is made). There may be a performance hit that way, but if you're doing writes to an NFS filesystem, you definitely want sync set.

  3. Following on the heels of that stackoverflow post, opening file with O_DIRECT works only if the kernel was compiled with CONFIG_NFS_DIRECTIO.

  4. Also, make sure you have the following settings in your httpd.conf file:

    • EnableMMAP off
    • EnableSendfile off

    From the apache performance tuning documentation:

    • If you memory-map a file located on an NFS-mounted filesystem and a process on another NFS client machine deletes or truncates the file, your process may get a bus error the next time it tries to access the mapped file content.
    • Turning off EnableSendfile won't specifically help with the sync/async behavior of NFS, but it needs to be turned off if you're using apache with NFS.

Solution 2

Within a given process, calling opendir and closedir on the parent directory of a file invalidates the NFS cache. I used this while programming a job scheduler. Very, very helpful. Try it!

Share:
36,564

Related videos on Youtube

Hugo
Author by

Hugo

Updated on September 18, 2022

Comments

  • Hugo
    Hugo about 1 year

    Related to this question on StackOverflow, I am wondering if there is a way for me to flush the NFS cache / force Linux to see the most up-to-date copy of a file that's on an NFS share.

    I have a situation where four Apache servers mount the same directory via NFS, and when one server make a change to a file, it takes about 5 - 10 seconds for the other servers to see that change. If a second change is made to that file within this window, it may overwrite the first change.

    The fstab entry for the filesystem is:

    172.16.1.15:/home               /media/home     nfs     vers=3,defaults,noauto,sync,acregmin=1          0       0
    

    Is there a command which will force Linux to flush cache of one file on an NFS share?

    • Admin
      Admin about 12 years
      What apache caching mechanism you using?
    • Admin
      Admin about 12 years
      @usermane: I'm not using an Apache caching mechanism. The files in question are PHP files, and when they're modified on one host, the other hosts don't see that change for a few seconds.
  • Hugo
    Hugo about 12 years
    Thanks! These are great suggestions. I am not using cto on the client and I will try that. I don't have either sync nor async on the server; I just added sync. O_DIRECT didn't help me because it's actually PHP that's reading these files and it can't open them with O_DIRECT in any way I can find.
  • RalfFriedl
    RalfFriedl about 4 years
    What is your definition of "non-cached program", and do you have a reference for that statement?