How do you force a process to flush the data written to an open file descriptor under Linux?

Question

I have a binary process (with no sources available) running on an embedded Linux system. The process opens a log file (/tmp/dmaosd.log) where it logs what it's doing.

The problem is the log gets updated in chunks (about 1000 bytes at a time), so I can't watch the log in real time with tail -f.

I wonder if there is any way to force the running process to flush its data (based on accessing /proc/1234/fd/3) without having access to its sources and without sending any signals to it (I'm not sure what signals it supports nor what they are supposed to do).

The process in question is an OSD for a media player, and the log info usually shows what elements are selected/shown onscreen, so it would be nice to have the data as quickly as possible.

Thanks!

Solution

It really depends on where the buffer is: if the application uses its own logging buffer then there's no way of forcing a flush.

If the buffering is done by the C library then you can use LD_PRELOAD to disable the buffering. Assuming that the program is using fopen() to open its log file you can do something like this:

#define _GNU_SOURCE 1
#include <dlfcn.h>
#include <stdio.h>

static FILE* (*libc_fopen)(char const *, char const *);

FILE * fopen(char const *name, char const *mode) {
    FILE *ret;
    if (!libc_fopen)
        libc_fopen = dlsym(RTLD_NEXT, "fopen");

    ret = libc_fopen(name, mode);
    if (!ret)
        return ret;

    setvbuf(ret, NULL, _IONBF, 0);

    return ret;
}

Compile as a shared library and then use LD_PRELOAD to "inject" into the program:

LD_PRELOAD=./nobuffering.so myprogram

If you want you can also check the name of the file and change the buffering only for the log file you're interested in.

Comments

[Admin]

Are you looking for "sync"? according to man page, sync() causes all buffered modifications to file metadata and data to be written to the underlying file systems.

[Namit Gupta]

Something like sync(), yes, but the problem is, in order to use it I would have to change the process' source code and add calls to sync() when adding data to the log file. This is something I can't do, because I don't have the sources. Can I call sync() on a file descriptor opened by a different process? If so, how?

[John Siu]

(1) Try sync in command line. (2) If it is always 1k each time, maybe be fs cache.

[Namit Gupta]

It shouldn't be filesystem cache. the log is written to a ramdrive (/tmp is a ramdisk). I will try running sync (and report back), but I thought it would only flush filesystem buffers to disk, not program buffers...

[Namit Gupta]

I confirm that running the sync command has no effect on the log's flush policy. The new content still gets flushed to file only when reaching a certain size... Any suggestions?

[Michael Hampton]

You probably should also hook setvbuf() in case the program is explicitly calling it.

[Namit Gupta]

That's an interesting idea - overriding what fopen does. I will see if I can give it a try and trick the binary to use my library calls.