Where to find the log file of specific service

Question

Using sudo journalctl -u {service} I can see the log of specific service.

1. How to find the associated log file?
2. What is the best way to monitor a log file programmatically? (I mean a program the react based on something appears in the log file)

Solution 1

Systems with s6, runit, perp, nosh, daemontools-encore, et al. doing the service management work this way. Each main service has an individual associated set of log files that can be monitored individually, and a decentralized logging mechanism.

systemd however does not work this way. There is no individual "associated log file" for any given service. There is no such file to be monitored.

All log output is funneled into a single central dæmon, systemd-journald, and that dæmon writes it as a single stream with all services' log outputs combined to the single central journal in /{run,var}/log/journal/.

The -u option to journalctl is a post-processing filter that filters what is printed from the single central journal, all journal entries being tagged with (amongst other things) the name of the associated service. Everything fans in, and it then has to be filtered to separate it back out to (approximately) how it was originally.

The systemd way is to use journalctl -f with appropriate filters added, or write your own program directly using the systemd-specific API for its journal.

Further reading

• https://unix.stackexchange.com/a/294206/5132

Solution 2

1.) If your Linux distribution uses journald to the full extent, there will be no easily readable traditional log file, as @JdeBP mentioned. The journal file uses a binary format and is not easily parseable by traditional log file tools.

2.) If journalctl -u {service} has the information you need, then you could use journalctl -f -u {service} and pipe the output to the program that parses the log and triggers any needed reactions. Anything beyond that is likely to be specific to your Linux distribution of choice.

For example, Debian 9 (Stretch) in a default configuration has journald maintaining the traditional /dev/log syslog input socket. But if rsyslogd is installed, journald is configured to forward incoming syslog messages to it.

Other Linux distributions might have the relationship between journald and a traditional syslog daemon structured differently.