Where are userspace programs supposed to save their logs?

Question

I am writing a script that I want to run without privileges. I want the errors that the script encounters to be logged to some log file. I do not not have privileges to write one to /var/log. And I do not want to have one in my home directory.

Is there a location where userspace scripts may log runtime information? What is the best practice to have my script log information to /var/log without creating any potential security issues? I am hesitant to set uid / gid on the script.

Solution 1

You can't write to /var/log as a normal user, but the syslog daemon will do it for you if you ask. If you'd like to log messages to the standard system logs (e.g. /var/log/syslog), the 4.4BSD utility logger might be available on your system. It's installed by default on Debian, and is in the bsdutils package on Debian derivatives.

You'll get the advantage of any pre-existing log rotation, maintenance, and monitoring tools, with the disadvantage of needing privileges to read the system logs, and of having your script's messages mixed in with messages from other programs.

$ logger Hello
$ echo Goodbye | logger
$ sudo tail -2 /var/log/syslog
Feb 19 21:16:15 debian-host jander: Hello
Feb 19 21:16:21 debian-host logger: Goodbye

There are several configuration options available; you can read more in man logger.

Solution 2

If you as an ordinary user decide to run a program, the natural place for its logs are in your home directory. Your home directory is meant for you to store all your files, whether they are logs of a program you run or anything else.

If the program is executed as part of the system, running as a typically dedicated system user, then the natural place for its logs is in /var/log. Create a subdirectory /var/log/myapp and give it appropriate permissions so that your application can write there.

If relevant and your operating system allows it, mark the log file as append-only. Only root can do this. This has the advantage that if your application is compromised, it won't be able to erase past logs, which can be very useful for forensic analysis of the compromise. You will need root's intervention to rotate the log: chown so that the log file is can no longer be opened by the application, rename the log file, create a new append-only file with appropriate ownership, then notify the application to open the new empty file.

You can make any application log to the system logs by calling logger(1) or syslog(3).

Solution 3

Generally for a daemon the log file is created by root then the permissions are altered so that the non-privileged user can write to it. logrotate is then set up to preserve permissions during rotation.

If it's a command, not a daemon, then log to /tmp (preferably using mktemp) and inform the user via STDOUT where the log went.

Solution 4

I'm under the impression userspace programs are expected to discard logs by default. I've seen various programs dump logs wherever they feel like, and its never particularly welcome on my systems; tending to build up in some location that is never noticed unless/until it gets huge.

I would prefer if there was a definite place for them, I'm playing around on my system trying to find a stable place for them.

My first idea was to use /var/run/user/$UID/log, but found that on my system, that is a TMPFS mount, not large enough, or really good for use with logs.

Create a place for them

Since I dont understand /var/run/user well enough to integrate with it, I've chosen to emulate it by hand, for user 1000.

# mkdir /var/log/user
# install -d /var/log/user/1000 --owner 1000 -g 1000 -m 0700

I would recommend sticking to the FHS /var/log spc for the structure within this folder, but the spec free-form so there isn't much to comply with.

Logrotate Config

There is no existing log rotation on this directory provided by your system, I recommend creating one for your system:

# /etc/logrotate.d/userlogs

/var/log/user/*/log/*.log 
/var/log/user/*/log/**/*.log
{
        daily
        missingok
        rotate 7
        compress
        notifempty
        nocreate
}

---

Below is my previous /var/run/user/1000/log post, I cannot recommend it unless you really know what your doing (And if you do, tell me how to too!)

maybe as follows, but I just made this up because it made sense to me.

/var/run/user/1000/log/<app>.log
/var/run/user/1000/log/<app>/<context>.log

Integrate with /var/log/user/1000:

# Integrate with above /var/run/user, probably a bad idea:
# ln -s /var/log/user/1000/ /var/run/user/1000/log

View more solutions

Comments

[Lord Loh.]

Good idea. I have a few lines in my ~/.profile that checks to see if dropbox demon is running and starts it if it is not. I was considering adding an & to avoid holding up the prompt, but starting dropbox writes to the console. I am currently redirecting it's output to /dev/null, but would like to have some way to debug if dropbox fails to start.

[bahamat]

Check out daemonize.

[Lord Loh.]

Just to know, "permissions are altered so that the non-privileged user can write to it." is this done by simply chmod 666? or do I have to add user to a group or something like that?

[vonbrand]

Better under /var/tmp, /tmp is not guaranteed to survive a boot.

[bahamat]

@LordLoh.: Yes, with chmod. Usually 0644 or 0664 and the user/group ownership is also changed to that of the daemon that is expected to be writing to it.

[ThorSummoner]

are there language bindings for this utility? Would rather not spawn sub-processes for each line of stdout in-app.

[dave_thompson_085]

@ThorSummoner: As Gilles' answer says, syslog is a C library routine, and C++ can call any C using extern "C". Other languages often provide either generic binding to anything C, or bindings to specific things, but it depends on the language.

[balasaheb barde]

in common lisp there is cl-syslog

[Aspiring Dev]

This is an awesome answer. Testing it now, thanks!

[ThorSummoner]

it might also make sense to simply run lodrotate as your user rather than make root do it for ya, always lately i just log to stderr and let the user decide if they care to keep them somewhere, i don't bother with lodrotate for my user as i normally buffer my logs into less, this one time i had a 50gb plus buffer in less, it was horribly slow but still worked, there may also be the option of emitting logs to syslog, or journald which will "manage" the log for you more or less (respectively)

[gregn3]

This is how to mark a log file append-only: https://unix.stackexchange.com/q/59864/68851

[Jared Beach]

Can you modify this answer to include the information that most user applications create a hidden folder in the home directory? I think the best solution for most applications would be to follow that practice. Your answer could be interpreted to just fill up the /home/my-user directory with a ton of log files

[Jason Harrison]

@thorsummoner using a separate process to write logs to files solves a lot of problems with log rotation that you otherwise would have to implement in your application or have implemented in your logging library. See jdebp.info/FGA/do-not-use-logrotate.html

[Jason Harrison]

Logging to stdout means that the user of your application can use one of many different log management tools. jdebp.info/FGA/do-not-use-logrotate.html

[they]

logger is actually a POSIX utility.

[they]

The premise for this question is that the utility runs without root permissions.