How to track the number of processes and forks per user?

Question

I need to figure out how many forks are done and how many concurrent processes are run by each user over time. It does not look like this information is tracked by my distribution.

I know how to sets limits, but I'm interested in tracking these numbers for each user.

Solution

Try the psacct package (GNU accounting), it should do just about everything you need, once installed and enabled (accton), then lastcomm will keep report on user processes (see also sa and dump-acct). See this for reference: User's executed commands log file

You might need to upgrade the version to log PID/PPID, see https://serverfault.com/questions/334547/how-can-i-enable-pid-and-ppid-fields-in-psacct-dump-acct , otherwise I suspect it will under-report on fork() without exec().

Update If your lastcomm outputs F in the 2nd column it means the process was a fork (that never called exec() to replace itself with a new process). The output of dump-acct should show you the PID (and PPID) in acct v3 format.

An alternative to psacct might be the new(ish) taskstats, there's not a huge amount of support for it yet AFAICT, see Documentation/accounting/taskstats.txt in your kernel version source. This might help get you started http://code.google.com/p/arsenalsuite/wiki/TrackingIOUsage https://code.google.com/archive/p/anim-studio-tools/ The specific code example is tasklogger.c, you will need to modify the printf() line in function print_delayacct2(), firstly to replace %u with %llu for the __u64 types and secondly to add the field ac_uid (and perhaps ac_gid) that you need to track by user. Invoke it with something like tasklogger -dl -m 0-1 (where -m 0-1 indicates CPUs 0-1). You will then see realtime details as each process exits.

There is also a perl module Linux::Taskstats::Read available on CPAN, though I have not used it.

You'll need to process the data based on timestamps if you want the concurrent process count per-user, this is not a simple as it sounds.

Update 2 Ok, the things to check for the required psacct support are:

1. (official) kernel >= 2.6.8 for v3 accounting support (or backport)
2. kernel with CONFIG_BSD_PROCESS_ACCT and CONFIG_BSD_PROCESS_ACCT_V3 enabled
3. v3 capable accounting (psacct) package, as noted above

All of the above should be true in CentOS 6, I've checked a 5.x and it does not have CONFIG_BSD_PROCESS_ACCT_V3=y, so you would have to rebuild your kernel to enable it.

The original psacct-6.3.2 is about 15 years old, the Red Hat/CentOS version has backported v3 and PID display support (I can't test it right now, but it should work).

To check a your kernel config:

zgrep BSD_PROCESS_ACCT /proc/config.gz /boot/config-`uname -r`

Comments

[Gilles 'SO- stop being evil']

Process accounting only tracks execs, not forks.

[mr.spuratic]

@Gilles, mine does (psacct-6.4pre1, kernel 2.6.31.6, Slackware 13). Tested with code that calls fork(), and using bash ( ... ) subshells. Answer updated to clarify.