Why won't strace/gdb attach to a process even though I'm root?

Question

• I logged in as root but strace gives me this:

  root@kyznecov-System:/home/kyznecov# ps -e | grep 111
   3807 pts/2    00:00:00 111
   3810 pts/2    00:00:00 111
  root@kyznecov-System:/home/kyznecov# strace -p 3810
  
  attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
  Could not attach to process.  If your uid matches the uid of the target
  process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
  again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
  root@kyznecov-System:/home/kyznecov
  
  root@kyznecov-System:/home/kyznecov# cat /proc/sys/kernel/yama/ptrace_scope
  0

• I then tried to use gdb to debug a multiprocess program in Eclipse CDT with forking, and it gave me the same result/error:

  

Any ideas?

Solution 1

One reason to get the error:

attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted

is because the process has already been attached to with gdb, strace or similar. To check if this is the case, run:

grep TracerPid /proc/$THE_PID/status

If it is nonzero, that is the pid of an existing program that is already running a trace on that process.

Solution 2

As izx has commented, this should only be able to happen due to a kernel bug. So anyone who can currently produce this problem--including and especially the original poster of this question--would be well-advised to report it as a bug by reading that page thoroughly and carefully, and then running ubuntu-bug linux on the affected machine. This should be reported against linux in Ubuntu, and not against a mainline (upstream) kernel, unless you can produce it on a mainline kernel (you'd have to have yama loaded).

The expected behavior in every version of Ubuntu starting with Ubuntu 10.10 is that process A cannot trace a running process B unless B is a direct child of A (or A runs as root). This is a security enhancement, which makes it so that a process that has been compromised by an attacker cannot use debugging facilities provided by the kernel to discover information from other processes. This is explained in the ptrace scope section of the Security Features community wiki page.

This restrictive behavior is the default but can be changed to allow a process A to trace any running process B that is run with the same user ID as process A's own. That is, you can configure your system to allow any of your processes to debug each other. This simplifies attaching debuggers to already-running processes.

The setting for this is exposed in by the /proc/sys/kernel/yama/ptrace_scope sysctl. 1 denotes the more restrictive behavior and 0 the less restrictive behavior. The setting can be read with:

cat /proc/sys/kernel/yama/ptrace_scope

The less restrictive (non-default) behavior can be set with:

echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

And the more restrictive (default) behavior can be set (or set back) with:

echo 1 | sudo tee /proc/sys/kernel/yama/ptrace_scope

Not only was the original poster of this question unable to attach an strace instance to a currently running process with ptrace-scope set to 0, but the original poster was then still unable to do so when running strace as root. It's hard to see how this could be anything but a bug--I strongly recommend reporting it as one.

At first, I had thought that I was able to reproduce the problem where a ptrace_scope setting of 0 is ignored and treated as though it is 1. But I no longer believe this is the case, as I have done all the same things again, and I cannot reproduce the problem. I have tested this on:

• The Lubuntu Precise amd64 physical machine I use daily as my main box.
• A VirtualBox virtual machine running a Lubuntu Precise i386 (12.04) live CD.
• An identical VirtualBox virtual machine running a Quantal i386 (Ubuntu+1) daily-live (20120608).

On all three machines, the expected behavior occurs, and I cannot reproduce the condition the original poster of this question is asking about. Here's some text from the Terminal (from the Precise live system):

lubuntu@lubuntu:~$ nano&
[1] 3492
lubuntu@lubuntu:~$ strace -p 3492
attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf

[1]+  Stopped                 nano
lubuntu@lubuntu:~$ cat /proc/sys/kernel/yama/ptrace_scope
1
lubuntu@lubuntu:~$ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
0
lubuntu@lubuntu:~$ strace -p 3492
Process 3492 attached - interrupt to quit
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(1, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
ioctl(1, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig -icanon -echo ...}) = ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---

strace continued producing messages until I suspended it, as expected.

I conclude by recommending again to report this as a bug. A maximally inclusive search on https://bugs.launchpad.net (which includes any reported Ubuntu bugs) for the text ptrace_scope produces just a handful of results, in which clearly none are reports for this bug. Reporting the bug would help others, may lead to workarounds or a fix, and is probably the only meaningful way to proceed forward in working on this problem (assuming the problem is still occurring).

Comments

[Oscar]

Since ptrace_scope is zero it shouldn't matter, but just as an experiment can you try a more simple usage of strace, where the tracing process is the parent of the process being traced. E.g. strace /bin/echo test? Does that give the same error message?

[ish]

@EliahKagan, in theory what OP described cannot occur in the absence of a kernel bug. If you have some time in the next 24 hours or so and can turn your comment into an answer, that would be great -- my testing on Precise has revealed no problems, and this way it would be a definitive answer that there was some quirk in the OP's config...he hasn't been back since his initial question on May 29.

[ish]

@EliahKagan, One thing just hit me on rereading the "operation not permitted" bit -- because I've seen it before tons of times when I'm supposedly root on OpenVZ VPS/VMs (they share the host machine kernel, so no, you're only the king of your own little island)...maybe this was the case? See this paste

[Kunal Tyagi]

@izx Your paste expired. And I have this issue (inside a container), any insight would be nice

[ish]

Thanks for the suggestion, I'll read it in detail tomorrow and maybe add some subheadings, but I think this is extremely thorough :)

[andreykyz]

in my post you can see: root@kyznecov-System:/home/kyznecov# cat /proc/sys/kernel/yama/ptrace_scope 0 I did it: echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

[andreykyz]

In concole all work easily. But if I ran program in eclipse and try strace from console I get said problem.

[Jamie Pate]

may be a side-effect of process already being traced? I had the same problem when using gdb on the parent process with allow-fork-mode child

[ernesto]

We don't notice this if we are debugging in Eclipse... which already is using GDB

[detly]

I'm getting this bug too on 14.04. Does anyone know if a bug was ever reported for it?

[jacman]

Thanks for pointing this out. I was running a program through an strace wrapper. The program forks and then execs itself. The execked child quickly dies as it runs the strace wrapper, which fails to attach because the parent process is already under strace's control.

[txs]

If a process is already attached to GDB, is it possible to attach Strace at the same time? Or it only works one at a time?