How to detect possible / potential stack overflow problems in a c / c++ program?

Question

Is there a standard way to see how much stack space your app has and what the highest watermark for stack usage is during a run?

Also in the dreaded case of actual overflow what happens?

Does it crash, trigger an exception or signal? Is there a standard or is it different on all systems and compilers?

I'm looking specifically for Windows, Linux and Macintosh.

Solution 1

On Windows a stack overflow exception will be generated.

The following windows code illustrates this:

#include <stdio.h>
#include <windows.h>

void StackOverFlow()
{
  CONTEXT context;

  // we are interested control registers
  context.ContextFlags = CONTEXT_CONTROL;

  // get the details
  GetThreadContext(GetCurrentThread(), &context);

  // print the stack pointer
  printf("Esp: %X\n", context.Esp);

  // this will eventually overflow the stack
  StackOverFlow();
}

DWORD ExceptionFilter(EXCEPTION_POINTERS *pointers, DWORD dwException)
{
  return EXCEPTION_EXECUTE_HANDLER;
}

void main()
{
  CONTEXT context;

  // we are interested control registers
  context.ContextFlags = CONTEXT_CONTROL;

  // get the details
  GetThreadContext(GetCurrentThread(), &context);

  // print the stack pointer
  printf("Esp: %X\n", context.Esp);

  __try
  {
    // cause a stack overflow
    StackOverFlow();
  }
  __except(ExceptionFilter(GetExceptionInformation(), GetExceptionCode()))
  {
    printf("\n****** ExceptionFilter fired ******\n");
  }
}

When this exe is run the following output is generated:

Esp: 12FC4C
Esp: 12F96C
Esp: 12F68C
.....
Esp: 33D8C
Esp: 33AAC
Esp: 337CC

****** ExceptionFilter fired ******

Solution 2

On Linux you get a segmentation fault if your code tries to write past the stack.

The size of the stack is a property inherited between processes. If you can read or modify it in the the shell using commands like ulimit -s (in sh, ksh, zsh) or limit stacksize (tcsh, zsh).

From a program, the size of the stack can be read using

#include <sys/resource.h>
#include <stdio.h>

int main() {
    struct rlimit l;
    getrlimit(RLIMIT_STACK, &l);
    printf("stack_size = %ld\n", l.rlim_cur);
    return 0;
}

I don't know of a standard way to get the size of the available stack.

The stack starts with argc followed by the contents of argv and a copy of the environment, and then your variables. However because the kernel can randomize the location of the start of the stack, and there can be some dummy values above argc, it would be wrong to assume that you have l.rlim_cur bytes available below &argc.

One way to retrieve the exact location of the stack is to look at the file /proc/1234/maps (where 1234 is the process ID of your program). Once you know these bounds you can compute how much of your stack is used by looking at the address of the latest local variable.

Solution 3

gcc places an extra block of memory between the return address and the normal variables in "unsafe" function calls, like (in this example the function is void test() {char a[10]; b[20]}:

call stack:
-----------
return address
dummy
char b[10]
char a[20]

If the function write 36 bytes in the pointer 'a', the overflow will 'corrupt' the return address (possible security breach). But it will also change the value of the 'dummy', that is between the pointer and the return address, so the program will crash with a warning (you can disable this with a -fno-stack-protector)

Solution 4

On Linux, the Gnu libsigsegv library includes the function stackoverflow_install_handler, which can detect (and in some cases help you recover from) stack overflow.

Solution 5

On windows, the stack (for specific thread) grows on-demand until the stack size specified for this thread prior to its creation has been reached.

On-demand growing is impelmented using guard pages, in that there's a only a fragment of stack available initially, followed by a guard page, which, when hit, will trigger an exception - this exception is special, and is handled by the system for you - the handling increases the available stack space (also checked if a limit has been reached!) and the read operation is retried.

Once the limit is reached, there's no more growing which results in stack overflow exception. The current stack base and limit are stored in thread environment block, in a struct called _NT_TIB (thread information block). If you have a debugger handy, this is what you see:

0:000> dt ntdll!_teb @$teb nttib.
   +0x000 NtTib  : 
      +0x000 ExceptionList : 0x0012e030 _EXCEPTION_REGISTRATION_RECORD
      +0x004 StackBase : 0x00130000 
      +0x008 StackLimit : 0x0011e000 
      +0x00c SubSystemTib : (null) 
      +0x010 FiberData : 0x00001e00 
      +0x010 Version : 0x1e00
      +0x014 ArbitraryUserPointer : (null) 
      +0x018 Self   : 0x7ffdf000 _NT_TIB

The StackLimit attribute will get updated on-demand. If you check the attributes on this memory block, you'll see something similar to that:

0:000> !address 0x0011e000 
    00030000 : 0011e000 - 00012000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000004 PAGE_READWRITE
                    State    00001000 MEM_COMMIT
                    Usage    RegionUsageStack
                    Pid.Tid  abc.560

And checking a page next to it reveals the guard attribute:

0:000> !address 0x0011e000-1000
    00030000 : 0011d000 - 00001000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000104 PAGE_READWRITE | PAGE_GUARD
                    State    00001000 MEM_COMMIT
                    Usage    RegionUsageStack
                    Pid.Tid  abc.560

Hope it helps.

View more solutions

Comments

[masterxilo]

Related: stackoverflow.com/questions/389219/…

[Nihar Karve]

"I'm looking specifically for Windows, Linux and Macintosh" - quite specific then :)

[Greg D]

I don't believe there is a standard way to get the size of the available stack. Does the standard even define the existence of a stack?

[Martin York]

Correct me if I am wrong as I am not sure. Your code illustrate the simple case. But if the stack temporarily overflows into the heap then re-treats this will not always trigger the overflow exception as this detection mechanism is running in another thread.

[jussij]

In the example I posted each call to the StackOverFlow function advances the stack pointer (as seen by the print out) and eventually that pointer hits invalid memory.

[Y00]

@GregD there is an indirect way 1. get max stack size 2. get current stack size 3. do A-B