What exactly is a trap handler?

Question

As far as I know, trap is something that happens when special events occur. In case of a system call, program executes trap instruction and jumps to kernel mode. Then, trap handler jumps to desired handler (e.g. fork, exec, open).

When the execution of fork, exec, open, etc. is finished, the OS calls return-from-trap instruction and makes the program go back to user mode.

But what exactly is a trap handler? (Also, if you may, what is a trap table?)

Solution

The trap handler is the code that will run when the trap is triggered. In your example, the OS will have installed a handler (i.e. told the CPU a memory address of code to run when the trap happens), and the handler will execute the system call. It is NOT the program that jumps to kernel mode. The program is interrupted immediately after it triggers the trap. Execution resumes with the trap handler.

This way, the three layers (program that runs in protected mode, operating system that runs in privileged mode and CPU/hardware that enforces that currently executing code cannot break out of protected mode) can hand off control between each-other.

Also note that a) modern CPU have dedicated instructions for system calls -- a mechanism that is more efficient than trapping but conceptually works the same and that b) there are other traps/interrupts used for different purposes as well -- they provide the basic mechanism for stopping sequential program execution and do "something else" in reaction to some kind of event.

Comments

[hansvb]

Read this: stackoverflow.com/questions/16649824/…

[Uwe Plonus]

Possible duplicate of The relation between privileged instructions, traps and system calls

[Van]

Thanks! That answered my question! Another question: do you know what a trap table is? And what is indexed into the trap table?

[hansvb]

That is the dispatch table that the OS uses to tell the CPU where it should jump to when the trap is triggered. It contains memory addresses of the handler code for each type of trap. The code there will be part of the OS, run in privileged mode and it will (in your case) execute the system call.

[Pankwood]

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.