PDP-11 stacks

Almost all PDP-11 models have some form of stack overflow protection, which causes a CPU trap when the stack overflows. The details of the mechanism vary from model to model; unfortunately, the details are often documented poorly, or not at all.

Fixed Limit

The first PDP-11 CPU, the KA11 CPU of the PDP-11/20, had a fixed address limit of 0400; if the stack went below this, a trap occurred after the offending operation (instruction, trap, and probably also interrupt) was over.

The KD11-B CPU of the PDP-11/05 did the exact same thing; as do the KD11-D CPU of the PDP-11/04 and the KD11-E CPU of the PDP-11/34 (see EK-KD1EA-MM-001, pg. 2-33); and the KD11-K CPU of the PDP-11/60. KDF11 CPUs have exact same mechanism (definitely on interrupts, as well).

KDJ11 CPUs have a similar mechanism, but although their CPU Error Register has separate Red and Yellow bits (below), interrupts with the SP set below 0400 only set the Yellow bit, and push both interrupt and trap PS/PC pairs; only a fault on the interrupt push causes the Red bit to be set, and then only the interrupt PS/PC are saved (in 0/2; see here).

Red/Yellow Zones

The most complex stack overflow protection mechanism first appeared in the KB11-A CPU of the PDP-11/45. In kernel mode, it had a 'two-zone' scheme: the 'Yellow Zone' is a 16-word grace area, in which operations were allowed, but result in a trap after the operation is completed; in the 'Red Zone', all operations are prohibited, and result in an immediate trap.

When Red Zone violations occur, the Stack Pointer is set to 4; the previous PC and PS are then saved in locations 0 and 2 by the resulting trap. (Odd stack addresses, and use of non-existent memory, result in identical handling.)

The address of the stack limit can be set with the Stack Limit Register register in the CPU. It is a word register (at 0777774), but the bottom byte is unused; it is cleared by a reset (e.g. when starting the CPU). The Red Zone runs up through the address given there plus 0337; the Yellow Zone starts at 0340, and runs up through 0377.

In user mode, there is a fixed Stack Overflow Boundary at 0400; attempts to write below that address cause an immediate trap.

The KB11-B CPU of the PDP-11/70 has exact same mechanism. The KD11-A CPU of the PDP-11/40 has two-zone stack limitation as above, but at a fixed address of 0400; a similar Stack Limit Register is an option, the KJ11-A Stack Limit Register. When present, it functions identically to the other two.

{PDP-11}}