PDP-11 stacks

The PDP-11 makes heavy use of stacks, for which it has good support via the auto-increment and auto-decrement addressing modes. However, there are a number of sometimes poorly-documented details of their operation, which ward off potential issues with the main stack (pointed to by R6, the main Stack Pointer).

• Byte operations (push and pop) on the R6 stack always adjust the SP by 2, not 1; this to ensure that it always contains a word address, should an un-expected interrupt or trap require it to be used to save the old PS and PC.

• On CPU models intended for use in time-sharing systems, which support User and Kernel modes, there are separate R6's for the two modes, so that the operating system can use the latter one for interrupt and trap handling, and thereby guarantee that the user cannot derange R6's contents before it needs to be used.

• Almost all models have some form of stack address limitation, which causes a trap when the stack overflows (or tries to). The details vary from model to model; see below for details.

• When an interrupt or trap occurs, if R6 does contain either an odd (i.e. non-word) address, or an address which does not point to functional main memory, on many models R6 is then bashed to contain 4, after which the old PC and PS are saved in 0 and 2 (respectively); after that, the CPU takes an odd-address/non-existent-memory trap. (KDF11 CPUs ignore odd stack addresses, and proceed as if the bottom address bit in their SP is 0. Apparently LSI-11 CPUs also ignores odd stack addresses.)

On the KA11 CPU (PDP-11/20), KD11-B CPU (PDP-11/05), KD11-D CPU (PDP-11/04), KD11-E CPU (PDP-11/34), KD11-Z CPU (PDP-11/44), and LSI-11 CPUs, a fault during a trap will halt the CPU.

Stack limits

Almost all PDP-11 CPU models have some form of stack overflow protection (in kernel mode only, for the models which support modes), which causes a CPU trap when the stack overflows. (LSI-11 CPUs have no overflow detection.) The details of the mechanism vary from model to model; unfortunately, the details are often documented poorly, or not at all.

Fixed limit

The first PDP-11 CPU, the KA11 CPU of the PDP-11/20, had a fixed address limit of 0400; if the stack went below this, a trap occurred after the offending operation (instruction, trap, and probably also interrupt) was over.

The KD11-B CPU of the PDP-11/05 did the exact same thing; as do the KD11-D CPU of the PDP-11/04 and the KD11-E CPU of the PDP-11/34 (see EK-KD1EA-MM-001, pg. 2-33). KDF11 CPUs have the same mechanism (definitely on interrupts, as well; two PS/PC pairs, one from the interrupt, and one from the trap, are pushed when this happens).

KDJ11 CPUs have a similar mechanism, but although their CPU Error Register has separate Red and Yellow bits (below), interrupts when the SP is set below 0400 only set the Yellow bit, and push both PS/PC pairs. Only a fault on the interrupt push causes the Red bit to be set, and then only the interrupt PS/PC are saved (in 0/2; see above for more).

Red/Yellow Zones and Stack Limit Registers

The most complex stack overflow protection mechanism first appeared in the KB11-A CPU of the PDP-11/45. In Kernel mode, it had a 'two-zone' scheme:

• The 'Yellow Zone' is a 16-word grace area, in which operations (either a 'push' by an instruction, or the pushes caused by the saving of the previous PC and PS in a trap or an interrupt) were allowed, but result in a trap after the operation is completed
• In the 'Red Zone', all operations are prohibited, and result in aborting of the operation, and an immediate trap, as below.

When a Yellow Zone trap occurs, further Yellow Zone violations are suppressed for the push of the previous PC and PS (since otherwise recursive violations would be caused by attempting to take the trap). However, the trap handler must not attempt to push anything onto the existing stack, since doing so could cause yet another violation, and resultant trap.

When Red Zone violations occur, the SP is set to 4; the previous PC and PS are then saved in locations 0 and 2 by the resulting trap. (Odd stack addresses, and use of non-existent memory, result in identical handling.)

The address of the stack limit can be set with the Stack Limit Register in the CPU. It is a word register (at 0777774), but the bottom byte is unused; it is cleared by a reset (e.g. when starting the CPU). The Red Zone runs up through the address given there plus 0337; the Yellow Zone starts at that address plus 0340, and runs up through plus 0377.

In User mode, there is a fixed Stack Overflow Boundary at 0400; attempts to write below that address cause an immediate trap.

The KB11-B CPU of the PDP-11/70 has exact same mechanism, as does the KD11-K CPU of the PDP-11/60.

The KD11-A CPU of the PDP-11/40 has the two-zone stack limitation as above, but at a fixed address of 0400. A similar Stack Limit Register is available as an option, the KJ11-A Stack Limit Register; when present, it functions identically to the others.