Microcode (in the form of a microprogram, or microinstructions) refers to a technique for implementing the functionality needed to control the internal operation of a CPU: fetching instructions, and then, in response to those instructions, routing data out of registers and/or memory, through the ALU, and then storing the results.
The earliest generation of CPU's used collections of gates and flip-flops to implement this functionality. As the instruction sets and overall architecture of early computers became more complex, the design of those combinatorial logic networks became more challenging.
In 1947, the Whirlwind computer introduced the concept of a control store; a wide read-only memory whose outputs directly controlled elements of the CPU. (E.g. one bit in a control word might control the latching of an internal register.) The CPU cycled through the words in the control store to execute instructions. Viewing (and implementing) the internal operation of the CPU in this way made the design task more feasible.
In 1951, Maurice Wilkes suggested adding conditional branching, to make microcode more flexible. Over time, more concepts from higher-level languages have been added to microcode, such as subroutines.
Although early microcode was implemented in read-only memory (the Whirlwind ROM was a diode matrix), some later computers included high-speed RAM for part of the control store, so that the instruction set could be dynamically extended to speed up applications.
Recently, microcode has been augments with programmable logic arrays, which are usually given the job of decoding instructions (something done more efficiently in combinatorial logic).