Microcode (in the form of a microprogram, or microinstructions) refers to a technique for implementing the functionality needed to control the internal operation of a CPU: fetching instructions, and then, in response to those instructions, routing data out of registers and/or main memory, through the ALU, and then storing the results.
The earliest generation of CPU's used collections of gates and flip-flops to implement this functionality. As the instruction sets and overall architecture of early computers became more complex, the design of those combinatorial logic networks became more challenging.
In 1947, the Whirlwind computer introduced the concept of a control store; a wide read-only memory whose outputs directly controlled elements of the CPU. (E.g. one bit in a control word might control the latching of an internal register.) The CPU cycled through the words in the control store to execute instructions. Viewing (and implementing) the internal operation of the CPU in this way made the design task more feasible.
In 1951, Maurice Wilkes suggested adding conditional branching, to make microcode more flexible. Over time, more concepts from higher-level languages have been added to microcode, such as subroutines.
Although early microcode was implemented in read-only memory (the Whirlwind ROM was a diode matrix), some later computers included high-speed RAM for part of the control store, so that the instruction set could be dynamically extended to speed up applications.
Recently, microcode has been augments with programmable logic arrays, which are usually given the job of decoding instructions (something done more efficiently in combinatorial logic).
With much of the control for a microprogrammed machine in the microprogram, bugs in the CPU could often be fixed by replacing the microprogram alone, without required hardware changes. (With the coming of the microprocessor, where the microprogram is often in ROM in the chip, this is usually no longer possible.)
Microprogramming also made it possible to offer a range (in performance and cost terms) of machines which all had the same basic architecture, but different internal implementations; microcode could be used to make the different machines all support the same instruction set. This too is now not common, since it is not usual to offer multiple implementations with differing performance at the same time; however, it can be used to meet an analogous goal, making new generations of microprocessors implement the identical instruction set to previous ones.
For much of the lifetime of computers, the CPU was faster than the main memory; in such an environment, it makes sense to make the instructions of the CPU as high-level as possible, so a single instruction fetch (which is, of course, pure overhead in terms ot the actual computation) does as much as possible. Such complex instructions are more easily implemented with a microcode approach, than with combinatorial logic.