Most CPUs support a linear (single axis) address space; memory locations are defined by a single number, the address. With segmentation, the CPU's address space can best be seen as a two-axis matrix: the segment number on one axis, and then the offset byte/word within the segment on the other; memory locations are named by a tuple (segment and offset).
Segmentation has been used to expand the amount of main memory accessible to a given CPU architecture (as in the Intel x86 architecture), but this usage is often somewhat of a kludge, a way to expand the amount of memory accessible to an existing CPU architecture later in its life.
Generally the preferred approach is to simply give a CPU a larger linear address space; this generally means making word, and instruction, sizes larger, which was not always an economically viable option when main memory was much smaller, and much more expensive. (For example, in the PDP-8 family.)
Segmentation is also used in building single-level store systems, where it is a much more natural tool. It is however possible to build such systems with CPUs which support only a very large 'flat' address space, mapping the SLS' segments into blocks of that address space.