A floating point processor or floating point unit was, until recently, an option which allowed a computer to handle floating point numbers in hardware. (Now that CPU chips typically contain extremely large numbers of transistors, floating point is usually supported in them by default.).
In early machines, the hardware involved was a complete extra set of printed circuit boards; later on, with the rise of microprocessors, it became an optional chip.
In architectural terms, it might be a co-processor, or on microcoded machines, it might be extra microcode, sometimes along with some additional hardware to speed up some aspects of the computation.