The kernel is the core of an operating system (in the modern, expansive sense of that term); it is usually permanently resident in main memory, and in time-sharing systems (which almost all OS's now are), it usually uses a special mode in the CPU which allows some operations (e.g. changing the configuration of the virtual memory hardware) which are forbidden to 'ordinary' user programs (usually running in processes).
In more complex systems such as Multics, there are 'layers' within the kernel, so e.g. only some of the kernel might be always resident in main memory (e.g. interrupt handlers, and the code to handle 'page faults' with virtual memory); other parts of the kernel might be 'paged'. Generally, even the higher layers will still have access to operations which ordinary user code is not allowed to use.
A recent trend is OS work is the micro-kernel, where only the most limited functionality is in the kernel; other functionality which is normally considered part of the kernel (e.g. the file system) might be run in some other environment. These sub-systems are only loaded, and started, once the micro-kernel is loaded and running.
There is still no general agreement on the use of micro-kernels; although they are generally more robust, and have capabilities that the 'classic' monolithic kernels do not have (e.g. the ability switch to a new version of the file system code while the system is running), the division usually imposes some overhead, which is un-acceptable to some.