Virtual DOS Machine

A Virtual DOS Machine (VDM for short) is a term used by Microsoft to denote a vm86 machine, a special mode of the i386 CPU and later CPUs up to the Nehalem-C architecture that would create a virtual 8086 machine. The Westmere architecture and later, uses a cpu feature called VT-x ( Virtual execution ) and VT-d ( Virtual device )

To use virtual 8086 mode, an operating system sets up a virtual 8086 mode monitor, which is a program that manages the real mode program and emulates or filters access to system hardware and software resources. The monitor must run at privilege level 0 and in protected mode. Only the 8086 program runs in VM86 mode and at privilege level 3. When the real-mode program attempts to do things like access certain I/O ports to use hardware devices or access certain regions in its memory space, the CPU traps these events and calls the V86 monitor, which examines what the real mode program is trying to do and either acts as a proxy to interface with the hardware, emulates the intended function the real-mode program was trying to access, or terminates the real-mode program if it is trying to do something that cannot either be allowed or be adequately supported (such as reboot the machine, set a video display into a mode that is not supported by the hardware and is not emulated, or write over operating system code).

The V86 monitor can also deny permission gently by emulating the failure of a requested operation—for example, it can make a disk drive always appear not ready when in fact it has not even checked the drive but simply will not permit the real-mode program to access it. Also, the V86 monitor can do things like map main memory pages, intercept calls and interrupts, and preempt the real-mode program, allowing real-mode programs to be multitasked like protected-mode programs. By intercepting the hardware and software I/O of the real-mode program and tracking the state that the V86 program expects, it can allow multiple programs to share the same hardware without interfering with each other. So V86 mode provides a way for real-mode programs designed for a single-tasking environment (like DOS) to run concurrently in a multi-tasking environment.

A strange side effect of the early implementation of V86 mode was that if paging was off, the DOS prompt would run in protected mode, and if paging was on, the DOS prompt would run as a virtual-86 machine mode task. There was a utility to test which mode the DOS prompt was running in.