Virtual Memory and Paging: How Operating Systems Manage RAM

Every program believes it owns a vast, contiguous block of memory. In reality, it shares physical RAM with dozens of other processes, and some of its memory may not even be in RAM at all. Virtual memory is the OS illusion that makes this work.

When you run a program, it operates with virtual addresses — addresses in its own private address space. On a 64-bit system, this space can be up to 128 TB, far larger than any physical RAM. The operating system, with help from the CPU's memory management unit (MMU), translates those virtual addresses into physical addresses every time the program reads or writes memory. Paging is the mechanism that performs this translation.

The Three Layers: Virtual Address, Page Table, Physical Frame

1. Virtual address space: Each process has its own virtual address space. Addresses in one process mean nothing in another. This isolation prevents processes from reading or corrupting each other's data and is one of the fundamental security guarantees of modern operating systems.
2. Page table: The OS maintains a page table for each process. A page table is a data structure that maps virtual page numbers to physical frame numbers. On a modern 64-bit system, page tables are multi-level (typically four levels on x86-64) to avoid storing an entry for every possible virtual page, most of which are not used.
3. Physical frame: Physical RAM is divided into fixed-size blocks called frames (typically 4 KB). When a virtual page is mapped to physical memory, it occupies one frame. The page table entry records which frame holds which virtual page, along with permission bits (readable, writable, executable) and status bits (present, dirty, accessed).

Key Terms

TLB Hits vs Misses: Why Translation Is Fast (Usually)

Walking a four-level page table takes four memory accesses before the CPU even gets to the actual data. On a modern processor running at 3 GHz, four cache misses could cost 200+ nanoseconds — an eternity. The TLB is a small, fast hardware cache inside the CPU that stores recent virtual-to-physical translations.

When the CPU translates an address, it first checks the TLB. A TLB hit provides the physical address in roughly 1 nanosecond. A TLB miss requires a page table walk (either in hardware or by the OS, depending on the CPU architecture) costing 20 to 100+ nanoseconds depending on whether the page table entries are in the CPU's data cache. Programs with high spatial and temporal locality rarely miss the TLB because they reuse the same pages repeatedly. Programs that access memory randomly across a large address space suffer frequent TLB misses — this is a significant source of hidden latency in large working-set applications like databases and in-memory analytics engines.

Page Faults: What Triggers Them and What Happens

A page fault occurs when the CPU encounters a PTE that is marked not-present. The hardware raises an exception, suspends the current instruction, and transfers control to the OS page fault handler. The handler determines why the page is not present and takes one of several actions:

• Minor fault (soft fault): The page exists in memory but is not yet mapped in this process's page table. This happens on first access to pages in an mmap region. The OS simply creates the PTE without disk I/O. Cost: a few microseconds.
• Major fault (hard fault): The page is on disk (either in swap or as a file-backed mapping). The OS must read the page from disk, which takes 0.1 to 10 milliseconds depending on storage speed. During this time the process is blocked. On an SSD, a major fault adds roughly 100 to 500 microseconds. On a spinning disk, 5 to 15 milliseconds.
• Segmentation fault: The accessed address is not mapped at all and violates the process's permissions. The OS sends SIGSEGV and the process terminates.

Memory-Mapped Files: Fast I/O Through Paging

Memory-mapped files (mmap) are one of the most useful applications of the paging mechanism. Instead of calling read() to copy file contents into a buffer, you ask the OS to map a file directly into the process's virtual address space. From that point on, accessing the mapped addresses reads the file through the paging mechanism: the first access to each page causes a minor or major fault that loads that page from the file. Subsequent accesses hit the page in RAM (or the TLB) with no system call overhead.

Databases, JVM runtimes, and many language interpreters use mmap extensively. SQLite reads its database file through mmap by default on platforms where it is safe. The kernel manages the file cache transparently, and multiple processes mapping the same file share physical pages, avoiding duplicate copies in RAM. The OS also handles writeback lazily: modified ("dirty") pages are flushed to disk when the OS decides, not on every write, making writes fast even for large files.

Understanding paging also explains copy-on-write (COW) semantics after fork(): parent and child share physical pages marked read-only. The first write to any shared page triggers a fault that copies just that one page, separating the two processes at minimal cost. This is why forking a large process is fast even when the process has gigabytes of address space.

Related: Compilers and Interpreters: How Code Actually Runs · Backtracking Algorithms: Solving Constraint Problems with Recursive Search