Processes, memory and signals

A process is the kernel's unit of "a running program": some memory, some open files, a place in the scheduler's queue, and a number. This page follows one from birth to death, then looks at the memory it thinks it owns, and at the messages the kernel uses to interrupt it.

The map

Read this first when short on time. Every branch is a section below.

A process: a program with its own address space and a number

A process is the kernel's record of one running program. The kernel keeps a structure for it (the task_struct) that holds everything the program owns or is owed.

• An address space. The memory the program can see, described by page tables. No other process can see it.
• A file descriptor table. The open files, sockets and pipes, numbered from 0.
• Credentials. User and group IDs, capabilities.
• A signal table. Which signals are blocked, which have handlers.
• A place in the scheduler. Its state, priority, how much CPU it has used.
• Numbers. Its PID, its parent's PID, its process group and session.

The PID is just an integer, handed out in increasing order and wrapped at pid_max (32768 by default, often raised). Every process has a parent, so all processes form one tree under PID 1. pstree draws it.

A thread is a process that shares its address space, file table and signal handlers with another. Linux does not have a separate object for threads. It creates them with clone and a set of flags saying what to share. Each thread has its own PID internally (called a TID in user space), its own stack and registers, and the group of them shares one thread group ID, which is what getpid returns. ps -T or ls /proc/PID/task shows them.

States

At any moment a process is in one state, shown as a letter in ps and top.

stateDiagram-v2
  [*] --> R: fork
  R --> S: waits for a lock, a socket, a timer
  S --> R: woken up
  R --> D: waits for disk or NFS
  D --> R: I/O done
  R --> T: SIGSTOP, Ctrl-Z, debugger
  T --> R: SIGCONT
  R --> Z: exit
  Z --> [*]: parent calls wait

PID

Process ID. A small integer, unique while the process exists, reused later.

PPID

Parent process ID. The process that created this one.

Thread group

A set of threads sharing one address space. Its ID is what user space calls the PID.

fork and exec: how every process is born

Linux makes a new process in two steps that are deliberately separate: fork copies the current process, and exec replaces the copy's program with a new one. Almost every process on the machine was created this way by its parent.

sequenceDiagram
  autonumber
  participant Sh as Shell (PID 500)
  participant K as Kernel
  participant C as Child (PID 501)
  Sh->>K: fork()
  K->>C: create a copy of the shell
  K-->>Sh: returns 501
  K-->>C: returns 0
  C->>K: execve("/bin/ls", argv, envp)
  K->>C: throw away the shell image, load ls
  Sh->>K: wait4(501)
  Note over Sh: sleeping in S
  C->>K: exit_group(0)
  K-->>Sh: wait returns, status 0

fork

fork creates a child that is a near-exact copy of the parent: same code, same memory contents, same open file descriptors, same signal handlers. It returns twice. In the parent it returns the child's PID; in the child it returns 0. That is how the code after it knows which one it is.

Copying a whole address space would be slow, so the kernel cheats with copy-on-write: both processes share the same physical pages, marked read-only, and a page is copied only when one of them writes to it. A fork of a 1 GB process copies page tables, not gigabytes. The details are in the page faults section below.

Open file descriptors are shared too, and they share the file offset. Two processes writing to the same descriptor after a fork will interleave correctly rather than overwrite each other, which is why redirecting a shell pipeline's output works.

exec

execve throws away the current program and loads a new one into the same process. The PID stays. Open file descriptors stay unless they were opened with O_CLOEXEC. Signal handlers are reset to default. The memory is gone: new code, new stack, new heap.

The kernel reads the file, checks the first bytes to pick a loader (ELF for binaries, #! for scripts, which runs the interpreter with the script as its argument), maps the segments into memory, and jumps to the entry point. For a dynamically linked program the entry point is actually the dynamic loader, which maps libc and the other libraries before calling main.

The reason for two calls instead of one is the gap between them. The child can change the environment before it loads the program: redirect standard output to a file, close descriptors, change directory, drop privileges, join a container namespace. Then exec. The new program starts with everything already arranged.

#include <unistd.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <stdio.h>

int main(void) {
    pid_t pid = fork();
    if (pid == 0) {
        // child: redirect stdout to a file, then become ls
        int fd = open("out.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
        dup2(fd, 1);          // fd 1 now points at out.txt
        close(fd);
        execlp("ls", "ls", "-l", (char *) NULL);
        perror("execlp");      // only reached if exec failed
        _exit(127);
    }
    // parent: wait for the child and read its exit status
    int status;
    waitpid(pid, &status, 0);
    if (WIFEXITED(status))
        printf("ls exited with %d\n", WEXITSTATUS(status));
    return 0;
}

That is exactly what the shell does for ls -l > out.txt. A pipeline is the same trick with a pipe's two ends dup'ed onto fd 1 of one child and fd 0 of the next.

wait, zombies and orphans

When a process exits, its memory and files are released at once, but its entry in the process table stays until the parent asks for the exit status with wait. Until then it is a zombie: a PID and an exit code, nothing else. Zombies take no memory to speak of, but they take a PID, and a parent that forks forever and never waits will run the table dry.

If the parent dies first, the child is an orphan and the kernel reparents it to PID 1, or to the nearest ancestor that declared itself a subreaper with prctl. PID 1 waits for its adopted children, which is how zombies eventually clear. This is also why a container whose PID 1 is a plain application can fill with zombies: the app never calls wait for children it did not know it had. tini and docker run --init exist to be a proper PID 1.

fork

Create a child process that is a copy of the caller. Returns the child's PID to the parent and 0 to the child.

exec

Replace the current process's program with another one. Same PID, new memory.

Zombie

A process that has exited but whose parent has not yet read its exit status.

Subreaper

A process that has asked to adopt orphaned descendants instead of PID 1 doing it.

The scheduler: who runs next

There are more runnable tasks than CPUs, so the scheduler picks. It keeps one run queue per CPU and tries to give every runnable task a fair share of time, weighted by priority.

The fair scheduler was CFS for many years, and since kernel 6.6 it is EEVDF. The idea behind both: track how much CPU each task has had, and run the one that is furthest behind its fair share. A task that just woke up after sleeping is behind, so it runs almost at once, which is what makes interactive programs feel responsive. A task that has been computing for a while is ahead, so it waits. There is no fixed time slice; the scheduler aims for a target latency and divides it among the runnable tasks.

nice shifts a task's weight. It runs from -20 (most CPU) to 19 (least). Each step is roughly 10 percent more or less CPU when the machine is busy, and it does nothing at all when the machine is idle. Real-time classes (SCHED_FIFO, SCHED_RR) sit above all of that: a runnable real-time task always beats a normal one, which is useful for audio and dangerous for anything with a bug.

The load average in uptime and top is the number of tasks that are runnable or in D state, averaged over 1, 5 and 15 minutes. On a 4-CPU machine a load of 4 means every CPU has one thing to do; 8 means tasks are queueing. Because D counts, a machine waiting on a dead NFS server can show a huge load with idle CPUs.

Run queue

The per-CPU list of tasks ready to run.

nice

A priority hint from -20 to 19. Higher is nicer, meaning less CPU.

Load average

Runnable plus uninterruptible tasks, smoothed over 1, 5 and 15 minutes.

Virtual memory: every process thinks it has the whole machine

A process never sees physical RAM addresses. It sees a private, contiguous address space of its own, and the CPU translates every access through the process's page tables. That indirection is what gives isolation, sharing and lazy allocation all at once.

On x86-64 the usable space is 48 bits, 128 TB for user space and another 128 TB for the kernel at the top. Inside it, a process's memory is laid out in regions. cat /proc/self/maps lists them.

The bases of the heap, stack and mmap area are randomised at every exec (ASLR) so an attacker cannot guess where anything is.

Page tables

Memory is managed in pages, 4 KB each. The page tables map each virtual page number to a physical one, plus permission bits: readable, writable, executable, present, accessed, dirty. With 48-bit addresses a flat table would be enormous, so it is a tree, four levels deep (five on machines that enable it). Most of the tree is empty and never allocated.

flowchart TD
  V["Virtual address from the program"] --> T{"TLB has it?"}
  T -- yes --> P["Physical address, done in a cycle"]
  T -- no --> W["MMU walks four levels of page tables"]
  W --> E{"Entry present?"}
  E -- yes --> C["Cache it in the TLB, continue"]
  E -- no --> F["Page fault: trap into the kernel"]
  F --> H["Kernel maps a page or kills the process"]

The MMU in the CPU does the translation in hardware. Walking four levels for every access would be far too slow, so the TLB, a small cache of recent translations, catches nearly all of them. A context switch to another process changes the page tables, which is part of why context switches are expensive: the TLB entries for the old process are no longer valid.

Because the mapping is per process, the same physical page can appear in many address spaces. libc is loaded once into RAM and mapped into every process on the machine. A file mapped with MAP_SHARED is one set of pages that every mapper sees, which is the basis for shared memory in the IPC article. MAP_PRIVATE gives a copy-on-write view instead.

Page

The unit of memory mapping, 4 KB on x86-64 and most ARM systems.

Page table

The per-process tree that maps virtual pages to physical ones with permission bits.

MMU

Memory management unit. The CPU hardware that walks the page tables.

TLB

Translation lookaside buffer. A cache of recent page table lookups.

ASLR

Address space layout randomisation. Random base addresses per exec.

Page faults, copy-on-write and swap

A page fault is what happens when a program touches a virtual page that has no valid entry in its page table. The CPU traps into the kernel, and the kernel decides what that access meant. Most page faults are normal and expected; they are how the kernel avoids doing work until it must.

flowchart TD
  A["Page fault on address X"] --> B{"Mapped region?"}
  B -- no --> S["SIGSEGV: segmentation fault"]
  B -- yes --> C{"Why missing?"}
  C -- "never touched" --> M1["Minor: hand out a zeroed page"]
  C -- "in the page cache" --> M2["Minor: map the cached page"]
  M1 ~~~ M3
  M2 ~~~ M3
  C -- "write to a COW page" --> M3["Minor: copy the page, make it writable"]
  M3 ~~~ MJ
  C -- "on disk or swap" --> MJ["Major: read it in, sleep meanwhile"]

• Minor fault. The data is already in memory, or does not exist yet. The kernel allocates a zero page, or maps a page cache page, or copies a shared page. Microseconds.
• Major fault. The data has to come from disk: a code page of a program not yet read, a file page that was dropped from the cache, or an anonymous page that went to swap. The process sleeps while the I/O happens. Milliseconds on a hard disk, tens of microseconds on NVMe.
• Segmentation fault. The address is not in any region, or the access violates the region's permissions (writing to text, executing the stack). The kernel sends SIGSEGV.

Demand paging

When a program starts, the kernel does not read the whole binary in. It maps the file and lets page faults pull in the pages that actually run. When malloc asks for 100 MB, the kernel adds 100 MB to the address space and hands out nothing; the pages appear one by one as the program writes to them. This is why a process's virtual size and its resident size differ so much, and why malloc almost never fails even on a machine that could not honour every allocation.

Copy-on-write

After a fork, parent and child point at the same physical pages, and the kernel marks every writable page read-only in both page tables. When either side writes, the CPU faults, the kernel copies just that one page, gives the writer the copy, and makes it writable. Pages nobody writes are never copied. The same mechanism serves MAP_PRIVATE file mappings and the zero page: every untouched page of a fresh allocation maps to one shared page of zeros until it is written.

Swap

When RAM is short, the kernel reclaims pages. File-backed pages that are clean (unchanged since they were read) are simply dropped; they can be read again from the file. Dirty file pages are written back first. Anonymous pages, meaning heap and stack, have no file behind them, so the only way to reclaim them is to write them to the swap area on disk. Touching a swapped page later is a major fault.

vm.swappiness (0 to 200, default 60) tunes how readily the kernel swaps anonymous pages rather than dropping file pages. A machine with no swap at all cannot reclaim anonymous memory, so under pressure it goes straight from "dropping the page cache" to the OOM killer. Some swap, even a small amount, gives the kernel room to move rarely used pages out.

Page fault

A trap raised when a virtual page has no valid mapping. Minor if no disk I/O is needed, major if it is.

Demand paging

Mapping memory lazily, one page at a time, when first touched.

Copy-on-write

Sharing a page read-only and copying it only when someone writes.

Anonymous memory

Pages with no file behind them: heap, stack, private allocations. Only swap can hold them out of RAM.

Allocation and measurement: what a process really uses

malloc is a user space allocator that gets big regions from the kernel and hands out small pieces. Because the kernel is lazy about backing pages, the numbers that describe memory use are less obvious than they look.

How malloc gets memory

glibc's allocator uses two system calls. For small requests it grows the heap with brk and carves pieces out of it. For large ones, 128 KB and up by default, it calls mmap for a private anonymous region and returns the whole thing to the kernel on free. Small freed pieces are kept for reuse and rarely given back, because the heap can only shrink from the top. A program whose heap grew to 2 GB and then freed most of it can still show 2 GB resident. Alternative allocators like jemalloc and tcmalloc are better at returning memory and at avoiding lock contention between threads.

Overcommit

Because pages are only backed when touched, the kernel usually promises more memory than it has. This is overcommit, controlled by vm.overcommit_memory: 0 refuses only obviously absurd requests (the default), 1 never refuses, 2 keeps total commitments under a fixed budget of swap plus a share of RAM. Overcommit is what lets a 20 GB process fork on a machine with 24 GB, since the child will touch very little. It is also why running out of memory is discovered at page fault time, in the middle of some unrelated write, and not at malloc.

The numbers

At the machine level, free -h is the summary, and its columns confuse everyone once. used is application memory. buff/cache is the page cache and kernel buffers, which the kernel will give up the moment an application needs it. free is memory nobody has touched, and on a healthy busy server it is small, because unused RAM is wasted RAM. The column that matters is available: an estimate of how much a new program could take without swapping. /proc/meminfo is where all of it comes from.

brk

The system call that moves the end of the heap. glibc uses it for small allocations.

Overcommit

The kernel granting more virtual memory than physical memory plus swap, betting most of it is never touched.

Page cache

File contents kept in RAM by the kernel after a read or before a write. Shown as buff/cache.

The OOM killer: what happens when memory runs out

When the kernel needs a page and has nothing left to reclaim, no cache to drop and no swap space to write to, it kills a process to get memory back. That is the out-of-memory killer. It is a last resort, and it is loud.

flowchart TD
  A["Page fault needs a free page"] --> B["Reclaim: drop clean cache, write dirty pages, swap out"]
  B --> C{"Got one?"}
  C -- yes --> D["Carry on"]
  C -- no --> E["Score every process: memory used, adjusted by oom_score_adj"]
  E --> F["SIGKILL the highest score"]
  F --> G["Log to dmesg: Out of memory: Killed process ..."]

The score is mostly the process's RSS plus swap use, so the biggest process is the usual victim. /proc/PID/oom_score_adj shifts it, from -1000 (never kill this) to 1000 (kill this first). systemd sets it for services with OOMScoreAdjust=, and sshd sets its own to -1000 so you can still log in. The kill is SIGKILL, so the process gets no chance to clean up, and the only evidence is a block in dmesg listing every process's score and the one chosen.

With cgroups, the same thing happens inside one container when it hits its memory.max, even if the host has plenty free. The kernel kills a process inside the cgroup. Docker reports it as exit code 137 (128 plus signal 9) and Kubernetes as OOMKilled. This is the most common way a container dies in production, and the fix is usually a limit that matches what the program really uses, measured with RSS over time rather than guessed.

OOM killer

The kernel routine that kills a process when there is no memory left to reclaim.

oom_score_adj

A per-process adjustment from -1000 to 1000 that makes it less or more likely to be chosen.

Exit code 137

128 plus 9: the process was killed by SIGKILL, in a container almost always by the OOM killer.

Signals: asynchronous messages to a process

A signal is a small integer delivered to a process to tell it something happened: the user pressed Ctrl-C, a child exited, it touched bad memory, someone asked it to stop. It carries no data beyond the number. It interrupts whatever the process was doing.

Each signal has a default action: terminate, terminate with a core dump, ignore, stop, or continue. A process can change that for most signals by installing a handler, a function the kernel will call, or by setting the signal to be ignored. SIGKILL and SIGSTOP are the exceptions: the kernel handles them and the process gets no say, which is what makes kill -9 reliable.

How a signal is delivered

sequenceDiagram
  autonumber
  participant S as Sender (kill, terminal, kernel)
  participant K as Kernel
  participant T as Target process
  S->>K: kill(pid, SIGTERM)
  K->>K: permission check, set the pending bit
  K->>T: wake it if sleeping in S
  Note over T: next return to user mode
  K->>T: run the handler on the user stack
  T->>T: handler sets a flag, returns
  K->>T: resume where it was interrupted

1. Something calls kill, or the kernel raises the signal itself (a fault, a child exit, a terminal key).
2. The kernel checks permission (same user, or root) and marks the signal pending on the target. If the target is asleep in S, it is woken; a process in D is not.
3. The next time the target returns to user mode, from a system call or a context switch, the kernel notices the pending signal.
4. If the action is default, the kernel does it: terminates, stops, or ignores. If there is a handler, the kernel sets up a frame on the process's stack and jumps to the handler.
5. The handler runs in the process, as ordinary user code. When it returns, the kernel restores the interrupted state and the process carries on, or, if it was in a slow system call, that call returns EINTR unless the handler was installed with SA_RESTART.

Standard signals are not queued. If the same signal is sent five times before the process gets to handle it, the handler runs once. Real-time signals (SIGRTMIN and up) do queue, and can carry a value.

Writing a handler

The handler can run between any two instructions of the program, including in the middle of malloc or printf. Calling those from a handler can deadlock or corrupt the heap. Only async-signal-safe functions are allowed: write, _exit, signal, and a fixed list of others. The safe pattern is to set a flag and return.

#include <signal.h>
#include <unistd.h>

static volatile sig_atomic_t stop = 0;

static void on_term(int sig) {
    stop = 1;                     // nothing else: no printf, no malloc
}

int main(void) {
    struct sigaction sa = {0};
    sa.sa_handler = on_term;
    sa.sa_flags = SA_RESTART;     // restart interrupted reads instead of EINTR
    sigaction(SIGTERM, &sa, NULL);
    sigaction(SIGINT, &sa, NULL);

    while (!stop) {
        // do work, check the flag between units
    }
    // clean up: flush, close, remove the pid file
    return 0;
}

Servers that would rather not be interrupted at all use signalfd, which turns signals into bytes readable from a file descriptor, so they can be handled in the normal event loop next to sockets. In a shell script, trap 'cleanup' TERM INT does the same job.

When a process is killed by a signal, the shell reports it as exit code 128 plus the signal number: 130 for Ctrl-C, 137 for SIGKILL, 143 for SIGTERM. That is how you tell "exited with an error" from "was killed".

Signal

A numbered asynchronous notification delivered to a process by the kernel.

Handler

A function the process registers with sigaction to run when a signal arrives.

Pending

Sent but not yet delivered, because the process has not returned to user mode or has the signal blocked.

Async-signal-safe

A function that is safe to call from a handler because it cannot be mid-way through in the interrupted code.

Recap