malloc(100 MiB) Succeeded, So Why Is RSS Barely Higher?

void *ptr = malloc(100 * 1024 * 1024);

When that call returns on Linux, you hold a usable, suitably aligned address range of at least 100 MiB. That is all it guarantees. The kernel has not necessarily put 100 MiB of physical memory into the process’s resident set — and the allocator may not even have asked it to.

I keep seeing the two events collapsed into one: malloc() succeeded, therefore the process is using 100 MiB of memory. The allocator’s answer and the kernel’s answer are different questions, and the gap shows up as confusing RSS numbers, OOM kills on over-committed boxes, and startup latency spikes.

malloc() is the front door of a user-space allocator. The allocator can satisfy a request by reusing free blocks the process already owns, or it can grow the address space through brk() or mmap(). For a fresh large anonymous mapping, the kernel usually records a virtual memory region first and leaves page tables and physical pages to be filled in when the program actually touches the memory.

This article traces that path on x86-64 with Linux 6.10 and the default glibc allocator. Allocator settings, page size, Transparent Huge Pages (THP), NUMA policy, and overcommit configuration all change the details; treat the numbers here as a concrete starting point, not a universal answer.

The short version

• malloc() first tries to reuse free blocks held by the user-space allocator; only when none fits does it ask the kernel for more address space.
• glibc’s mmap threshold starts near 128 KiB but is dynamically adjusted by default. It is not a fixed boundary, and it can be overridden with tunables or mallopt().
• A fresh 100 MiB private anonymous mapping usually just creates or merges a VMA and runs address-space and commit checks — user data pages are not populated up front.
• Read faults on an anonymous mapping can map the shared zero page; write faults allocate a private zeroed anonymous page. “The first read and the first write both allocate a private page” is wrong.
• Prefaulting moves page-fault cost to startup; it does not stop the kernel from reclaiming or swapping those pages later. For strict tail latency, combine prefaulting with NUMA placement, mlock(), limits, and measurements.

An experiment: watch VSZ and RSS separately

The demo below splits allocation and per-page writes into two stages. The buffer pointer is volatile so the optimizer cannot delete a write loop with no later use:

/* test_malloc.c */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(void) {
    const size_t size = 100UL * 1024 * 1024;
    long page_size = sysconf(_SC_PAGESIZE);

    if (page_size <= 0) {
        perror("sysconf");
        return 1;
    }

    printf("PID: %ld\n", (long)getpid());
    printf("1. press Enter to call malloc(100 MiB)...\n");
    getchar();

    void *ptr = malloc(size);
    if (ptr == NULL) {
        perror("malloc");
        return 1;
    }

    printf("2. malloc returned %p, press Enter to write each page...\n", ptr);
    getchar();

    volatile unsigned char *buf = ptr;
    for (size_t offset = 0; offset < size; offset += (size_t)page_size)
        buf[offset] = 1;

    printf("3. writes done, press Enter to free and exit...\n");
    getchar();

    free(ptr);
    return 0;
}

Compile and run:

gcc -O2 -Wall -Wextra test_malloc.c -o test_malloc
./test_malloc

From another terminal, watch the memory counters:

ps -o pid,vsz,rss,cmd -p <PID>
grep -E '^(VmSize|VmRSS|RssAnon):' /proc/<PID>/status

Use strace to confirm which path the request took:

strace -e trace=brk,mmap,munmap ./test_malloc

The typical trend (values vary with glibc, link mode, and system configuration):

So the accurate statement is: for a fresh large anonymous mapping, RSS usually does not jump by ~100 MiB the moment malloc() returns. It is just as wrong to claim the kernel allocated “not a single byte” — the VMA, page-table pages, and allocator metadata cost memory too, and malloc() may have reused an existing block.

When does malloc() use brk(), and when mmap()?

malloc() is not a system call. In glibc, it first looks through tcache, bins, or arena for a reusable chunk. On a hit, the whole allocation stays in user space and VSZ/RSS may not change at all.

Only when existing space is insufficient does the allocator ask the kernel for more address space. The common paths:

• extend the main heap, which ultimately changes the program break via brk();
• extend or create an arena mapping;
• for larger standalone chunks, use a private anonymous mmap().

MMAP_THRESHOLD is not a fixed 128 KiB

glibc’s initial M_MMAP_THRESHOLD is typically 128 KiB, but by default the threshold adapts to allocation and free patterns. It becomes a fixed value only after you set glibc.malloc.mmap_threshold, MALLOC_MMAP_THRESHOLD_, or call mallopt(M_MMAP_THRESHOLD, ...).

The threshold is not the only deciding factor either:

• a request at or above the threshold can still be served from a free chunk when one is large enough;
• requests below the threshold can still go through mmap() in some cases;
• replacing the allocator (jemalloc, tcmalloc, …) changes the path completely.

For a 100 MiB request in a fresh process you can safely say “typically mmap() under common glibc defaults”, but not “always”. Confirm with strace or the allocator’s own statistics.

One detail worth separating: sbrk() is a libc interface; the Linux kernel exposes the brk system call. They are not two independent kernel allocation mechanisms.

How an anonymous mapping becomes physical pages

mmap() manages an address range first

In Linux 6.10, do_mmap() validates arguments, permissions, address space, and mapping limits, then enters mmap_region(). For private writable mappings that count against commit, mmap_region() also performs the overcommit check.

The kernel then merges the new range with an adjacent compatible VMA or allocates a new vm_area_struct. Linux 6.10 manages a process’s VMAs with a Maple Tree rather than the red-black tree of older kernels.

Without MAP_POPULATE, MAP_LOCKED, or similar pre-populate flags, this step does not build user data pages for the whole mapping. What has happened so far:

• a legal virtual address range was chosen;
• access permissions and mapping attributes were recorded;
• address-space and commit accounting were updated;
• a VMA was created or merged, and its kernel metadata maintained.

That work costs a little kernel memory; it is not the same as allocating 100 MiB of resident anonymous pages.

The x86-64 page-fault path

The first access to an address with no valid mapping makes the MMU raise #PF — a CPU exception, not a device interrupt. In Linux 6.10 on x86-64 the main path looks like this:

The kernel first finds the VMA covering the address and checks whether the read or write is allowed. Addresses with no valid VMA, or accesses that violate permissions, end in SIGSEGV; only valid faults continue to page-in.

Read faults and write faults differ

In do_anonymous_page(), Linux 6.10 treats read and write faults differently:

• Read fault: when the shared zero page is permitted, the kernel can point the PTE at the read-only global zero page without allocating a private anonymous page for the process.
• Write fault: the kernel allocates a zeroed anonymous folio for the process, installs a writable PTE, and counts the pages into anonymous RSS.

This is kernel mapping behavior; it does not make it safe to read uninitialized malloc() output in C. malloc() does not promise zeroed memory — use calloc(), or write explicitly, when you need zeros.

Anonymous pages must be zeroed before being mapped to user space, or they would leak the contents of whatever process last used that physical page. After the fault handler finishes, the CPU returns to user mode and re-executes the faulting instruction.

Not necessarily 25,600 faults

With 4 KiB base pages, THP disabled, and an allocator that has not already touched the range, writing 100 MiB one page at a time triggers about 25,600 page faults.

The real count is usually different:

• the base page size may not be 4 KiB;
• THP or multi-size THP may map a larger folio at once;
• glibc may already have touched a few pages in the mapping;
• alignment, NUMA policy, and kernel configuration change allocation behavior.

The first write to an anonymous page is normally a minor fault because no file data has to be read from disk. “Minor” means no disk I/O; it does not mean the handling is free.

Overcommit decides what can be promised; RSS shows what is resident

Linux keeps virtual mappings, committed memory, and currently resident pages separate. /proc/sys/vm/overcommit_memory controls the commit policy, not “whether virtual memory is used”:

In mode 2, CommitLimit is roughly swap plus a fraction of physical memory, configurable through vm.overcommit_ratio or absolute via vm.overcommit_kbytes. Read the actual limit with:

grep -E '^(CommitLimit|Committed_AS):' /proc/meminfo

Committed_AS is the total memory the kernel has promised it may have to provide later; it is not the current RSS. Private writable anonymous mappings normally count against commit even when most of their pages are not resident.

Even in mode 2, a successful malloc() is no guarantee against OOM everywhere: containers or systemd units may be limited by memory cgroups, and a process can still hit page-table, kernel-memory, NUMA-node, or other resource pressure.

OOM is not necessarily the whole machine

Under pressure, the kernel first tries reclaim: write back dirty pages, swap, compact. If that does not free enough, it may trigger a global OOM or only the current memory cgroup’s OOM.

When the OOM killer picks a victim it weighs memory usage together with oom_score_adj and other factors — it does not simply kill the process with the largest RSS. Useful diagnostics:

journalctl -k -g 'Out of memory|Killed process|oom-kill'
dmesg -T | grep -Ei 'out of memory|killed process|oom-kill'

cat /proc/<PID>/oom_score
cat /proc/<PID>/oom_score_adj
cat /sys/fs/cgroup/memory.events

The last command reads cgroup v2 memory events and must be run in the cgroup the process belongs to. Application code still has to check whether malloc() returned NULL; overcommit is not a substitute for error handling and memory limits.

Engineering takeaways

Prefaulting only moves the cost

Low-latency services can walk each page and write during pool initialization, moving most first-write faults to startup. Do not substitute “read each page once” — a read fault may map the shared zero page and never establish a private writable page.

static int prefault_write(void *addr, size_t len) {
    long page_size = sysconf(_SC_PAGESIZE);
    if (page_size <= 0)
        return -1;

    volatile unsigned char *p = addr;
    for (size_t offset = 0; offset < len; offset += (size_t)page_size)
        p[offset] = 0;

    if (len != 0)
        p[len - 1] = 0;
    return 0;
}

When the application manages memory directly with mmap(), evaluate MAP_POPULATE or madvise(..., MADV_POPULATE_WRITE) depending on kernel version and error-handling needs. MADV_POPULATE_WRITE (Linux 5.14+) prefills page tables for an existing writable mapping and reports failure as an error.

After prefaulting, pages can still be reclaimed or swapped. For stronger residency guarantees, consider mlock() or mlockall(), but weigh RLIMIT_MEMLOCK, privileges, and the effect of locking a lot of memory on the whole system’s reclaim capacity.

On NUMA machines, remember first-touch: the physical page is usually placed on the node of the CPU that first writes it. One startup thread prefaulting everything onto a single node can leave worker threads doing remote accesses for a long time. Better: bind threads, then have the threads that will actually use the pages initialize them in parallel, and validate with real latency and NUMA metrics.

VSZ, RSS, and PSS answer different questions

Common checks:

grep -E '^(VmSize|VmRSS|RssAnon|RssFile|VmSwap):' /proc/<PID>/status
grep -E '^(Rss|Pss|Anonymous|Swap):' /proc/<PID>/smaps_rollup

For leak diagnosis, combine object counts, allocator statistics, RSS/PSS, swap, and memory-cgroup data. A growing VSZ may just be address-space reservation; a flat RSS may be the allocator retaining free blocks for reuse.

RSS does not drop immediately after free()

Large standalone mmap() chunks usually return to the kernel via munmap() when freed. Small chunks inside an arena may go back to glibc’s free lists and stay available for later allocations.

malloc_trim(0) asks glibc to try to return free pages. Since glibc 2.8 it scans all arenas for chunks containing whole free pages, but it is a best-effort attempt: a return value of 0 means nothing was released, and RSS is not guaranteed to drop to any specific value.

MADV_DONTNEED is stronger: for private anonymous mappings it discards the range’s contents, and subsequent accesses see demand-zeroed pages. It requires a page-aligned start address and must not be used on certain special or locked mappings.

Do not call MADV_DONTNEED on arbitrary live blocks still owned by malloc(): the aligned range may cover allocator metadata or other objects on the same page, and discarding changes program semantics. For precise recycling of large reusable buffers, manage whole page ranges with mmap() yourself and use madvise() over explicit lifecycles.

Conclusion

In a typical Linux 6.10 + glibc environment, a fresh malloc(100 MiB) large request follows roughly these steps:

1. glibc checks existing free blocks; when none fits, a 100 MiB request usually gets its address space through a private anonymous mmap().
2. The kernel validates address, permissions, and overcommit, then creates or merges a VMA; unless prefilled explicitly, user data pages are not established yet.
3. The first write to an unmapped page raises a fault; the kernel allocates a zeroed anonymous page and updates page tables. Read faults may use the shared zero page instead.
4. RSS tracks currently resident pages, Committed_AS tracks the commit promise, and VSZ tracks the virtual mapping. The three numbers are not interchangeable.

So malloc() succeeding and physical memory being resident are two different events, and physical residency and “OOM will never happen” are two more. When diagnosing memory, check the allocator path, fault behavior, commit policy, RSS/PSS, memory cgroup, and NUMA placement — not just one column in top.

Related reading: mmap() vs read(): When Zero-Copy Is Actually Slower covers the page-fault and page-cache trade-offs on the file side, and What Actually Happens During a Linux System Call on x86-64? covers the boundary where brk(), mmap(), and madvise() enter the kernel.

References

1. GNU C Library, Malloc Tunable Parameters
2. GNU C Library, The GNU Allocator
3. Linux 6.10, mm/mmap.c
4. Linux 6.10, mm/memory.c
5. Linux 6.10, x86 page-fault exception handling
6. Linux Kernel Documentation, Overcommit Accounting
7. Linux Kernel Documentation, Transparent Hugepage Support
8. Linux man-pages, madvise(2)
9. Linux man-pages, malloc_trim(3)
10. Linux man-pages, proc_pid_smaps(5)