sendfile() Promises Zero-Copy. Which Copies Does It Actually Remove?

When a file server sends a file over the network, the most obvious implementation loops over read() and write(): read the data from the file into a user-space buffer, then write it to the socket. sendfile() offers a different deal — let the kernel transfer data directly between two file descriptors.

“Zero-copy” here does not mean the data never moves. It means that, on the ideal path, file contents are no longer copied back and forth by the CPU just to pass through user space.

The short version

• On the typical read()+write() file-sending path, the CPU performs two payload copies: page cache to user buffer, then user buffer to the kernel send path.
• sendfile() can eliminate both of those user-space-related CPU copies. DMA still moves bytes when the storage device delivers data to memory and when the NIC reads the data to transmit.
• One read() plus one write() usually crosses the user/kernel boundary four times; one sendfile() usually crosses it twice. Those are boundary crossings, not four versus two “thread context switches”.
• “Zero-copy” describes an achievable fast path, not a fixed guarantee for every combination of file system, protocol stack, and NIC. Unsupported paths, data that must be rewritten, or device constraints can still force copies.

Where do the two CPU copies in read()+write() happen?

A simplified file-sending loop looks like this:

char buf[64 * 1024];

for (;;) {
    ssize_t n = read(file_fd, buf, sizeof(buf));
    if (n <= 0)
        break;

    ssize_t off = 0;
    while (off < n) {
        ssize_t sent = write(socket_fd, buf + off, n - off);
        if (sent > 0) {
            off += sent;
            continue;
        }
        if (sent < 0 && errno == EINTR)
            continue;

        /* wait for writability or abort; this unsent data cannot be skipped */
        handle_write_result(sent, errno);
    }
}

Assume the file data is not yet in the page cache. This path decomposes into four data movements:

1. The storage device DMA-reads file data into the page cache.
2. read() has the CPU copy data from the page cache into the user buffer.
3. write() has the CPU copy data from the user buffer into the kernel’s socket send path.
4. The NIC DMA-reads the outgoing data and puts it on the wire.

Steps 2 and 3 are the CPU payload copies that sendfile() primarily wants to eliminate. Steps 1 and 4 are device-to-memory transfers; calling a “zero-copy” interface does not make them disappear.

If the file is already in the page cache, step 1 does not happen for this request. So “the traditional path always has four copies” is a cold-cache conceptual model, not an event count that necessarily occurs on every call.

Entering the kernel is not a thread context switch

A successful read() enters the kernel from user mode and returns; write() repeats the cycle. One round therefore usually has four user/kernel boundary crossings.

Entering the kernel for a system call requires saving the necessary execution state and switching privilege level, but that does not mean the scheduler switched threads. A scheduling-level context switch only happens when the current thread blocks, exhausts its time slice, or is preempted. Conflating the two concepts inflates the perceived cost of system calls — What Actually Happens During a Linux System Call on x86-64? breaks down what that crossing really costs.

What does sendfile() actually eliminate?

The Linux call form is:

off_t offset = 0;
ssize_t n = sendfile(socket_fd, file_fd, &offset, count);

The application only tells the kernel the input file, the output descriptor, the offset, and the length — it no longer prepares a user-space data buffer. On the ideal path, the kernel lets the output side reference the file pages already in the page cache, and the NIC reads the data out of those pages. The CPU still handles the system call, protocol headers, page references, and socket metadata, but it no longer performs these two byte-by-byte copies:

• page cache → user buffer;
• user buffer → kernel send buffer.

So the more precise meaning of “zero-copy” is: the file payload never passes through the user address space, and in the ideal case there is no CPU-performed payload copy. It is not “no DMA”, and it is not “the CPU does nothing at all”.

In the current Linux implementation, sendfile() from a regular file to a socket enters do_sendfile() and then uses do_splice_direct() to move data between the input and output sides. The latter uses a per-process internal pipe as the connecting mechanism: pipe buffers hold references to pages, so the data never has to be copied into user space first. This implementation detail also shows that modern sendfile() cannot be simplified to “hand a NIC descriptor from the disk controller to the NIC”.

Why mmap()+write() is not equivalent

mmap() can map file pages into the process address space, avoiding the read() copy from page cache to user buffer. But a subsequent plain write() usually still requires the kernel to read the data from that user address space and place it into the socket send path.

It also introduces page-fault handling, mapping management, and unmapping costs. mmap() is a good fit when the application needs random access or must process file contents directly; if the application only ships the file unchanged to a socket, sendfile()’s interface and data path are usually more direct. mmap() vs read(): When Zero-Copy Is Actually Slower covers when the mapping itself becomes the bottleneck.

Why is the runtime path not always “zero-copy”?

sendfile() provides an opportunity to bypass user-space copies. The final path still depends on whether every layer supports page references and scatter-gather I/O.

The following situations can add copies or push the application toward other interfaces:

• The file system or the output side does not support the required transfer path.
• The protocol stack, a filter, or the device needs to linearize the data.
• The NIC’s scatter-gather capability, fragment limits, or alignment requirements cannot accommodate the current data layout.
• The application must compress, transcode, modify the response body, or compute business data that requires reading the content.
• TLS encryption happens in user space, so plaintext must be handed to the TLS library first.

If the system uses kernel TLS (kTLS) and the kernel, cipher, socket configuration, and send path all meet the requirements, file data can still be encrypted and sent without leaving the kernel. But that is not something HTTPS automatically grants — whether it applies must be verified against the actual TLS stack and performance data.

Cold cache is another easily confused variable. sendfile() removes user-space copies but not disk reads. The first send of a large file may be dominated by the storage device and fault-driven reads; only when repeatedly sending files already in the page cache can you observe the difference in CPU copies and system-call counts in isolation.

Engineering edges when using sendfile()

1. The return value may be smaller than requested

A single call is not guaranteed to transfer all count bytes. Signals, non-blocking sockets, send-buffer space, and other conditions can all cause short writes, and the application must resume based on the return value. A non-blocking output can also return EAGAIN.

while (remaining > 0) {
    ssize_t n = sendfile(out_fd, in_fd, &offset, remaining);
    if (n > 0) {
        remaining -= n;
        continue;
    }
    if (n < 0 && errno == EINTR)
        continue;
    if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
        /* wait until the output side is writable, then retry */
        break;
    }
    /* handle EOF or other errors */
    break;
}

A single Linux sendfile() transfers at most 0x7ffff000 bytes; larger files must be sent in chunks.

2. Input and output descriptors have constraints

The input descriptor must support mmap()-like operations, so a plain socket cannot serve as a regular in_fd. Before Linux 2.6.33, out_fd had to be a socket; since then it can be any writable file. Since Linux 5.12, when the output is a pipe, sendfile() delegates to splice() and inherits its capabilities and limits.

If the output side uses the zero-copy capability, do not modify the corresponding input file region before the send completes — otherwise the receiver may observe the modified content.

3. splice() is not a direct pipe between arbitrary descriptors

splice() can also keep data out of the user address space, but it requires at least one pipe end. Forwarding socket-to-socket usually takes two calls: input socket to pipe, then pipe to output socket.

SPLICE_F_MOVE is only a hint; the Linux man page states explicitly that since Linux 2.6.21 the flag currently has no effect. Do not read it as “when set, pages are definitely moved instead of copied”.

How should you verify the benefit?

Do not compare interface names alone. At minimum, observe all of the following:

1. Compare system-call counts with strace -c.
2. Test cold cache and warm cache separately, so disk-read time is not misattributed to copy overhead.
3. Watch CPU utilization, throughput, and tail latency — not just single-call latency.
4. Use the same file system, TLS configuration, NIC, and response-body sizes as production.
5. Test small files separately. Fixed costs of system calls, protocol processing, and queue management can dominate, and zero-copy may show no visible gain.

sendfile() fits best when file content needs no application processing and goes straight to the output — static file serving is the canonical case. When content must be modified, going back through user space is not a design flaw; choose among read/write, mmap, sendfile, or other interfaces based on how the business logic touches the data.

Conclusion

In the end, sendfile() removes the two CPU copies that exist only to route data through user space. It reduces memory-bandwidth consumption and system-call count, but it does not stop data from moving — and its real-world effect cannot be discussed apart from the specific kernel, protocol stack, and hardware underneath.

Related reading: mmap() vs read(): When Zero-Copy Is Actually Slower examines the other “avoid a copy” tool and when it backfires, and What Actually Happens During a Linux System Call on x86-64? details the boundary crossings that sendfile() halves.

References

1. Linux man-pages, sendfile(2)
2. Linux man-pages, splice(2)
3. Linux 6.10, fs/read_write.c
4. Linux 6.10, fs/splice.c
5. Linux Kernel Documentation, Kernel TLS