Transport

What you get, and what you don't

UDP header — 8 bytes total
  ┌──────────────┬──────────────┬──────────────┬──────────────┐
  │  source port │   dest port  │    length    │   checksum   │
  └──────────────┴──────────────┴──────────────┴──────────────┘

Provides:  ports (which process) · a length · an optional checksum
             message BOUNDARIES — one send = one datagram = one receive

Does NOT provide:
  · connection setup      → no handshake, so zero round trips of overhead
  · reliability           → lost datagrams are simply gone
  · ordering              → datagram 2 may arrive before datagram 1
  · flow control          → you can overwhelm a slow receiver
  · congestion control    → you can overwhelm the network
                            (which is why unresponsive UDP flooding is
                             antisocial and gets rate-limited by ISPs)

Who chooses UDP, and why

TIME_WAIT — the state everyone asks about

The side that closes first waits 2×MSL (Maximum Segment Lifetime, so typically ~60 s on Linux) before fully releasing the socket. Two reasons, and knowing both is the good answer:

1. The final ACK might be lost. If it is, the peer retransmits its FIN and needs someone still there to re-acknowledge it. Releasing immediately would send an RST and leave the peer thinking the close failed.
2. Old duplicate packets must expire. A delayed segment from this connection could otherwise arrive during a new connection using the same four-tuple and be accepted as valid data. Waiting two maximum segment lifetimes guarantees any straggler is gone.

Why it matters operationally: a busy proxy or load balancer that initiates closures accumulates tens of thousands of TIME_WAIT sockets, exhausting ephemeral ports. The right fixes are connection reuse (keep-alive) and tcp_tw_reuse; the wrong fix is SO_LINGER with zero timeout, which sends an RST and can discard data in flight.

Also worth distinguishing: FIN is a graceful close that flushes pending data; RST is an abort that discards it. You get an RST when connecting to a closed port, or from a peer that has already torn the connection down — which is what "connection reset by peer" means.

The mechanisms, in dependency order

• Sequence numbers count bytes, not packets, and start at a random value (predictable ISNs were a real hijacking vulnerability). They give ordering, duplicate detection, and a way to identify exactly what's missing.
• Cumulative ACKs. "ACK 5000" means "I have everything up to byte 4999". It's cumulative, so a lost ACK is harmless — the next one covers it.
• Sliding window. Send up to W unacknowledged bytes before waiting. This is what makes TCP fast: without it you'd send one segment per round trip.
• Retransmission timeout (RTO), derived from a smoothed RTT estimate plus a variance term, and doubled on each successive failure (Karn's algorithm ignores RTT samples from retransmitted segments, since you can't tell which copy was acknowledged).
• Fast retransmit. Waiting for a timeout is slow. Three duplicate ACKs for the same sequence number strongly imply one segment was lost while later ones arrived — so retransmit immediately rather than waiting.

The TCP header fields worth knowing

source port · dest port          which processes
sequence number                  byte offset of this segment's first byte
acknowledgement number           next byte expected FROM the peer
flags   SYN  open        ACK  acknowledging        FIN  graceful close
        RST  abort       PSH  deliver now          URG  urgent (unused)
window size                      receiver's free buffer — FLOW CONTROL
checksum                         covers header AND payload (unlike IP's)
options                          MSS, window scale, SACK permitted, timestamps

Only 16 bits for the window, i.e. a 64 KB maximum — which was a hard
throughput ceiling on fast long paths until the WINDOW SCALE option
(negotiated in the handshake) allowed multiplying it up to ~1 GB.
See topic 16 for why that mattered so much.

Flow control

The receiver advertises a window size in every ACK: "I have this much buffer space left." The sender never has more than that in flight. If the application stops reading, the window shrinks to zero, the sender stops, and it periodically sends a window probe until the receiver advertises space again.

Two named problems: silly window syndrome, where the receiver advertises tiny windows and the sender ships tiny segments with 40 bytes of header each — fixed by having the receiver wait until a decent chunk is free (Clark's solution) and the sender wait for a full segment (Nagle's). And zero-window deadlock, avoided by the persist timer.

The four phases

AIMD — additive increase, multiplicative decrease — is the core. It's provably stable and fair: flows converge on an equal share regardless of when they started, because the multiplicative decrease penalises large flows more in absolute terms than small ones.

Modern algorithms

Example: 1 Gbps link, 100 ms RTT (London → Singapore)
  BDP = 1,000,000,000 bits/s × 0.1 s = 100,000,000 bits = 12.5 MB

  You need a 12.5 MB window to use that link fully.
  TCP's base window field is 16 bits → 64 KB maximum.

  Throughput with a 64 KB window = 65,536 B / 0.1 s
                                 = 655 KB/s ≈ 5 Mbps
  …on a 1 Gbps link. You are using 0.5% of it.

Fix: the WINDOW SCALE option, negotiated in the handshake, shifts the
window left by up to 14 bits → windows up to ~1 GB.
This is why "long fat networks" needed a protocol extension, and why
window scaling being stripped by a middlebox produces mysteriously
slow-but-working transfers.

Also: a single TCP flow is limited by loss. Parallel connections, or a
protocol like QUIC, work around a single flow's ceiling — which is
partly why download managers and CDNs open several connections.

Latency contributors, and what fixes each

The reason connection reuse matters so much: a cold HTTPS request costs a DNS lookup, a TCP handshake and a TLS handshake — three or four round trips before the request is even sent. At 150 ms RTT that's half a second of nothing. On a warm connection it's one round trip. This is the entire argument for keep-alive, HTTP/2 multiplexing and connection pooling.

Two more worth knowing: TCP keep-alive (default 2 hours on Linux, far too long for NAT tables that expire in minutes — which is why applications implement their own heartbeats) and the fact that ephemeral port exhaustion caps a single client at ~28,000 concurrent connections to one destination, since the four-tuple must be unique.

1. TCP head-of-line blocking. HTTP/2 multiplexes many streams over one TCP connection — but TCP delivers a single ordered byte stream, so one lost packet stalls every stream until it's retransmitted. HTTP/2 solved application-layer blocking and left transport-layer blocking untouched.
2. Handshake cost. TCP's 1 RTT plus TLS's 1–2 RTT means 2–3 round trips before any data. QUIC merges transport and cryptographic handshakes into one, and offers 0-RTT for a resumed connection.
3. Ossification. TCP lives in the kernel and is inspected by middleboxes that reject anything unfamiliar, so changes take a decade to deploy. QUIC runs in user space over UDP and is fully encrypted, so it can be updated with a browser release.
4. Connection death on network change. A TCP connection is identified by its four-tuple, so switching from Wi-Fi to mobile changes your IP and kills every connection. QUIC uses a connection ID independent of the addresses, so it survives the switch.