Chapter 01 · Part I · The basics

What a network is

A network is a way to move bits from one machine to another, shared by everyone who wants to use it. This chapter covers the few big ideas underneath every other chapter: bits, packets, layers, addresses, and the two very different numbers people mean when they say a connection is "fast."

1,500 bytesthe most data one standard Ethernet frame can carry, its MTU
~5 µs per kmhow long light takes to cross a kilometer of optical fiber
94.9%the best share of a wire's time your data can get with TCP over IPv4 and Ethernet (1,460 of 1,538 bytes)

Common mix-up: a "faster" connection does not move bits faster. A 1 Gb/s link and a 10 Mb/s link carry each bit at the same speed, a large fraction of the speed of light. The gigabit link just packs the bits a hundred times closer together. That's bandwidth. How long one bit takes to arrive is latency, and no amount of money buys less of it over a fixed distance.

The raw material

Bits on a wire

Underneath every network is something that can be put into one of two states and read back at the other end. On copper Ethernet it's a voltage. On fiber it's a pulse of light, or a change in its brightness or phase. On Wi-Fi it's a radio wave nudged in amplitude and phase. Each state change carries a bit, a 0 or a 1, and that is the only thing a network ever really moves. Text, photos, video calls and this page all get turned into bits first.

Eight bits make a byte, enough to count from 0 to 255. The capital letter H is stored as the number 72, which in binary is 01001000. A four-letter word is 32 bits; a phone photo is tens of millions.

A link's speed is quoted in bits per second, and the prefixes are powers of ten: 1 Mb/s is a million bits a second, 1 Gb/s is a billion. Watch the lowercase b. Network speeds are bits; file sizes are usually bytes (capital B). A 1 Gb/s link moves at most 125 megabytes a second before any overhead, which surprises everyone the first time they copy a big file.

The sender and receiver have to agree on everything about the signal: what counts as a 1, how long each bit lasts, how to tell where a byte starts. Those agreements are the physical layer, and every kind of link has its own. The beautiful trick of networking is that everything above that layer doesn't care which one you used.

Go deeper: it is rarely one bit per wiggle

Early 10 Mb/s Ethernet over twisted pair (10BASE-T) used Manchester encoding: every bit is a transition in the middle of its time slot, high-to-low for one value and low-to-high for the other. That guarantees the receiver sees a transition every bit, so its clock never drifts, at the price of needing twice the signaling rate.

Faster links get cleverer. 100BASE-TX maps every 4 bits to a 5-bit code (4B5B) and sends it with a three-level signal called MLT-3. Gigabit Ethernet over copper (1000BASE-T) uses all four wire pairs at once, in both directions at the same time, with five voltage levels per symbol. Each step up packs more bits into each change of the signal, because the cable can only change state so fast before the changes smear together.

None of that leaks upward. A frame looks the same to the layers above whether it arrived on a 10 Mb/s Manchester signal or a 10 Gb/s laser.

Two ways to share a wire

Circuits versus packets

The telephone network solved sharing with circuits. When you dialed, switches along the way reserved a path from your phone to the other one, and that path was yours until you hung up. In the digital phone network each call got a fixed 64 kb/s channel. The upside is predictability: nothing else can crowd your call. The downside is waste. In a typical conversation each direction is silent more than half the time, and the reserved capacity sits idle through every pause.

Computer traffic is even burstier. A web page is a flurry of activity followed by a minute of you reading. Reserving a circuit for that would be like booking a whole train car for a commute that only uses it for a few seconds.

The alternative, worked out independently in the early 1960s by Paul Baran at RAND and Donald Davies at Britain's National Physical Laboratory, was packet switching. Chop each message into small blocks. Put the destination address on every block. Send each one to the next switch, which stores it for a moment, looks at the address, and forwards it on whichever link leads closer. Davies named the blocks packets. Nothing is reserved; every link is shared by whoever has something to send right now, one packet at a time.

That sharing is called statistical multiplexing, and it works because not everyone bursts at once. Silence costs nothing. The price is uncertainty: when too many packets arrive at once they wait in a queue, which adds delay, and when the queue overflows, some get dropped. Most of the clever machinery in later chapters, from TCP's retransmissions to congestion control, exists to live with that price.

Go deeper: from four nodes to the internet

The ARPANET, the US research network that grew into the internet, sent its first message between UCLA and the Stanford Research Institute on October 29, 1969. The plan was to type LOGIN; the system crashed after "LO". By the end of that year it had four nodes.

The internet itself is a network of networks, glued together by the Internet Protocol. In 1983 the ARPANET switched over to TCP/IP. Today the phone network has turned inside out: most voice calls, including mobile ones, are carried as packets over IP. The circuits survive mostly as an idea, emulated on top of packets when someone needs guaranteed capacity.

Small is good

Why we chop messages up

Imagine sending a 4-megabyte photo as one giant block on a 10 Mb/s link. It occupies the wire for 3.2 seconds, and everything else waits. Chop it into 1,500-byte packets and each takes only 1.2 milliseconds. Other conversations slot in between, and a short message never gets stuck behind a long one for more than a moment.

Small packets also contain damage. Noise on a link flips a bit now and then, and every frame carries a checksum so the receiver can spot it. A corrupted frame is thrown away. If the photo were one block, a single flipped bit would mean sending all four megabytes again. With packets, only the 1,500 bytes around that bit are resent.

Packets travel independently, so they can use whatever path is working. If a link fails, routers send the rest of the stream another way. Packets can arrive out of order, or not at all; sequence numbers in the TCP header let the receiver put them back in order and ask for what's missing.

Finally, packets let a network pipeline. A switch can be forwarding packet 1 to the next hop while packet 2 is still arriving. A 4 MB file across five hops arrives in roughly the time of one transmission plus a little per hop, rather than five full transmissions back to back.

The cost is the header. Every packet carries its own addresses and bookkeeping, typically 40 bytes of IP and TCP plus 18 of Ethernet, whether it holds 1,460 bytes of data or a single keystroke. The biggest packet a link will take is its MTU (maximum transmission unit). On Ethernet that is 1,500 bytes of payload, a number fixed in the 1980s that the whole internet still lives with.

packets = ⌈ message ÷ (MTU − headers) ⌉
With TCP over IPv4 on Ethernet, each packet carries up to 1,500 − 20 − 20 = 1,460 bytes of your data. A 1 MiB photo (1,048,576 bytes) becomes 719 packets.
Divide the work

The layer cake

Moving a web page from a server to your screen involves dozens of jobs: turning bits into voltages, deciding which machine on a cable gets a frame, finding a route across the planet, recovering lost data, and understanding what "GET /index.html" means. Networking splits those jobs into layers. Each layer does one kind of work, uses the layer below it as a service, and offers a service to the layer above. A layer only has to agree with its counterpart on the other machine; it doesn't need to know how the layers beneath it get the bits there.

The best-known layering is the OSI reference model, published by the International Organization for Standardization in 1984, with seven layers. Its protocols mostly lost out, but its vocabulary won: engineers still say "a layer 2 switch" or "a layer 7 load balancer."

The internet itself was built on the simpler TCP/IP model, written down in RFC 1122 with four layers. Link moves frames across one physical network. Internet (IP) moves packets from any host to any other, hop by hop. Transport (TCP or UDP) connects a program on one host with a program on another. Application is everything the programs actually say.

OSI layerTCP/IP layerUnit of dataIts job, and examples
7 ApplicationApplicationmessageWhat programs say to each other: HTTP, DNS, SMTP, SSH, RDP. TLS encryption is usually counted here too.
6 Presentation
5 Session
4 TransportTransportsegment / datagramProgram to program, by port number. TCP adds ordering, retries and flow control; UDP just delivers.
3 NetworkInternetpacketHost to host across many networks, by IP address. Routers work here.
2 Data linkLinkframeNeighbor to neighbor on one network, by MAC address: Ethernet, Wi-Fi. Switches work here.
1 PhysicalbitsVoltages, light pulses, radio symbols. Cables, connectors, transceivers.

The payoff is that layers can be swapped independently. Your browser speaks the same HTTP whether the bits beneath it ride Wi-Fi, fiber, a cell tower or a satellite. IPv6 can replace IPv4 without changing Ethernet. A VPN, which you'll meet in Chapter 12, is just a trick that runs a whole network's lower layers inside another network's upper ones.

Go deeper: the layers leak

Real protocols don't sit neatly in boxes. ARP, the protocol that finds a neighbor's hardware address (Chapter 2), lives between layers 2 and 3. TLS sits on top of TCP but below HTTP. A home router does layer 3 routing, layer 4 port translation (NAT, Chapter 7) and a layer 2 switch all in one box.

The cleanest statement of the internet's philosophy is in RFC 1122 and its companion RFC 1123: keep the network in the middle simple and put the intelligence, like reliability, in the hosts at the ends. That "end-to-end" idea is why a router can forward packets without understanding what's in them, and why NAT, which does peek into them, causes so much trouble in Part III.

Envelopes inside envelopes

Encapsulation: data becomes bits

When your program sends data, it travels down the stack, and each layer wraps what it was handed in a header of its own. This is encapsulation.

The transport layer puts a TCP header in front of your data: source and destination port numbers, a sequence number, an acknowledgment number, flags and a window size, 20 bytes at minimum. Data plus TCP header is a segment. (With UDP the header is only 8 bytes: two ports, a length and a checksum, and the result is called a datagram.)

The internet layer puts an IP header in front of the segment, with the source and destination IP addresses, a time-to-live counter that stops packets from looping forever, and a field saying "the thing inside is TCP." IPv4's header is 20 bytes; IPv6's is a fixed 40. Now it's a packet.

The link layer puts an Ethernet header in front, with the destination and source MAC addresses and a type field (0x0800 means "IPv4 inside"), 14 bytes in all, and a 4-byte checksum, the frame check sequence, at the end. Now it's a frame. If the frame is shorter than Ethernet's 64-byte minimum, padding fills it out.

Finally the physical layer sends an 8-byte preamble so the receiver can lock onto the signal, then the frame's bits, then stays quiet for at least 12 bytes' worth of time, the interframe gap, before the next frame. At the far end it all runs in reverse: each layer reads its own header, strips it, and hands the rest up.

Type a message below and watch it get wrapped. The bottom panel is the real first frame, byte for byte: valid IPv4 or IPv6, TCP or UDP checksums, and a correct Ethernet CRC-32.

Instrument 1

Encapsulation builder

Type anything, or pick a size. Switch between TCP and UDP, IPv4 and IPv6, and see how many packets it takes, how much is header, and what the first frame looks like on the wire.

Your data–
Packets needed–
Headers + trailers–
Total sent–
Overhead–

Who, where, which

An address at every layer

Notice that three of those headers carry addresses, and they answer different questions.

The MAC address in the Ethernet header (48 bits, written like 00:00:5e:00:53:01) answers "which network card on this cable or Wi-Fi network?" It only means something on one local network. When a router passes your packet on, it throws away the old Ethernet header and writes a new one for the next link. Your MAC address never leaves your LAN.

The IP address in the IP header (32 bits for IPv4, like 192.0.2.10; 128 for IPv6) answers "which host, anywhere?" In the original design it rides unchanged from end to end, and every router uses it to choose the next hop. (NAT, the subject of Part III, breaks that rule on purpose, and much of this site is about the consequences.)

The port number in the TCP or UDP header (16 bits, 0 to 65,535) answers "which program on that host?" Web servers listen on 443, SSH on 22, Remote Desktop on 3389. Your side of a connection gets a temporary high-numbered port picked by the operating system.

AddressLayerSizeScopeChanges along the way?
MACLink48 bitsOne local networkYes, rewritten at every router
IPInternet32 or 128 bitsWhole internetNo, unless a NAT rewrites it
PortTransport16 bitsOne hostNo, unless a NAT rewrites it

A connection is identified by five things together: protocol, source IP, source port, destination IP, destination port. That 5-tuple is how your laptop keeps forty browser connections to the same server apart, and it's the key in the translation table every NAT router keeps.

Names like example.com aren't addresses at all. They're labels the DNS turns into IP addresses before any packet is sent (Chapter 4).

Go deeper: why two addresses, MAC and IP?

Couldn't one address do both jobs? A MAC address is burned in at the factory (or chosen at random by the OS) and says nothing about where the device is. Routers need addresses that are grouped by location, so that one routing entry like "everything starting 198.51.100 goes that way" covers thousands of hosts. IP addresses are assigned by the network you join, precisely so they can be grouped that way. When you carry a laptop from home to a café, its MAC stays the same and its IP changes.

The glue between them is ARP for IPv4 and Neighbor Discovery for IPv6: "I need to send to IP X on this network; what MAC address is that?" That's the second half of Chapter 2.

What "fast" means

Latency, bandwidth and throughput

Three numbers get called "speed," and mixing them up causes most networking frustration.

Bandwidth (or link rate) is how many bits per second a link can accept. It decides the serialization delay: how long it takes to push a packet's bits onto the wire, one after another. A 1,500-byte packet is 12,000 bits. At 10 Mb/s that takes 1.2 ms; at 10 Gb/s, 1.2 µs.

Propagation delay is how long each bit takes to travel the distance. Signals in fiber and copper move at roughly two-thirds the speed of light in vacuum, about 200,000 km a second, or 5 µs per kilometer. New York to London is about 5,600 km in a straight line, so 28 ms one way at the very least, and real cables are longer. No upgrade changes it.

Latency is the total delay for a packet: serialization at every hop, plus propagation, plus time waiting in queues, plus processing. People usually measure it as a round trip, the time ping reports.

Throughput is what you actually get, end to end, in useful bits per second. It can never exceed the slowest link on the path, the bottleneck, and it is often much lower, because protocols wait for replies. A sender that has to stop after each packet and wait for an acknowledgment is limited by the round trip, not the link: one 1,500-byte packet per 70 ms round trip is only 171 kb/s, even on a 10 Gb/s cable.

delay = bits ÷ rate + distance ÷ speed
Serialization plus propagation, for one hop with no queueing. Small packets over long distances are ruled by the second term; big files over short links by the first.

That's why a remote desktop session can feel sluggish on a gigabit connection: every keystroke has to make a round trip before you see its echo, and if the path goes through a relay server far away, as the double-NAT story in Part III does, that round trip grows even though the bandwidth looks plentiful.

Go deeper: the bandwidth-delay product

Multiply a link's rate by its round-trip time and you get the number of bits that are "in flight" when the pipe is full: the bandwidth-delay product. A 1 Gb/s path with a 70 ms round trip holds 70 million bits, almost 9 megabytes, in transit at once.

To fill that pipe, a TCP sender must be allowed to have that much data sent but not yet acknowledged. Its limit is the receive window. TCP's original header has only 16 bits for the window, a maximum of 65,535 bytes, which caps throughput at 65,535 × 8 ÷ RTT: about 7.5 Mb/s at 70 ms, no matter how fat the link. The window scale option (RFC 7323) multiplies it by up to 2¹⁴, which is how modern systems fill fast long-distance paths. Chapter 5 shows the sliding window at work.

Instrument 2

Latency & bandwidth calculator

Set a size, a link speed and a distance. The animation shows the message as it really sits on the wire: its front edge racing ahead at the speed of the medium while the sender is still pushing out its tail. Time is slowed down to fit, but the proportions are real.

Serialization–
Propagation–
Arrives after–
Ping round trip–
Bits in flight–
Stop-and-wait throughput–

Side by side

Delays, from tiny to huge

WhatDelayWhy
1,500 B at 10 Gb/s1.2 µsSerialization on a fast data-center link.
1 km of fiber~4.9 µsLight in glass covers about 204 m per microsecond.
1,500 B at 10 Mb/s1.2 msSerialization on old Ethernet: a thousand times the 10 Gb/s figure.
NYC → London, fiber~28 ms one wayStraight-line 5,600 km; real cables and routing add more, ~70 ms round trip is typical.
Geostationary satellite~240 ms one wayUp 35,786 km and back down, at the speed of light.
4 MB at 56 kb/s~10 minutesDial-up modem: serialization dominates everything.
Cheat sheet

Terms from this chapter

Bit, byte
A bit is a single 0 or 1. A byte is 8 bits. Link speeds count bits (b); file sizes usually count bytes (B).
Circuit switching
Reserving a fixed path and capacity for the whole of a conversation, as the old phone network did.
Packet switching
Chopping data into addressed packets that share links and are forwarded one at a time.
Layer
One slice of the networking job, using the layer below and serving the one above. OSI has 7; TCP/IP has 4.
Encapsulation
Each layer wrapping the data from the layer above in its own header (and sometimes trailer).
Segment, packet, frame
The unit of data at the transport, internet and link layers. A UDP unit is a datagram.
MTU
Maximum transmission unit: the largest payload a link carries in one frame. 1,500 bytes on standard Ethernet.
MSS
Maximum segment size: the most TCP data that fits in one packet. 1,460 bytes with IPv4 on Ethernet.
MAC address
A 48-bit hardware address, meaningful only on one local network.
IP address
A 32-bit (IPv4) or 128-bit (IPv6) address identifying a host across networks.
Port
A 16-bit number identifying a program, or one conversation, on a host.
Bandwidth
How many bits per second a link can carry.
Latency
How long data takes to arrive: serialization, propagation, queueing and processing added up.
Throughput
The useful data rate actually achieved end to end. Never above the bottleneck link.
Where this comes from

Sources