The Network · Chapter 10
How do bits cross a wire?
Transcript
How do bits cross a wire? Two machines, one wire between them. Noise can flip any bit on the way.
I figured you'd just add up the bytes and send the total, as a check. But two flips can cancel out, and the total still matches.
First, a bit on the wire is just a level, held for one tick of a shared clock. High is one, low is zero. The receiver samples each tick: zero, one, one, zero, one, zero, zero, zero: the letter h. The wire never stops, so a flag byte marks where each message starts and ends. For the check, don't add. Divide the bytes by a fixed number, in binary, and send the remainder along. Flip one bit, and the remainder changes, so the receiver knows.
So bits cross as levels, and a remainder catches the flips! The wire moved bytes; the link layer made corruption visible.
Type a message below, and watch it framed for the wire: flags, remainder and all.
- The browser
- The network
- Software and languages
- The CPU
- Memory
- Arithmetic
- Logic gates
- The switch
Two machines, and one wire strung between them. Real computing is many machines talking to each other, something no program in Parts I and II ever had to do, and Part III builds it one layer at a time, starting at the bottom: with that wire, and the problem of knowing whether a byte arrived intact. This is the data link layer, and it has two jobs. It must mark where one message ends and the next begins, and it must notice when the wire corrupts a bit.
10.1 A bit on the wire
Underneath everything is the physical layer, where a bit is nothing but a voltage. To send a byte, the sender holds the wire high or low, one bit at a time, for a fixed slice of time set by a shared clock. The receiver watches the wire and reads its level once per tick. Figure 10.2 shows the letter h, the byte 0x68, crossing a wire this way.
That is the whole of the physical layer for our purposes: a way to move bytes, one bit at a time, from here to there. It is also unreliable. The clock can drift, a voltage can be misread, noise can flip a bit. And on its own it gives no hint where a message starts or stops; it is just an unbroken stream of bytes. The link layer turns that raw stream into something usable. From here up, we work with the bytes the wire delivers and leave the voltages behind.
10.2 Framing: finding the boundaries
The receiver sees a stream of bytes with no markers. Where does one message end and the next begin? The answer is to wrap each message in a frame: a chosen byte, the FLAG, marks the start and the end. When the receiver sees a FLAG, it knows a frame boundary is there.
This raises an immediate problem. What if the message itself contains a byte equal to FLAG? The receiver would read it as a boundary and split the message in the wrong place. The fix is byte stuffing. Before sending, the sender replaces any FLAG or ESC byte in the data with a two-byte escape sequence: a special ESC byte followed by the original byte with one bit flipped. A real FLAG now appears only at the true boundaries. The receiver reverses the substitution after splitting. Figure 10.3 shows a frame and the stuffing rule.
FLAG opens and closes it; between them sit the
payload and the CRC. Below, the
byte-stuffing rule: a payload byte equal to a delimiter becomes ESC followed by
that byte masked, so no bare delimiter survives inside a frame.The sender’s half is one function. It appends the checksum, stuffs the bytes, and adds the flags.
Example 10-1. Framing a payload for the wire
def frame(payload: bytes) -> bytes: """Wrap a payload into a frame for the wire. Append the CRC, byte-stuff any FLAG or ESC bytes so they cannot be mistaken for delimiters, and surround the result with FLAG bytes. """ crc = crc16(payload) body = payload + bytes([crc >> 8, crc & 0xFF])1 stuffed = bytearray() for byte in body: if byte in (FLAG, ESC): stuffed.append(ESC) stuffed.append(byte ^ _MASK)2 else: stuffed.append(byte) return bytes([FLAG]) + bytes(stuffed) + bytes([FLAG])
- the two-byte CRC is appended to the payload before stuffing, so the checksum is protected by the same framing as the data it covers
- byte stuffing: a FLAG or ESC byte is replaced by ESC and the byte with one bit flipped, which the receiver undoes; a bare delimiter can then never occur inside the frame
The receiver does the reverse. It splits the incoming stream at the FLAG bytes, un-stuffs each frame to recover the original bytes, and then checks the CRC described next. A frame that fails the check is dropped.
10.3 Error detection: the CRC
Framing finds the message; it does not tell whether the message survived. A wire can flip a bit, and the receiver has no other copy to compare against. The standard answer is a cyclic redundancy check, a CRC: a checksum computed from the data and sent alongside it. The receiver recomputes the checksum from what it received; if the two disagree, a bit was corrupted.
A CRC is not a sum. It is the remainder of a long division, carried out in binary, bit by bit, with a fixed divisor called the generator polynomial. Treating the whole message as one enormous binary number and dividing leaves a remainder that changes if a single bit flips, which makes it a useful error detector. Some multi-bit changes can share a remainder, so a matching CRC is not proof that arbitrary corruption was absent.
Example 10-2. The CRC, one bit at a time
def crc16(data: bytes) -> int: """The CRC-16-CCITT checksum of data, computed one bit at a time. This is polynomial division modulo 2: the message is treated as a long binary number and divided by the generator polynomial; the remainder is the checksum. A single flipped bit changes the remainder, so the receiver can detect it. """ crc = 0xFFFF for byte in data: crc ^= byte << 81 for _ in range(8): if crc & 0x8000: crc = (crc << 1) ^ _POLY2 else: crc <<= 1 crc &= 0xFFFF return crc
- each message byte is brought into the top of the 16-bit register
- the division step, repeated for all eight bits of the byte: when the leading bit is 1 the register is shifted and the polynomial is subtracted, which modulo 2 is an XOR. The leftover after the whole message is the checksum.
This is CRC-16/CCITT-FALSE, the variant with the initial value 0xFFFF and no bit reflection. Feeding it the bytes 123456789 gives 0x29B1, the published check value for this variant, a reference vector used by the test.
10.4 The link, run
Running ftb.link sends two messages over a simulated wire. The second deliberately contains a FLAG byte and an ESC byte, to exercise the stuffing. Then it flips one bit and sends them again.
messages sent: [b'hi', b'~}!'] on the wire: 7e 68 69 62 03 7e 7e 7d 5e 7d 5d 21 45 d3 7e received: [b'hi', b'~}!'] after flipping one bit: [b'~}!'] the corrupted frame failed its CRC and was dropped.
Both messages come back exactly as sent, the stuffed bytes recovered, and the second message survives even though it contained the very bytes used as delimiters. When a single bit is flipped, the CRC of that frame no longer matches, and the receiver drops it while keeping the good one. The wire moved bytes; the link layer made corruption visible.
Figure 10.2 watched a single bit cross the wire, and then we moved up to bytes; the next chapter climbs back down and builds that crossing by hand, wire and clock and all.
Exercises
- Flip one byte of a framed message, and watch the checksum catch the damage.
- Send a payload that itself contains the frame’s flag byte, and confirm it still arrives whole.
