With his marriage to Norma Levor over, Claude Shannon was a bachelor again, with no attachments, a small Greenwich Village apartment,âŠ
An article about the invention of information theory in Nautilus. Excerpt:Â
Information was something guessed at rather than spoken of, something implied in a dozen ways before it was finally tied down. Information was a presence offstage. It was there in the studies of the physiologist Hermann von Helmholtz, who, electrifying frog muscles, first timed the speed of messages in animal nerves just as Thomson was timing the speed of messages in wires. It was there in the work of physicists like Rudolf Clausius and Ludwig Boltzmann, who were pioneering ways to quantify disorderâentropyâlittle suspecting that information might one day be quantified in the same way. Above all, information was in the networks that descended in part from the first attempt to bridge the Atlantic with underwater cables. In the attack on the practical engineering problems of connecting Points A and Bâwhat is the smallest number of wires we need to string up to handle a dayâs load of messages? how do we encrypt a top-secret telephone call?âthe properties of information itself, in general, were gradually uncovered. [...]
Each year that Shannon placed a call, he was less likely to speak to a human operator and more likely to have his call placed by machine, by one of the automated switchboards that Bell Labs grandly called a âmechanical brain.â In the process of assembling and refining these sprawling machines, Shannonâs generation of scientists came to understand information in much the same way that an earlier generation of scientists came to understand heat in the process of building steam engines. [...]
Where Nyquist used the vague concept of âintelligenceâ and Hartley struggled to explain the value of discarding the psychological and semantic, Shannon took it for granted that meaning could be ignored. In the same way, he readily accepted that information measures freedom of choice: What makes messages interesting is that they are âselected from a set of possible messages.â It would satisfy our intuitions, he agreed, if we stipulated that the amount of information on two punch cards doubled (rather than squared) the amount of information on one, or that two electronic channels could carry twice the information of one.That was Shannonâs debt. What he did next demonstrated his ambition. Every system of communicationânot just the ones existing in 1948, not just the ones made by human hands, but every system conceivableâcould be reduced to a radically simple essence.
âą The information source produces a message. âą The transmitter encodes the message into a form capable of being sent as a signal. âą The channel is the medium through which the signal passes. âą The noise source represents the distortions and corruptions that afflict the signal on its way to the receiver. âą The receiver decodes the message, reversing the action of the transmitter. âą The destination is the recipient of the message.
The beauty of this stripped-down model is that it applies universally. It is a story that messages cannot help but play outâhuman messages, messages in circuits, messages in the neurons, messages in the blood. You speak into a phone (source); the phone encodes the sound pressure of your voice into an electrical signal (transmitter); the signal passes into a wire (channel); a signal in a nearby wire interferes with it (noise); the signal is decoded back into sound (receiver); the sound reaches the ear at the other end (destination).
In one of your cells, a strand of your DNA contains the instructions to build a protein (source); the instructions are encoded in a strand of messenger RNA (transmitter); the messenger RNA carries the code to your cellâs sites of protein synthesis (channel); one of the âlettersâ in the RNA code is randomly switched in a âpoint mutationâ (noise); each three-âletterâ code is translated into an amino acid, proteinâs building block (receiver); the amino acids are bound into a protein chain, and the DNAâs instructions have been carried out (destination).
Those six boxes are flexible enough to apply even to the messages the world had not yet conceived ofâmessages for which Shannon was, here, preparing the way. They encompass human voices as electromagnetic waves that bounce off satellites and the ceaseless digital churn of the Internet. They pertain just as well to the codes written into DNA. Although the moleculeâs discovery was still five years in the future, Shannon was arguably the first to conceive of our genes as information bearers, an imaginative leap that erased the border between mechanical, electronic, and biological messages.
First, though, Shannon saw that information science had still failed to pin down something crucial about information: its probabilistic nature. When Nyquist and Hartley defined it as a choice from a set of symbols, they assumed that each choice from the set would be equally probable, and would be independent of all the symbols chosen previously. Itâs true, Shannon countered, that some choices are like this. But only some. For instance, a fair coin has a 50-50 chance of landing heads or tails. This simplest choice possibleâheads or tails, yes or no, 1 or zeroâis the most basic message that can exist. [...]
The new unit of Shannonâs science was to represent this basic situation of choice. Because it was a choice of 0 or 1, it was a âbinary digit.â In one of the only pieces of collaboration Shannon allowed on the entire project, he put it to a lunchroom table of his Bell Labs colleagues to come up with a snappier name. Binit and bigit were weighed and rejected, but the winning proposal was laid down by John Tukey, a Princeton professor working at Bell. Bit.
Read the whole thing.
