Neurons, Feedback and Information

In the 1940s, brain scientists, engineers and mathematicians began describing minds and machines in the same language. Their ideas — the artificial neuron, feedback, information, learning by strengthening connections — became the roots of both great schools of AI.

In a laboratory in Bristol around 1949, two small robots wander across the floor. Each is little more than a shell on three wheels, with a light sensor, a touch switch and two radio valves for a brain. Yet they seek out lights, back away from obstacles, circle each other and, when their batteries run low, find their way back to a hutch to recharge. Their builder, the neurophysiologist W. Grey Walter, called them tortoises. They were a demonstration of an idea that was electrifying the 1940s: that lifelike, purposeful behaviour might come from very simple parts wired together in the right way.

The neuron as a logic switch (1943)

In 1943 the neurophysiologist Warren McCulloch and Walter Pitts, a largely self-taught logician still in his teens when they began working together, published “A Logical Calculus of the Ideas Immanent in Nervous Activity”. They proposed a drastically simplified model of a nerve cell. Try it in your head:

  1. InputsThe neuron receives signals from other neurons. Each signal is either on (1) or off (0).
  2. Add them upIt counts the active excitatory inputs. In their model, an active inhibitory input could veto firing altogether.
  3. ThresholdIf the total reaches a set threshold, the neuron fires (outputs 1); otherwise it stays silent (0). Set the threshold to 2 with two inputs and you get AND; set it to 1 and you get OR.
  4. Networks computeWire such units together and you can build any logical expression. McCulloch and Pitts argued that networks of them, given memory, could compute anything a Turing machine can.

The model was too simple to describe real neurons accurately, and it could not learn: its connections and thresholds were fixed by the designer. But it made a startling claim plausible — that a brain is a kind of logical computing machine, and so a computing machine might be a kind of brain. It is the direct ancestor of every artificial neural network, and John von Neumann borrowed its notation in his influential 1945 report describing the design of the EDVAC computer.

Feedback and cybernetics (1948)

During the Second World War the MIT mathematician Norbert Wiener worked with the engineer Julian Bigelow on predicting the path of enemy aircraft so that guns could aim ahead of them. The work convinced Wiener that a single principle linked guns, thermostats, and a hand reaching for a glass: negative feedback. A system measures the gap between where it is and where it wants to be, and acts to shrink that gap. Purpose, on this view, need not be mysterious; it can be built.

In 1948 Wiener published Cybernetics: Or Control and Communication in the Animal and the Machine, naming the new science after the Greek word for a steersman. The book was a surprise bestseller. Cybernetics drew together physiologists, engineers, mathematicians, psychologists and anthropologists, many of whom met at a series of conferences sponsored by the Macy Foundation in New York between 1946 and 1953.

Grey Walter’s tortoises (1948–49), built at the Burden Neurological Institute in Bristol, were cybernetics made visible. So was the psychiatrist W. Ross Ashby’s Homeostat (1948), a machine that kept rearranging its own wiring at random until it found a stable state — a crude form of adapting to its surroundings, which Ashby described in Design for a Brain (1952).

Information and the first game-playing programs

Also in 1948, Claude Shannon of Bell Labs published “A Mathematical Theory of Communication”. He showed how to measure information precisely, in units called bits (a word he credited to his colleague John Tukey), and proved how much information any channel — a telephone wire, a radio link — can carry reliably despite noise. Information theory underlies every phone call, file download and video stream today, and its ideas of probability and prediction reappear in how language models are trained to predict the next word.

Shannon also turned to thinking machines. His 1950 paper “Programming a Computer for Playing Chess” set out how a computer could choose moves by looking ahead through possible replies and scoring the resulting positions — the basic plan of nearly every game-playing program until deep learning, including Deep Blue. He estimated the number of possible chess games at around 10120, far too many to search exhaustively, so a program would need rules of thumb to look ahead selectively. The same year he built Theseus, a mechanical mouse driven by telephone relays hidden under a maze. Theseus explored the maze by trial and error, “remembered” the route in its relays, and could then run straight to the goal: one of the earliest machines to learn from experience.

Learning by strengthening connections (1949)

The missing piece in the McCulloch–Pitts neuron was learning. In 1949 the Canadian psychologist Donald Hebb proposed a mechanism in The Organization of Behavior. When one cell repeatedly helps to fire another, he suggested, “some growth process or metabolic change” makes the first cell more effective at firing the second. Connections that are used together grow stronger. This is Hebbian learning.

Popular myth The catchy summary “cells that fire together wire together” is often quoted as Hebb’s own words. It is not: it is a later paraphrase, popularised in the early 1990s (the neuroscientist Carla Shatz used it in a 1992 Scientific American article). It is a fair shorthand, though Hebb stressed that the first cell must take part in causing the second to fire.

Hebb’s rule gave later researchers a way to make artificial networks adapt. Modern neural networks mostly learn by a different method, backpropagation, but the core idea — that knowledge is stored in the strengths of connections, adjusted by experience — comes from this moment.

The people and the clubs

These ideas spread through an unusually small and sociable network. Tap each card for a key figure or group not covered above.

JvNJohn von NeumannComputers and brains

The Hungarian-American mathematician wrote the 1945 report that defined the stored-program computer layout still called the “von Neumann architecture”, describing its components in McCulloch–Pitts terms. He spoke at the 1948 Hixon Symposium on how brains and machines might be compared, studied machines that could reproduce themselves, and left an unfinished set of lectures published as The Computer and the Brain (1958).

RATIOThe Ratio ClubLondon, 1949–1958

A British dining club founded in 1949 by the neurologist John Bates, bringing together about twenty young physiologists, engineers and mathematicians to discuss cybernetic ideas. Members included Alan Turing, Ross Ashby, Grey Walter, the neuroscientist Horace Barlow, the physicist Donald MacKay and the engineer Albert Uttley. It shows that Britain had its own lively scene, not just a branch of the American one.

MACYThe Macy conferencesNew York, 1946–1953

Ten meetings on “circular causal and feedback mechanisms” where Wiener, McCulloch, Pitts, von Neumann and the anthropologists Margaret Mead and Gregory Bateson argued across disciplines. They spread the vocabulary of feedback and information far beyond engineering.

Two schools from one root

This guide’s framing Looking back, the 1940s ideas split into two traditions that would compete for the next seventy years. Compare them:

Took from the 1940s: McCulloch and Pitts’ claim that thought is logic; Turing’s and von Neumann’s general-purpose computers; Shannon’s chess plan of searching through possibilities and scoring them with rules of thumb.

Became: symbolic AI — Newell and Simon’s Logic Theorist (1956), LISP, and the expert systems of the 1980s. Dominant until the late 1980s.

Quick check

What could the original McCulloch–Pitts neuron not do?

Its weights and thresholds were fixed by the designer. Learning came later, with ideas such as Hebb’s rule and the perceptron.

Who wrote “cells that fire together wire together” in 1949?

Hebb’s 1949 wording is longer and more careful; the rhyming version was popularised in the 1990s.

Which idea is at the heart of Wiener’s cybernetics?

Searching game trees was Shannon’s chess plan, and bits were Shannon’s information theory. Feedback was Wiener’s unifying principle.

// Pause and reflect

  1. Grey Walter’s tortoises looked purposeful with only two valves for a brain. What does that suggest about how easily we read intelligence into behaviour?
  2. The McCulloch–Pitts neuron fed both the symbolic and the neural schools. Why might the same idea lead people in opposite directions?
  3. Cybernetics lost its name but its ideas survived. Can you think of other ideas that won while their labels lost?

Your notes are saved privately in this browser.

What happened next

By the mid-1950s these strands were ready to be pulled together into one field. Claude Shannon, along with three younger colleagues, wrote a proposal for a summer workshop — and in doing so gave the field its name. Continue with Dartmouth 1956.

Key terms on this page

Sources & further reading