CHAPTER 04/1960–1971

Inside the First Boom and Its Forecasts

Search and reasoning. What AI could do, and the big forecasts (1960–1971)

Finding the way through a maze, turning word problems into equations, reading handwritten digits. The AI of the 1960s did startling things by using rules, estimates and templates that people had written in. And from it came big predictions: “within twenty years,” “within a generation.” Open the original sources, and both the predictions and the famous stories show a different face.

011960–1966 · Japan

Telling Voices Apart, Sound by Sound

Where does a machine tell “ah” from “ee”?

In 1960, Toshiyuki Sakai and colleagues at Kyoto University were researching a that would recognize a person's voice and turn it into letters. According to articles of the time, the aim was a device that could identify the 100 single syllables of Japanese, one sound at a time. It identifies vowels by the positions of the two peaks of strong resonance in the voice (the first and second formants), and consonants by combining clues such as how many times the waveform crosses zero.

The figure on the right is a plane with the frequencies of the two peaks as its axes. The five vowels (a, i, u, e, o) gather on separate islands. Move the point, and the vowel of the nearest island is the answer. Add noise, and mistakes start to appear between the islands.

answer=the vowel k that minimizes √((F1−F1ₖ)² + (F2−F2ₖ)²)
F1, F2
heights (frequencies) of the voice's first and second peaks
F1ₖ, F2ₖ
center of the island for vowel k (rough values chosen by this site)

The Japanese term for (人工知能) can be found in papers from 1962. The 1966 symposium at Tohoku University gave its theme in English as , and more than 200 people gathered for talks such as Kyoto University's on “recognition machines” for reading characters.

What the figure shows:A voice becomes 1 point on a map using 2 numbers, the first peak and the second peak. The machine simply answers with the nearest vowel island, so points that spread beyond their island can be mistaken for a neighboring vowel.

Voice waveform and the number of zero crossings
021961–1969 · Robots

An Arm That Memorizes and Repeats

What did the first robot to work in a factory “remember”?

The Devol patent granted in 1961 imagined an arm that, instead of a cam (a part whose rotating shape produces only one fixed motion), could store a program of motions in memory and have it swapped out, and called it . The same year, the Unimation company's robot Unimate entered a General Motors factory, where it took hot parts out of the mold and stacked them.

Try swapping the motion program in the figure on the right. The arm just follows the points it was taught in order, and it does not know what the motion is for. In 1969, when Kawasaki Heavy Industries completed Japan's first industrial robot, the “Kawasaki-Unimate 2000”, it caused a big stir in Japan as a .

x=L₁cos θ₁ +L₂cos(θ₁+θ₂)
x
horizontal position of the hand (the vertical position y has the same form with cos changed to sin)
θ₁, θ₂
the angles of the two joints (the numbers stored as the program)
L₁, L₂
arm lengths

On January 1, 1963, the TV anime (Tetsuwan Atom) began, and a boy robot with a heart appeared on television every week. The robot in the factory was an arm that repeated the motions it had memorized.

What you think you knowThe popular story: was Japan's first TV anime
The original source says:Broadcasting-library and production-company wording is . Short Japanese TV animations existed even before it.

What the figure shows:The robot arm remembers only the angles of 2 joints at each point. Reordering the steps lets the same arm do a different job, but it does not know what its movements are for.

031962–2007 · Games

Was Checkers ?

What does it mean to “solve” a game?

In 1962, a checkers program (checkers is a game played on an 8×8 board, capturing by jumping over pieces) by Arthur Samuel of IBM beat the blind player Nealey in one game, and checkers was reported as . In 1965, Richard Bellman wrote that .

To solve a game is to decide, for every position, whether it is a win, a loss or a draw when both sides play their best. The reliable method is to trace back from the final positions (backward analysis). Press "Go back one step" in the stone-taking game on the right. The player who is to move with 0 stones left loses. If there is a move that hands the opponent a losing position, the position is a win.

Checkers has about 5×10²⁰ positions. In 2007, a team at the University of Alberta, computing since 1989, proved that with perfect play by both sides the game is a draw.

value of a position=win (there is at least one move to a losing position)/loss (every move leads to a winning position)
position
the state of the board at some moment
move
one move that goes to the next position
What you think you knowThe popular story:In 1962 Samuel's program beat a checkers master, and checkers was solved
The original source says:It won only one game, and the opponent was one class below master. In a match the next year, the opponent won without losing a game, and in 1965 the program lost all 8 games against two players from the world championship. Checkers was truly solved in 2007.

What the figure shows:Solving a game means working backward from final positions and marking every position as a win or loss. A stone-taking game is solved quickly, but checkers has about 5×10²⁰ positions and took until 2007.

041964 · Word Problems

Turning Word Problems into Equations

How did a machine turn “15 percent less than …” into an equation?

In 1964, Daniel Bobrow, a graduate student at MIT, built for his doctoral thesis a program called STUDENT that read and solved algebra word problems written in English. He rewrote some of the word problems from a first-year high-school textbook into English that STUDENT could read, and tried them.

STUDENT does not think about the meaning of a sentence. It replaces fixed phrases with parts of an equation: the English word is becomes an equals sign, and percent less than becomes a multiplication. Press "Go forward one step" in the figure on the right. When it lacked an equation, it filled in known relations such as and told the user the assumptions it had made. A phrase that is not in its rules cannot become an equation, even if it means the same thing.

69.70= (1 − 15/100) × listed price
is
make it an equals sign
percent less than
make it × (1 − p/100)
sum of
make it an addition

Bobrow defined what he meant by saying as this: it , for that range of sentences.

What the figure shows:STUDENT solved word problems by replacing fixed phrases with parts of equations, without considering their meaning. A phrase outside its rules cannot be solved, even if it means the same thing.

051966 · Conversation

The Trick Behind ELIZA

What did a program that talks with people in text actually “understand”?

In January 1966, Joseph Weizenbaum of MIT published a paper on ELIZA, a program that converses with a computer user in text. ELIZA looks for a keyword in the input sentence. A decomposition rule for that keyword cuts the sentence into parts, and a reassembly rule builds the reply. If there is no keyword, it returns an empty acknowledgment.

Keywords have ranks. In the paper's example, everybody is ranked higher than I. Then ELIZA reacts to everybody and can ask who is meant. Type a sentence into the figure on the right to see the keyword it found and the rule it used. The set of keywords and rules (the script) is data separate from the program, and swapping it changes the role of the conversation partner.

(0 YOU 0 ME) → (WHAT MAKES YOU THINK I 3 YOU)
0
a part that matches any number of words
3
insert the third decomposed part as it is (the paper's example)

At the time of the paper, even the biggest script had about 50 keywords. Besides English, there were scripts in Welsh and German.

What you think you knowThe popular story:ELIZA was built to be an AI that could converse like a person
The original source says:At the very start the paper says that , and goes on: .

What the figure shows:ELIZA only finds keywords in a sentence and reshapes them into a fixed reply pattern. It may seem to understand, but revealing the trick shows that it does not examine meaning.

061964 or 1966 · A Japanese Teaching Material

What Year Is ELIZA From?

What is the “1964” in the timeline the year of?

A table in a teaching material for high-school teachers of “Information II” (an elective informatics course in Japan) from 2020 says . This table has been copied from a 2016 government survey report to a white paper of the Ministry of Internal Affairs and Communications, and then to the teaching material.

Lining up the original sources: Weizenbaum's paper was accepted in September 1965 and appeared in the January 1966 issue. MIT's annual report of the time (covering July 1964 to July 1965) says that development of a system for natural-language conversation between people and computers began during that period. The original program that has been found was the 1965 version.

Press the tags on the right to see what that year is the year of. 1964 is plausible as around when development began, but it is not the year of completion or announcement. The table in the teaching material and the paper are writing about different moments. Once you know what each year is the year of, both numbers can be read correctly.

English-language sources also waver on the year. IEEE's profile of computing pioneers describes him as , and gives 1965 as the year of the journal issue too (the printed issue is 1966).

What the figure shows:The timeline's 1964 and the paper's 1966 refer to different stages, roughly the start of development and publication, rather than necessarily showing that either date is wrong.

071965 · Chemistry

Build All Candidates, Narrow by Rules

When there are millions of candidate answers, how does an expert narrow them down?

At Stanford University, Joshua Lederberg had been building DENDRAL (1964), a procedure that counts, as a tree, all the possible molecular structures for a given chemical formula. From 1965, Edward Feigenbaum and colleagues turned it into a program that narrows the candidates by building chemists' experience-based knowledge into rules. This was knowledge for reading mass spectrometry data (a method that breaks a molecule apart and weighs the fragments). For one compound, 14,715,813 possible structures were cut to 1,284,792 by mass spectrometry, and to just 1 when another measurement (nuclear magnetic resonance) was also used.

The figure on the right is a small version of this idea. It builds every carbon-only skeleton, predicts the weights of the fragments, and removes the structures that do not match the measured weights.

build → predict → compare (keep only what matches)
fragment weight
a fragment of k carbons is taken to weigh 14k+1 (this site's simple rule; real mass spectrometry is much more complicated)

A 1970 paper wrote:

What you think you knowThe popular story:The first expert system was MYCIN (1972)
The original source says:Japan's white paper says so, but DENDRAL, which began in 1965, came earlier as a program that put experts' experience-based knowledge into rules. The developers themselves, in the title of a 1993 paper, call DENDRAL .

What the figure shows:Generating all candidates and then using expert knowledge, the measured weights of fragments, to remove those that do not fit sharply reduces a huge pile of possibilities.

081965 · Sets

From What Height Is a Person “Tall”?

Can a collection with no clear in-or-out be expressed as a number?

In 1965, Lotfi Zadeh of the University of California, Berkeley, published the paper . Collections such as or the have no clear boundary between in and out. Yet such collections play an important role when people recognize patterns and communicate information. So Zadeh expressed how much something belongs to the collection as a number from 0 to 1 (the degree of membership).

Move the height in the figure on the right. An ordinary set switches suddenly from 0 to 1 at a certain height. A fuzzy set rises smoothly. The paper defined “or” as the larger value, “and” as the smaller value, and “not” as 1 minus the number.

or=Max[μA, μB] and=Min[μA, μB] not=1 − μA
μA
degree of membership in set A (0–1)

The shape of the degree (0 at 160 cm, 1 at 185 cm) is an example chosen by this site. Who decides it, and how, becomes a convention for each use.

What the figure shows:Even a group with unclear boundaries, such as tall people, can be used in calculations by expressing membership as a degree from 0 to 1. Ordinary sets switch suddenly at a boundary, while fuzzy sets change smoothly.

091960–1966 · Predictions

Can Machines Do Any Work People Can?

Did the “within twenty years” prediction mean that people's jobs would disappear?

In 1965, Herbert Simon wrote in a book that a person can do. In the original 1960 paper he foresaw , but right after that he went on to say the technology .

What Simon used was the economists' idea of . Even if machines are faster than people at every job, machine time is limited. If each takes on what it is relatively good at, the whole produces more. In the figure on the right, even when you raise the machine's speed, under "Divide by what each is good at" the people's work remains.

work handled by people=the work with the smallest ratio of (machine speed ÷ human speed)
comparative advantage
what you are relatively good at compared with others (not speed itself)
What you think you knowThe popular story:Simon predicted that in 20 years machines would take all of people's work
The original source says:The original paper separated the forecast of technical capability from how it would be used. Looking at the long run, he thought almost everyone who wanted to work could. In 1966 he argued back that what decides the level of employment is not technology but the workings of the economy.

What the figure shows:Even if machines become faster than people at every job, machine time is limited. Giving people the jobs they are relatively good at increases total output, so machines being able to do everything does not mean people have no work.

101965 · Predictions

Doubling Every Year for 10 Years

By how many times does something that doubles every year grow in 10 years?

In 1965, Gordon Moore of Fairchild Semiconductor wrote in the article that the number of components on one circuit had been roughly doubling every year (the number at which the cost per component is lowest). He said this rate would continue , and forecast 65,000 components in 1975.

Select “Doubling scale” in the graph on the right, and the line becomes straight. This is a sign of growing at the same rate every year. If you change the doubling time T, the forecast for 1975 changes a lot. The lower graph under “Details: article points, costs and sources” shows how the number of components at the lowest cost per component moves to the right as the years go by.

number of components=64 × 2(year − 1965) ÷ T
T
years to double (about 1 year in the article)
64
2⁶ (the 1965 point)

In 1975, Moore himself looked back and said the number had doubled almost every year until then, and revised his forecast: from now on it might be . The name was given by someone else.

What the figure shows:Doubling every year for 10 years gives about a 1000-fold increase. Even a small change in the doubling time can make the numbers after 10 years differ by dozens of times.

Cost per component and number of components (the cheapest point each year)
111965–1967 · Criticism

Narrowing Down the Moves to Read

Did chess computers read every move they could play?

In 1965, the philosopher Hubert Dreyfus argued in the RAND report that after its first successes, AI was running into a wall bigger than expected. People think using clues at the edge of their field of view and a sense of what matters, he wrote, but when that is translated into the language of computers, it drops out.

In 1967, the chess program Mac Hack VI, built by Richard Greenblatt at MIT, beat Dreyfus in a game. This program does not read all the moves. Using about 50 clues, it gives each move a plausibility score and reads only the high-scoring moves. At the tournament, it narrowed the number of moves read to 15 for the 1st and 2nd moves, 9 for the 3rd and 4th, and 7 beyond that. In the figure on the right, change how it narrows and how deep it looks.

positions read=w₁ × w₂ × … × wd (for all moves, bd)
wk
number of moves read at the k-th move
b
number of possible moves (a made-up number in the figure)
d
depth
What you think you knowThe popular story:Dreyfus predicted that computers could not beat even an amateur at chess, and then lost himself
The original source says:The 1965 original sentence described the situation of the time: . He himself answered that (according to Papert's memo).

What the figure shows:The chess program did not examine every legal move. Restricting its search to plausible moves reduced the number of positions enormously, allowing it to look further ahead.

121966–1972 · Robots

Bump, Fix the Map, Search Again

What should a robot do when it bumps into something it could not see?

From 1966 to 1972, SRI (the Stanford Research Institute) researched a wheeled robot called . It carried a TV camera, a range finder and bump sensors, and a large computer linked by radio built a map of the room from what it saw, planned routes and moved the robot. It shook when it stopped suddenly, so it was called Shakey (the shaky one).

In the working example in a 1970 report, Shakey plans a route and moves along it. When it bumps into something it did not see, it stops, takes a picture again, and plans again. The figure on the right moves automatically. Press “Stop” to pause it and "Keep going" to resume. The dotted objects are not yet on the map. The search method can be switched between breadth-first, which spreads out from nearby places, and depth-first, which goes as far as it can and then backs up.

breadth-first ← queue depth-first ← stack
queue
examine first the place found first (first in, first out)
stack
examine first the place found last (last in, first out)
What you think you knowThe popular story:Life magazine called Shakey the “first electronic person”
The original source says:In the article that phrase is quoted as the words of a young researcher who showed the reporter around. In the same issue, the reporter wrote that the tasks he saw . SRI's 1984 summary describes Shakey as a robot with .

What the figure shows:Shakey maps what it sees, plans a route, and moves along it. If it hits something missing from the map, it stops, corrects the map, and plans again.

131966 · Computer Vision

Separating Objects from the Background

How does a computer find the “objects” in a photograph?

In July 1966, Seymour Papert of MIT wrote a memo planning to have summer workers build . The plan was to separate objects from the background in the images from a scanner, examine the properties of their surfaces, and name the objects it knew. In July it would handle only spheres, bricks and cylinders placed without overlapping (uniform surface color and background). In August, it would extend to cigarette packs and batteries, and then to tools and cups.

The figure on the right is the first step, the mechanism for "separating objects from the background". Points darker than the threshold are treated as objects, and connected points are grouped and counted. If the background is uniform, the count is right. But if the background is uneven, even the pattern becomes an object, and objects that touch merge into one.

object point ⇔ brightness<threshold group=a set of object points connected up, down, left or right

Nilsson (2010), who wrote a book on the history of AI, writes that this plan was much harder than expected, but became the start of continuing vision research at MIT.

What you think you knowThe popular story:MIT tried to have one undergraduate solve “computer vision” in a single summer
The original source says:The memo lists about 11 names as people in charge, and plans to handle only spheres, bricks and cylinders on a uniform background in July, widening step by step in August. However, later history books also write that it was much harder than expected.

What the figure shows:An initial way to find objects in a photo is to connect points darker than a threshold and count the resulting groups. It works with a uniform background, but uneven backgrounds or touching objects quickly cause counting errors.

141967 · Japan · How to Learn

Learning Little by Little from Errors

How does a machine learn from data that cannot be cleanly separated?

In 1967, Shun'ichi Amari studied learning methods mathematically in the paper . It is a method in which, each time one example is seen, the weights (how heavily each feature counts) are corrected a little according to the error. The abstract showed that the weights approach the optimum even when the two groups cannot be cleanly separated, and showed the trade-off between speed and accuracy. A 1968 explanation in Japanese calls this the “stochastic descent method”.

In the figure on the right, the points of the two groups overlap. A large learning rate η gets close quickly but keeps shaking forever. A small one settles down but takes a long time. Making η gradually smaller gets the best of both.

w ← w − η ∇L (x, t ; w)
w
weights
η
learning rate (size of the correction)
L
error on that one example
∇L
the direction showing which way to correct to reduce L

Schmidhuber (2022) cites this as probably the first paper to train a multilayer neural network with this method. The 1967 abstract, however, describes a theory of linear classifiers and their extensions.

What the figure shows:Each example adjusts the weights a little according to the error, moving toward the best line even when the data cannot be separated perfectly. Large updates are fast but unstable; gradually shrinking them combines the benefits of speed and stability.

Number of examples seen and error rate (dotted line: the best line)
151968 · Search

Shortest Path: Traveled + Estimated

When searching a maze, which square should be examined first?

In July 1968, Hart, Nilsson and Raphael of SRI published a paper on a theory for finding the minimum-cost path using a clue (a heuristic). According to Nilsson's recollection, for Shakey's path finding the idea arose of adding to an estimate of the to the goal. If the estimate is never larger than the true remaining distance, the shortest path is always found. This search method was named A* (A-star).

Compare three cases in the maze on the right. Without an estimate, it examines everything around the start in order of closeness. A* stretches toward the goal. If you triple the estimate, fewer squares are examined, but the path is no longer the shortest.

f(n)=g(n)+h(n) examine the square with the smallest f first
g(n)
distance from the start to n
h(n)
estimate from n to the goal (straight-line distance, vertical and horizontal)

In this maze, the squares examined were 133, 97 and 52, and the path lengths were 29, 29 and 39. A* is now also used in car-navigation route guidance and in games (Nilsson 2010).

What the figure shows:A* checks squares in order of the smallest sum of the distance traveled so far and the estimated distance to the goal. The estimate steers the search toward the goal, but making it too large gives a faster route that may not be the shortest.

161968 · Japan

0 if Negative, As Is if Positive

If you stack neurons that only add, does what they can do increase, however many layers you use?

In 1968, Kunihiko Fukushima of the NHK Science and Technical Research Laboratories used experiments on the eyes and brains of cats and monkeys as a model, and simulated a layered network on a computer. It was a first step toward a machine that reads handwritten characters. As a figure moves through the layers, it is broken into isolated points and line segments of each orientation, and then the end points of the segments are found.

The neuron he used outputs its input sum as it is if positive, and 0 if negative. This is the same form as the ReLU widely used in deep learning today. He also wrote that if everything is built from neurons that only add, the result is the same as two layers however many layers you stack. In the figure on the right, turn off "0 if negative (ReLU)", and negative values in places with no line flow into the next layer, so the ends are not found correctly.

output=max(0, Σ weight × input)
max(0, u)
0 if u is negative, u as it is if positive (today's name: ReLU)

The weights in the figure are simplified by this site (they are not the weights of the 1968 network itself). The neuron that finds ends subtracts neighbors along a line.

What you think you knowThe popular story:ReLU was an invention of 2010s deep learning
The original source says:An activation function of the same form (0 if negative, as is if positive) was already used as an imitation of how the eye works in Fukushima's 1968 explanation in Japanese and his 1969 paper in English.

What the figure shows:In 1968, Fukushima stacked layers of neurons that return 0 for negative inputs and leave positive inputs unchanged to find lines and edges. Without the negative-to-0 step, even stacked layers cannot find the edges correctly.

Sum of inputs and the neuron's output
171968–1969 · Japan

Reading Handwritten Postal Codes

How did a machine deal with characters it could not read?

On July 1, 1968, the day the postal code system began, the automatic postal-code reader and sorter that Toshiba had developed under the guidance of the Ministry of Posts and Telecommunications was shown at the Tokyo Central Post Office. Reading and sorting freely handwritten digits is said to have been . It analyzed about 300,000 handwritten digits collected from across the country, erased the red frames with an optical filter, and read the first 3 digits at a speed of about 6 items per second.

According to a 1969 paper by the developers, it examined each character in horizontal bands, sorted the features into 19 types, and compared their top-to-bottom sequence with the sequences of each digit's variations stored in a dictionary. A digit with 2 or more mismatches is removed. If one remains, that is the answer; if none or 2 or more remain, it is a “reject”, and the item was passed to a person.

remaining digits= { d|number of mismatches (character's sequence, d's dictionary sequences)<2 }
exactly one remains
that is the answer
0 or 2 or more
reject (a person reads it)

The characters and features (5 types) in the figure are examples made by this site. At the first International Joint Conference on Artificial Intelligence in 1969, 3 of the 5 papers from Japan were studies of character recognition.

What the figure shows:The postal code reader turned each character into a sequence of features from horizontal strips and compared it with a dictionary; if it could not settle on 1 answer, it marked it “reject” and passed it to a person. It chose human review over making a mistake.

Amount of noise and the reading results
181967–1970 · Predictions

A Generation, 3–8 Years, Unlikely Soon

Where were the forecasts of those years actually written?

In 1967, Marvin Minsky wrote in a book: . In 1970, an article by a Life magazine reporter said that Minsky had told him would be in a machine, and also reported another researcher saying . The 1968 film features HAL 9000, a computer that talks in a human voice.

In the same year, 1968, Japan's Shun'ichi Amari wrote: (translated). The figure on the right connects, with lines, the year each prediction was made to its deadline. The color and the shape of the tags show where the words came from. Move the "current year" to see what happened to the predictions whose deadlines had come.

The book does not say how many years is. A historian's book says Minsky later wrote a rebuttal denying the quotation, but I could not confirm whether what he denied was the statement.

What you think you knowThe popular story:Minsky stated flatly in 1970 that a machine with human-level intelligence would come in 3–8 years
The original source says:The words were originally in a magazine reporter's article (Life, issue of November 20, 1970, page 58D). The wording in his own book (1967) was (the book's pages could not be seen; this was confirmed through 2 books that cite it).

What the figure shows:Predictions from the same period about how many years something would take came from different sources: the person's own book, a reporter's article, or a secondhand quotation. What happened by each deadline is viewed alongside its source.

191971 · Language

In a World of Blocks, Conversation Works

If you narrow the range, does the computer “understand” language?

In January 1971, Terry Winograd of MIT presented in his doctoral thesis a program (later called SHRDLU) that answers questions and carries out commands in English conversation. Its subject was a world in which a simple robot with a hand and an eye moves blocks and pyramids on a table. To it first moved the block on top out of the way, picked it up, and answered To it asked back because there were three in the scene.

Press “Give a command” in the figure on the right. It narrows the objects by words for shape, color and size: if exactly one is left, it acts; if two or more, it asks back; if none, it says so. The thesis's vocabulary was 200 words, and one sentence took 5 to 20 seconds.

1 match → act 2 or more → ask back 0 → answer that none exists
matching objects
objects whose shape, color and size fit the words of the command
What you think you knowThe popular story:SHRDLU moved real blocks with a robot arm and understood English
The original source says:In the thesis, the robot and the blocks were a simulation on a screen. The scene data was given from the start, assuming , and the conversation was done by typing text. In 2004 he himself wrote that there is a fundamental gap between how SHRDLU worked and how the brain works (according to Nilsson).

What the figure shows:In a limited world such as a set of blocks, words can narrow down an object, allowing action when there is 1 match and a follow-up question when there are 2 or more. But words outside that world are unknown.

Closing

What We Still Don't Know in This Chapter

Whether Minsky himself denied the “3–8 years” statement is not known, because the rebuttal text itself could not be found. What span of years refers to is not written in the book either. The source that first put ELIZA's “1964” into the teaching-material table is also still unknown.

Dreyfus wrote that big advances, such as understanding ambiguous scenes, . How far that ability has been achieved by today's learning AI is judged differently by different people. Whether getting answers right is the same as understanding the meaning is also a question that has run on since the days of STUDENT and SHRDLU.

The AI of this chapter ran on rules, estimates and templates that people had thought up and written in. It could do astonishing things within a narrow range, but did not work outside it. In Japan around the same time, seeds that connect to today's deep learning were also growing, such as Amari's learning method that corrects weights little by little, and Fukushima's layered network of “0 if negative” neurons. The next chapter looks at the era of the , when expectations went unmet, and the seeds that grew during it.