Search and reasoning. What AI could do, and the big forecasts (1960–1971)
Finding the way through a maze, turning word problems into equations, reading handwritten digits. The AI of the 1960s did startling things by using rules, estimates and templates that people had written in. And from it came big predictions: “within twenty years,” “within a generation.” Open the original sources, and both the predictions and the famous stories show a different face.
011960–1966 · Japan
Telling Voices Apart, Sound by Sound
Where does a machine tell “ah” from “ee”?
In 1960, Toshiyuki Sakai and colleagues at Kyoto University were researching a that would recognize a person's voice and turn it into letters. According to articles of the time, the aim was a device that could identify the 100 single syllables of Japanese, one sound at a time. It identifies vowels by the positions of the two peaks of strong resonance in the voice (the first and second formants), and consonants by combining clues such as how many times the waveform crosses zero.
The figure on the right is a plane with the frequencies of the two peaks as its axes. The five vowels (a, i, u, e, o) gather on separate islands. Move the point, and the vowel of the nearest island is the answer. Add noise, and mistakes start to appear between the islands.
answer=the vowel k that minimizes √((F1−F1ₖ)² + (F2−F2ₖ)²)
F1, F2
heights (frequencies) of the voice's first and second peaks
F1ₖ, F2ₖ
center of the island for vowel k (rough values chosen by this site)
The Japanese term for (人工知能) can be found in papers from 1962. The 1966 symposium at Tohoku University gave its theme in English as , and more than 200 people gathered for talks such as Kyoto University's on “recognition machines” for reading characters.
Why the peak heights change with the vowel
A voice is a sound made in the throat that resonates in the mouth. The way the mouth opens and the position of the tongue change the frequencies that resonate. For “ah,” with the mouth wide open, the first peak is high; for “ee,” with the tongue pushed forward, the second peak is high.
An article from December 1960 describes the device as still under research. Later accounts say it was built around 1960 and told sounds apart one at a time on the spot.
Sources & confidence (3)Telling Voices Apart, Sound by Sound
1960–1962Verified in the originalSources disagreeJapan
Kyoto University's (a machine that recognizes Japanese one sound at a time)
Toshiyuki Sakai and colleagues at Kyoto University studied a , which would recognize a person's voice with a machine and turn it into written characters. A 1960 article says they were studying a device that recognizes 100 single syllables (one sound at a time) of Japanese, judging vowels from the peaks in the sound's resonance (formants) and consonants by combining things such as how many times the waveform crosses zero. A later commentary says it was a circuit of vacuum tubes and transistors as big as a large computer, and that it recognized one sound at a time on the spot.
3 sources and how they were checked
Where sources disagree:When it was “completed”: a December 1960 article describes it as “under study”, and Kawahara's commentary says it was “made around 1960”. The English-language paper was presented in 1962 (IFIP congress).
The Japanese term for : in papers from 1962, in the National Diet from 1983
A search of J-STAGE (Japan's database of academic papers) turned up, in a December 1962 paper by Moriya Oda of Nagoya University, the phrase (translated) (a sentence introducing a survey by Minsky and others). Meanwhile, at a 1966 symposium at Tohoku University, the theme was written in English as . In the National Diet proceedings, the Japanese term for had zero hits through 1982. Its first appearance was in February 1983, when an expert witness brought it up in the phrase (translated).
What we still don’t know:J-STAGE's full-text search cannot search old scanned papers that have no text data, so there may be uses from before 1962. Newspapers (paid article databases such as the Asahi and Yomiuri) and books of the time have not been checked.
A symposium on at Tohoku University (more than 200 people)
The Research Institute of Electrical Communication at Tohoku University chose as the theme of its 3rd symposium and co-hosted it with the Information Processing Society of Japan and others. There were talks on pattern recognition (Taizo Iijima of the Electrotechnical Laboratory), one of the “recognition machines”, which reads characters (Toshiyuki Sakai and Makoto Nagao of Kyoto University), speech information processing (Osamu Fujimura of the University of Tokyo), “self-organizing systems” that change their own criteria for judgment, like the perceptron (Shoichi Noguchi of Tohoku University), and learning control (Kahei Nakamura of Nagoya University), among others, and more than 200 people took part. The theme was kept in English, and the Japanese term for is not used even in the report.
What we still don’t know:The report does not say why the theme was left in English (it is not known whether the Japanese term had not yet been settled or whether English was chosen on purpose).
What the figure shows:A voice becomes 1 point on a map using 2 numbers, the first peak and the second peak. The machine simply answers with the nearest vowel island, so points that spread beyond their island can be mistaken for a neighboring vowel.
Voice waveform and the number of zero crossings
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_vowel_formant.py
021961–1969 · Robots
An Arm That Memorizes and Repeats
What did the first robot to work in a factory “remember”?
The Devol patent granted in 1961 imagined an arm that, instead of a cam (a part whose rotating shape produces only one fixed motion), could store a program of motions in memory and have it swapped out, and called it . The same year, the Unimation company's robot Unimate entered a General Motors factory, where it took hot parts out of the mold and stacked them.
Try swapping the motion program in the figure on the right. The arm just follows the points it was taught in order, and it does not know what the motion is for. In 1969, when Kawasaki Heavy Industries completed Japan's first industrial robot, the “Kawasaki-Unimate 2000”, it caused a big stir in Japan as a .
x=L₁cos θ₁ +L₂cos(θ₁+θ₂)
x
horizontal position of the hand (the vertical position y has the same form with cos changed to sin)
θ₁, θ₂
the angles of the two joints (the numbers stored as the program)
L₁, L₂
arm lengths
On January 1, 1963, the TV anime (Tetsuwan Atom) began, and a boy robot with a heart appeared on television every week. The robot in the factory was an arm that repeated the motions it had memorized.
What you think you knowThe popular story: was Japan's first TV anime The original source says:Broadcasting-library and production-company wording is . Short Japanese TV animations existed even before it.
Same machine, different reception
In the United States, workers worried that industrial robots would . In Japan during the period of rapid economic growth, they were welcomed as a way to make up for labor shortages.
However, Japan's first unit was at first used only for evaluation, and real use in car factories began only in 1973, with a welding robot.
Sources & confidence (3)An Arm That Memorizes and Repeats
1961Confirmed in summariesSources disagree
The first robot to work in a factory, the — in 1961 it takes hot castings out of machines at a GM plant
The patent , which the inventor George Devol applied for in 1954 and which was granted on June 13, 1961, described a mechanical arm usable for many purposes. Instead of cams that could only make fixed motions, or manual operation by people, the sequence of motions (the program) can be recorded and swapped. The patent called this . The first robot of Unimation, Inc., the company founded by Devol and Joseph Engelberger, was installed in 1961 at a General Motors (GM) plant in Trenton, New Jersey, USA. Its job was to take hot parts out of a die-casting machine (casting by pouring metal into a mold) and stack them. It weighed about 3,000 pounds (about 1.4 tonnes).
4 sources and how they were checked
Where sources disagree:On the year of invention or birth, material from the Japan Institute of Invention and Innovation says “created the first industrial robot arm, ‘Unimate’, in 1959”, and Kawasaki Heavy Industries' 50-year history writes “triggered by Devol and Engelberger's meeting in 1956” (a record document on the 1969 Kawasaki Unimate). That it was installed in a factory in 1961 is consistent across three sources.
Japan's first domestically built industrial robot, the “Kawasaki Unimate 2000” — a big response in a Japan short of workers
In 1966, Kawasaki Heavy Industries invited Engelberger of Unimation in the United States and held a seminar. Against expectations it was a great success and set off the industrial robot boom in Japan (Japan Institute of Invention and Innovation). Kawasaki Heavy Industries formed a technology partnership with the company in 1968 and in 1969 completed the first unit of Japan's first domestically built industrial robot, the “Kawasaki-Unimate 2000”, at its Akashi Works. It drew a big response as something to make up for the labor shortage of the period of rapid economic growth (Kawasaki Heavy Industries). At first it stayed at the evaluation stage, but after spot-welding robots were delivered to Nissan Motor and Toyota Motor in 1973, it began to be used in earnest at car factories (Japan Institute of Invention and Innovation).
What we still don’t know:Even for the same industrial robots, in the United States workers worried that robots would (translated) (Japan Institute of Invention and Innovation). In Japan during the high-growth period, on the other hand, they drew a big response as the arrival of a (translated) (Kawasaki Heavy Industries). How the reception of the same technology changes with a country's labor situation is a question that connects to today's AI as well.
3 sources and how they were checked
Where sources disagree:The year the Unimate was born: the Japan Institute of Invention and Innovation says it created the first industrial robot arm, the Unimate, in 1959, while Kawasaki Heavy Industries writes that a 1956 meeting was the start. An American museum that holds the actual machine has a record of the first unit, installed at a GM factory in 1961 (the record of the Unimate at the GM factory in 1961).
The TV anime (Tetsuwan Atom) begins broadcasting — a robot boy with a heart appears on TV every week
Osamu Tezuka's manga (serialized from 1952; a separate record) was made into a TV anime, produced by Tezuka's Mushi Production. Broadcasting began on Fuji Television on Tuesday, January 1, 1963 at 6:15 p.m. It was the first Japanese-made series of 30-minute weekly TV anime, and it aired in black and white for 193 episodes until December 31, 1966. Episode 1, , tells how Dr. Tenma, who has lost his son in a traffic accident, builds a robot that looks just like him, but gets angry that it does not grow and sells it. It recorded a peak audience rating of 40.7% (Tezuka Productions) and was also broadcast in the United States through NBC FILMS (Mushi Production).
The popular story:People often say that “Astro Boy was Japan's first TV anime” The original source says:the wording used by the production company and the broadcast library is (translated). Before that there were already short Japanese-made TV anime, such as a trial work aired on Nippon Television in 1958 and , a 4-minute-per-episode program aired on Fuji Television from 1961 (WEB Anime Style).
What the figure shows:The robot arm remembers only the angles of 2 joints at each point. Reordering the steps lets the same arm do a different job, but it does not know what its movements are for.
031962–2007 · Games
Was Checkers ?
What does it mean to “solve” a game?
In 1962, a checkers program (checkers is a game played on an 8×8 board, capturing by jumping over pieces) by Arthur Samuel of IBM beat the blind player Nealey in one game, and checkers was reported as . In 1965, Richard Bellman wrote that .
To solve a game is to decide, for every position, whether it is a win, a loss or a draw when both sides play their best. The reliable method is to trace back from the final positions (backward analysis). Press "Go back one step" in the stone-taking game on the right. The player who is to move with 0 stones left loses. If there is a move that hands the opponent a losing position, the position is a win.
Checkers has about 5×10²⁰ positions. In 2007, a team at the University of Alberta, computing since 1989, proved that with perfect play by both sides the game is a draw.
value of a position=win (there is at least one move to a losing position)/loss (every move leads to a winning position)
position
the state of the board at some moment
move
one move that goes to the next position
What you think you knowThe popular story:In 1962 Samuel's program beat a checkers master, and checkers was solved The original source says:It won only one game, and the opponent was one class below master. In a match the next year, the opponent won without losing a game, and in 1965 the program lost all 8 games against two players from the world championship. Checkers was truly solved in 2007.
Wobbles in years and names
The 1996 paper by Schaeffer and colleagues and Nilsson (2010) date this game to 1962, while the 2007 paper says 1963. The player's name is also spelled two ways, Nealey and Nealy.
This site has not been able to check the text of Bellman's paper. The words come from a quotation in the 1996 paper by the team that solved checkers. The 2007 proof worked backward from the end through every position with 10 or fewer pieces (about 3.9×10¹³ positions).
Sources & confidence (3)Was Checkers ?
1962Confirmed in summariesSources disagree
What “beat a checkers master” really meant (1962, Samuel's program vs. Nealey)
In 1962, Arthur Samuel's program beat Robert Nealey, a blind checkers player from Connecticut, in one game. As a result, checkers was widely reported as . According to Schaeffer and colleagues, who later solved checkers completely, Nealey called himself a master, but his actual highest rank was one below master. The game he lost contained a simple mistake, and in a match a year later Nealey won without losing a single game. In 1965, the program played four games each against two men who were fighting for the world championship, and lost all eight. Samuel himself is said to have apologized in a letter for the mistaken belief, which had spread, that checkers had been solved.
The popular story:People tend to say that in 1962 Samuel's program beat a checkers master and that checkers was solved The original source says:it won only one game, and the opponent was one rank below master. In the match the next year the opponent won without losing, and in 1965 the program lost all 8 games to two players from the world championship. That checkers was completely solved (it is a draw if both sides play the best moves) was proved in 2007.
2 sources and how they were checked
Where sources disagree:Nealey's strength: “blind checkers master” (Nilsson p.128)/ “claimed to be a master, but his highest rank was one below master” (Schaeffer & Lake 1996 p.120, citing Fortman 1978). Spelling of the name: Nealey (Nilsson)/Nealy (Schaeffer & Lake)
Richard Bellman, who had criticized Simon and Newell's predictions in 1958, also stated a prediction in a 1965 paper. He wrote that within ten years checkers would become a game in which the best move is completely known (Schaeffer and colleagues present this as a forecast made in response to the reports of Samuel's one win in 1962).
What happened next:It was announced in September 2007 that checkers had been completely solved and that, if both sides play their best, the game ends in a draw. The team that solved it had been computing since 1989.
Checkers : computers prove it is always a draw if neither side makes a mistake
Jonathan Schaeffer and his colleagues at the University of Alberta kept dozens of computers running almost nonstop from 1989 and proved that Checkers has about 5×10²⁰ (500 quintillion) board positions, making it the hardest well-known game solved so far. One of the main tools in the proof was the search method (df-pn) from Ayumu Nagai's 2002 doctoral thesis at the University of Tokyo. Akihiro Kishimoto, who by the time of the paper had moved to a university in Hakodate, Hokkaido, is also among the co-authors.
The popular story:People tend to think “game AI is a Western story” The original source says:In the 2007 checkers proof, df-pn, the search method of Ayumu Nagai of Japan (University of Tokyo), was used as a core tool, and Akihiro Kishimoto of Japan was among the authors
3 sources and how they were checked
Checkers Is Solved (Science 317, 1518–1522) (Jonathan Schaeffer, Neil Burch, Yngvi Björnsson, Akihiro Kishimoto, Martin Müller, Robert Lake, Paul Lu, Steve Sutphen · 2007 · p.1518 abstract · Original source)
What the figure shows:Solving a game means working backward from final positions and marking every position as a win or loss. A stone-taking game is solved quickly, but checkers has about 5×10²⁰ positions and took until 2007.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_retrograde_stones.py
041964 · Word Problems
Turning Word Problems into Equations
How did a machine turn “15 percent less than …” into an equation?
In 1964, Daniel Bobrow, a graduate student at MIT, built for his doctoral thesis a program called STUDENT that read and solved algebra word problems written in English. He rewrote some of the word problems from a first-year high-school textbook into English that STUDENT could read, and tried them.
STUDENT does not think about the meaning of a sentence. It replaces fixed phrases with parts of an equation: the English word is becomes an equals sign, and percent less than becomes a multiplication. Press "Go forward one step" in the figure on the right. When it lacked an equation, it filled in known relations such as and told the user the assumptions it had made. A phrase that is not in its rules cannot become an equation, even if it means the same thing.
69.70= (1 − 15/100) × listed price
is
make it an equals sign
percent less than
make it × (1 − p/100)
sum of
make it an addition
Bobrow defined what he meant by saying as this: it , for that range of sentences.
The problems in the model
The example problems were built by this site, in Japanese split into phrases, from the problems in Figure 1 of the thesis (the radio price, two numbers, the jet plane, the age of a building). In Figure 1 of the thesis, STUDENT answers the radio problem with , and for the jet-plane problem it shows that it used a known relation because an equation was missing.
Sources & confidence (1)Turning Word Problems into Equations
3 June 1964Verified in the original
STUDENT, a program that solves algebra word problems written in English (Bobrow, 1964)
In his doctoral thesis, MIT graduate student Daniel Bobrow developed STUDENT, a program that solves algebra word problems written in English (written in LISP and run on an IBM 7094; his advisor was Marvin Minsky). STUDENT builds equations from sentences written in restricted English and solves them. For example, given a problem asking for the list price of a radio that costs $69.70, which is 15 percent less than the list price, it answered . When there were not enough equations, it used general knowledge not stated in the problem, such as (distance = speed × time), and told the user the assumptions it had made. First-year high-school textbooks contain many word problems, and Bobrow tested some of them, rewritten into English STUDENT could handle. Bobrow wrote that it solved most of the problems it could handle as fast as a human or faster. He also used the phrase with a clear definition: it within that range of sentences (an operational definition, in the paper's words).
What we still don’t know:Bobrow used as defined by accepting sentences in the target range and answering questions. Whether a correct answer is the same thing as knowing what it means is a question that is still unsettled even for today's text AI (the fourth boom) (this site's view; it connects to the part about what we still don't know in u13).
What the figure shows:STUDENT solved word problems by replacing fixed phrases with parts of equations, without considering their meaning. A phrase outside its rules cannot be solved, even if it means the same thing.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_student_equations.py
051966 · Conversation
The Trick Behind ELIZA
What did a program that talks with people in text actually “understand”?
In January 1966, Joseph Weizenbaum of MIT published a paper on ELIZA, a program that converses with a computer user in text. ELIZA looks for a keyword in the input sentence. A decomposition rule for that keyword cuts the sentence into parts, and a reassembly rule builds the reply. If there is no keyword, it returns an empty acknowledgment.
Keywords have ranks. In the paper's example, everybody is ranked higher than I. Then ELIZA reacts to everybody and can ask who is meant. Type a sentence into the figure on the right to see the keyword it found and the rule it used. The set of keywords and rules (the script) is data separate from the program, and swapping it changes the role of the conversation partner.
(0 YOU 0 ME) → (WHAT MAKES YOU THINK I 3 YOU)
0
a part that matches any number of words
3
insert the third decomposed part as it is (the paper's example)
At the time of the paper, even the biggest script had about 50 keywords. Besides English, there were scripts in Welsh and German.
What you think you knowThe popular story:ELIZA was built to be an AI that could converse like a person The original source says:At the very start the paper says that , and goes on: .
Sources & confidence (1)The Trick Behind ELIZA
January 1966Verified in the original
ELIZA, a program that converses in text. One aim of its author's paper was to explain away a magic that seems impressive
Joseph Weizenbaum of MIT presented in a paper ELIZA, a program that can converse with a computer in text. It finds keywords in a sentence typed by a person and rearranges them according to rules to produce a reply. The role of the conversation partner can be changed by swapping a , and as of 1966 there were scripts in Welsh and German as well as English. At the start of the paper, the author writes, in effect, that if the inside of the program is explained in plain words the magic crumbles and it is seen to be just a collection of procedures, and that this paper is meant for that purpose.
The popular story:People tend to think “ELIZA was built to be an AI that could converse like a person” The original source says:in the 1966 paper by its creator, he writes at the start that when it is , and that . Explaining away something that seems impressive was one of the aims of the paper (for the story of how, in his 1976 book, he himself warns about AI, see u05).
What the figure shows:ELIZA only finds keywords in a sentence and reshapes them into a fixed reply pattern. It may seem to understand, but revealing the trick shows that it does not examine meaning.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_eliza_ja.py
061964 or 1966 · A Japanese Teaching Material
What Year Is ELIZA From?
What is the “1964” in the timeline the year of?
A table in a teaching material for high-school teachers of “Information II” (an elective informatics course in Japan) from 2020 says . This table has been copied from a 2016 government survey report to a white paper of the Ministry of Internal Affairs and Communications, and then to the teaching material.
Lining up the original sources: Weizenbaum's paper was accepted in September 1965 and appeared in the January 1966 issue. MIT's annual report of the time (covering July 1964 to July 1965) says that development of a system for natural-language conversation between people and computers began during that period. The original program that has been found was the 1965 version.
Press the tags on the right to see what that year is the year of. 1964 is plausible as around when development began, but it is not the year of completion or announcement. The table in the teaching material and the paper are writing about different moments. Once you know what each year is the year of, both numbers can be read correctly.
English-language sources also waver on the year. IEEE's profile of computing pioneers describes him as , and gives 1965 as the year of the journal issue too (the printed issue is 1966).
Tracing the citations
The 2016 report says this section is based on writing by Hitoshi Matsubara and Yutaka Matsuo, but does not give the source of each row of the table. The earliest source that put 1964 in the table is unknown.
The year Weizenbaum came to MIT also splits between 1963 and 1964, depending on the source.
Sources & confidence (1)What Year Is ELIZA From?
January 1966Verified in the originalSources disagreeJapan
Is ELIZA's year 1964 or 1966? — the national teaching material's table says , but the original paper was received in September 1965 and published in the January 1966 issue. Work began sometime between July 1964 and July 1965
The 2020 teaching material for teachers of the high-school course “Information II” (the advanced high-school informatics course) from Japan's Ministry of Education, Culture, Sports, Science and Technology carries the table , reprinted from the Ministry of Internal Affairs and Communications' 2016 White Paper on Information and Communications. There, ELIZA (a dialogue program that answers in text to sentences a person types) is written as “Artificial dialogue system ELIZA developed (1964)” (translated). The source of the white paper's table is the report of a March 2016 survey that the Ministry commissioned from Nomura Research Institute. It has the same table and also a sentence saying ELIZA was (translated). The report says this section is based on Hitoshi Matsubara's and Yutaka Matsuo's book , but it does not give a source for each row of the table. Meanwhile, the paper by Joseph Weizenbaum, who developed it, appeared in the January 1966 issue of Communications of the ACM (pages 36–45), and its last page says it was received in September 1965. The annual report of MIT's research program Project MAC at the time (for July 1964 to July 1965) says, in the Weizenbaum section, that a major effort on a system in which people and machines converse in natural language began and advanced during this period, and that it can now converse on limited topics. The researchers who found ELIZA's original program in an MIT archive also put the development period at about 1964–1966 and say the version they found dates from 1965. 1964 is possible as a year when the work began, but it is not the year of completion or announcement. The table and the paper point to different moments.
The popular story:The national teaching material and the white paper's table say “Artificial dialogue system ELIZA developed (1964)” (translated) The original source says:in the original sources, the paper was received in September 1965 and published in the January 1966 issue. The MIT annual report of the time says the dialogue-system effort began between July 1964 and July 1965. 1964 fits the sources as a year when the work began, but it is not the year of completion or announcement. The English sources also vary on the year: the IEEE's profile of computer pioneers describes Weizenbaum as the and gives the issue of the paper as Jan. 1965 (the actual page says January, 1966).
What we still don’t know:The earliest source that put 1964 in a timeline is not known. The 2016 report says this section is based on Hitoshi Matsubara's (President of the Japanese Society for Artificial Intelligence, according to the report's note) and Yutaka Matsuo's book (2015), but this site has not been able to open either. The month the work began is also unknown (somewhere between July 1964 and July 1965, going by the annual report). What year Weizenbaum himself gave in later works (1976 and so on) has not been checked by this site either.
9 sources and how they were checked
Where sources disagree:ELIZA's year: 1964 (the table in the 2020 Ministry of Education “Information II” teaching material, Figure 4-2-1-5 of the Ministry of Internal Affairs and Communications' FY2016 White Paper on Information and Communications, the table and text of the March 2016 Nomura Research Institute report)/written around 1964–1966 (findingeliza.org of the ELIZA Archaeology Project)/invented in 1965 (the IEEE Computer Society's Computer Pioneers; the same page also gives the paper's issue as Jan. 1965)/the original program found is a 1965 version (the versions page of findingeliza.org)/the effort began between July 1964 and July 1965 (Project MAC Progress Report II)/paper received September 1965, published in the January 1966 issue (the printed paper). The year Weizenbaum came to MIT also differs: 1963 (MIT News 2008) and 1964 (the IEEE biography, a note in Shrager 2024 on arXiv).
What the figure shows:The timeline's 1964 and the paper's 1966 refer to different stages, roughly the start of development and publication, rather than necessarily showing that either date is wrong.
071965 · Chemistry
Build All Candidates, Narrow by Rules
When there are millions of candidate answers, how does an expert narrow them down?
At Stanford University, Joshua Lederberg had been building DENDRAL (1964), a procedure that counts, as a tree, all the possible molecular structures for a given chemical formula. From 1965, Edward Feigenbaum and colleagues turned it into a program that narrows the candidates by building chemists' experience-based knowledge into rules. This was knowledge for reading mass spectrometry data (a method that breaks a molecule apart and weighs the fragments). For one compound, 14,715,813 possible structures were cut to 1,284,792 by mass spectrometry, and to just 1 when another measurement (nuclear magnetic resonance) was also used.
The figure on the right is a small version of this idea. It builds every carbon-only skeleton, predicts the weights of the fragments, and removes the structures that do not match the measured weights.
build → predict → compare (keep only what matches)
fragment weight
a fragment of k carbons is taken to weigh 14k+1 (this site's simple rule; real mass spectrometry is much more complicated)
A 1970 paper wrote:
What you think you knowThe popular story:The first expert system was MYCIN (1972) The original source says:Japan's white paper says so, but DENDRAL, which began in 1965, came earlier as a program that put experts' experience-based knowledge into rules. The developers themselves, in the title of a 1993 paper, call DENDRAL .
Sources & confidence (1)Build All Candidates, Narrow by Rules
1965Verified in the originalSources disagree
DENDRAL, a program with chemical expertise — from general cleverness toward specialist knowledge
Joshua Lederberg, a Stanford University geneticist (a Nobel laureate), had been building DENDRAL, a procedure that uses a tree structure to enumerate the molecular structures possible for a given chemical formula (a 1964 NASA report). Edward Feigenbaum and colleagues, who moved from the University of California, Berkeley to Stanford in 1965, added to it the experience-based knowledge chemists use when reading mass spectrometry data (a method that breaks molecules apart and measures the weights of the fragments). They built that knowledge in as rules of the form , creating Heuristic DENDRAL, which narrows down the candidates. For one compound, more than 14.71 million candidates were reduced to about 1.28 million by the mass spectrometry data, and to one when other measurements were added (Nilsson). In their 1970 paper the three separated the breadth of problems that can be handled (generality), the quality of the answers, and whether they can be produced at a reasonable cost (specialization). They wrote and chose the path of valuing knowledge of a specific field. This idea led to the expert systems of the 1980s (AI with built-in specialist knowledge).
The popular story:Even Japan's white paper says (translated) The original source says:a program that builds in rules based on experts' experience came earlier, with Heuristic DENDRAL, which began in 1965. The developers themselves call DENDRAL in the title of their 1993 paper. In 1970 the idea of specialist knowledge over general cleverness had also been written down. The central idea of the second boom (the 1980s) was born in the middle of the first boom.
5 sources and how they were checked
Where sources disagree:“The first expert system”: the Ministry of Internal Affairs and Communications' FY2016 White Paper on Information and Communications describes MYCIN (1972) as and “said to be the world's first”. The DENDRAL developers, on the other hand, call DENDRAL in the title of their 1993 paper. It depends on where you start counting expert systems.
What the figure shows:Generating all candidates and then using expert knowledge, the measured weights of fragments, to remove those that do not fit sharply reduces a huge pile of possibilities.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_dendral_isomers.py
081965 · Sets
From What Height Is a Person “Tall”?
Can a collection with no clear in-or-out be expressed as a number?
In 1965, Lotfi Zadeh of the University of California, Berkeley, published the paper . Collections such as or the have no clear boundary between in and out. Yet such collections play an important role when people recognize patterns and communicate information. So Zadeh expressed how much something belongs to the collection as a number from 0 to 1 (the degree of membership).
Move the height in the figure on the right. An ordinary set switches suddenly from 0 to 1 at a certain height. A fuzzy set rises smoothly. The paper defined “or” as the larger value, “and” as the smaller value, and “not” as 1 minus the number.
or=Max[μA, μB] and=Min[μA, μB] not=1 − μA
μA
degree of membership in set A (0–1)
The shape of the degree (0 at 160 cm, 1 at 185 cm) is an example chosen by this site. Who decides it, and how, becomes a convention for each use.
His own look back
In 1980, Zadeh wrote that he had the idea in New York in July 1964, and that he knew from the start that the word fuzzy carried a belittling ring and that the idea would be taken as unscientific.
At the first International Joint Conference on Artificial Intelligence in 1969, one of the Japanese papers already used the word . In Japan in the late 1980s it spread through automatic subway operation and home appliances labeled (chapters 6 and 8).
Sources & confidence (1)From What Height Is a Person “Tall”?
June 1965Verified in the original
Zadeh's — expressing collections with blurry edges, like , with numbers
Lotfi Zadeh of the University of California, Berkeley published the paper . A collection with no clear line between being in and out, such as or the , is not an ordinary mathematical set. But Zadeh wrote that such collections play an important role in human thinking, particularly in recognizing patterns and communicating information. He then devised , which express how much an object belongs to the collection as a number from 0 to 1 (the degree of membership). Looking back in 1980, he said the idea came to him in New York in July 1964. He said he knew from the start that the word has a belittling ring and is unscientific, which would cause problems, but no other word fit. In the late 1980s in Japan the idea spread as in automatic subway operation and home appliances (a separate record).
What we still don’t know:In 1980 Zadeh himself wrote that criticisms include that it is and . Who decides the degree of membership, and how, becomes a matter of agreement in each application.
What the figure shows:Even a group with unclear boundaries, such as tall people, can be used in calculations by expressing membership as a degree from 0 to 1. Ordinary sets switch suddenly at a boundary, while fuzzy sets change smoothly.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_fuzzy_tall.py
091960–1966 · Predictions
Can Machines Do Any Work People Can?
Did the “within twenty years” prediction mean that people's jobs would disappear?
In 1965, Herbert Simon wrote in a book that a person can do. In the original 1960 paper he foresaw , but right after that he went on to say the technology .
What Simon used was the economists' idea of . Even if machines are faster than people at every job, machine time is limited. If each takes on what it is relatively good at, the whole produces more. In the figure on the right, even when you raise the machine's speed, under "Divide by what each is good at" the people's work remains.
work handled by people=the work with the smallest ratio of (machine speed ÷ human speed)
comparative advantage
what you are relatively good at compared with others (not speed itself)
What you think you knowThe popular story:Simon predicted that in 20 years machines would take all of people's work The original source says:The original paper separated the forecast of technical capability from how it would be used. Looking at the long run, he thought almost everyone who wanted to work could. In 1966 he argued back that what decides the level of employment is not technology but the workings of the economy.
Worries and answers of the same time
The Triple Revolution memo (1964), signed by 35 people, warned the President that human labor would become less and less needed because of computers and automatic machines. The 1966 report of the national commission wrote that , and held that the cause of the high unemployment was not technology but too little growth in purchasing power.
The speed numbers in the figure are examples chosen by this site.
Sources & confidence (3)Can Machines Do Any Work People Can?
1965Verified in the original
Simon's prediction (1965 book/originally a 1960 paper)
In his 1965 book, Herbert Simon wrote that within 20 years machines would be able to do any work a human can do. In the 1960 paper it was based on, he wrote that in a period much shorter than 25 years machines would have the to replace any human function in organizations. Immediately afterward, however, he went on to say that what is technically possible is separate from how it is actually used. He then assumed that, in the long run, nearly everyone who wants to work can work (full employment), and argued that the division of labor between people and machines is decided by (each side takes on what it is relatively better at).
The popular story:People tend to say “Simon predicted that machines would take all human jobs in 20 years” The original source says:the original paper was a forecast that machines would have . Right after, he writes that even though such capability is predicted, the forecast . And he thought full employment would continue in the long run (comparative advantage). In 1966 he answered a book reviewer that what decides the level of employment is the workings of the economy, not technology.
What happened next:No source was found saying that by 1985 machines could do any human work. Nilsson (2010) says that, even as of 2010, Simon's similar statement from 1957 (that the range of problems machines can handle would become as wide as the human mind) was still far from coming true. Simon himself, on the other hand, thought of technical capability and employment separately and expected that, over the long run, unemployment would not rise (in his 1966 reply he wrote that the unemployment rate was below 4% and falling).
memorandum — warning the President that computers and automatic machines will make more and more human labor unnecessary
Thirty-five people calling themselves sent the memorandum to President Johnson in a letter dated March 22, 1964 (also to the leaders of the Senate and House of Representatives and to the Secretary of Labor). The three revolutions are , formed by the combination of computers and automatic control machines, the weaponry revolution, and the human rights revolution. The memorandum pointed out that cybernation would create nearly unlimited productive capacity while the amount of human labor needed would keep shrinking. It argued that the root cause of excessive unemployment is that the power of machines is growing faster than the power of many people to keep up. It then proposed that, because the link between jobs and income is breaking, every person and family should be guaranteed a sufficient income as a right. Signers included Nobel Peace Prize winner Linus Pauling, the economist Robert Heilbroner, and Gerard Piel, publisher of .
The popular story:People tend to think the worry that “machines (AI) will take people's jobs” first appeared with today's AI The original source says:already in 1964, 35 people warned the President that computers and automation would make human labor less and less necessary, and even put forward a plan to guarantee income as a right, separate from employment. The economist Autor (2015) also writes that worries about unemployment caused by automation have recurred over 200 years.
What happened next:The national commission's report that came out two years later (February 1966, → The U.S. national commission's report T…) concluded that the high unemployment after the Korean War was caused not by accelerating technological progress but by insufficient growth in demand. Unemployment, which was 5.1% in August 1964 when the law was enacted, had fallen to 4.0% when the report came out. Over the long run, the economist Autor (2015) writes that in 20th-century America the employment rate relative to population rose, and no long-term upward trend in unemployment is seen (this long-term part rests on one summary and is weakly confirmed).
What we still don’t know:Autor (2015) writes that computers, AI and robots may replace human labor on a scale never seen before, and that this cannot be judged by past experience alone. Whether the 1964 warning came too early or was wrong has still not been settled in the age of AI.
The (14 members, including chairman Howard Bowen), created by a 1964 law, issued its report to the President and Congress in February 1966. The report first said that the pace of technological change had increased over recent decades but that there had been no sudden break and none was likely in the next ten years (page 1). It also concluded that the cause of high unemployment after the Korean War was not accelerating technological progress but demand (the power to buy things) growing too slowly compared with productivity and the growth in the number of workers (page 9). And it wrote that . The unemployment rate, 5.1% when the law was made, had fallen to 4.0% by the time of the report (page 15). At the same time, the report admitted that technology is a major factor in taking away particular people's jobs. It then recommended public jobs with the government as employer of last resort, a minimum guaranteed household income, and a study of a (pages 40 and 110).
The popular story:It is often summed up briefly (for example in Autor 2015's summary) as The original source says:the report said technology is not the cause of unemployment overall, but admitted that it is a major factor in taking away particular people's jobs. It then went on to recommend a minimum guaranteed household income, a system where the government is the employer of last resort, and a study of a negative income tax. Five of the commission members added a note that the report lacks a sense of urgency (page 6).
What the figure shows:Even if machines become faster than people at every job, machine time is limited. Giving people the jobs they are relatively good at increases total output, so machines being able to do everything does not mean people have no work.
101965 · Predictions
Doubling Every Year for 10 Years
By how many times does something that doubles every year grow in 10 years?
In 1965, Gordon Moore of Fairchild Semiconductor wrote in the article that the number of components on one circuit had been roughly doubling every year (the number at which the cost per component is lowest). He said this rate would continue , and forecast 65,000 components in 1975.
Select “Doubling scale” in the graph on the right, and the line becomes straight. This is a sign of growing at the same rate every year. If you change the doubling time T, the forecast for 1975 changes a lot. The lower graph under “Details: article points, costs and sources” shows how the number of components at the lowest cost per component moves to the right as the years go by.
number of components=64 × 2(year − 1965) ÷ T
T
years to double (about 1 year in the article)
64
2⁶ (the 1965 point)
In 1975, Moore himself looked back and said the number had doubled almost every year until then, and revised his forecast: from now on it might be . The name was given by someone else.
About the points and the curve
The upper points under “Details: article points, costs and sources” are approximate values read by eye from Figure 3 of the 1965 article. The article's text says the lowest cost is at about 50 components in 1965 and about 1,000 five years later. The shape of the lower curve in the same section was drawn by this site.
The Computer History Museum says this forecast became a goal for the semiconductor industry and . The familiar “doubling every 18 months” does not appear in any source this site has opened.
Sources & confidence (1)Doubling Every Year for 10 Years
19 April 1965Verified in the original
Moore's prediction: the number of components on an integrated circuit roughly doubles every year, reaching 65,000 in 1975 (1965) — later called
Gordon Moore, head of research and development at Fairchild Semiconductor, wrote the article in the magazine Electronics. The number of components integrated on one circuit (the number at which the cost per component is lowest) had been roughly doubling every year. He forecast that roughly the same pace would continue for at least ten years and that the number would reach 65,000 in 1975. In the same article he also said that integrated circuits would make possible home computers (at least terminals connected to a central computer), automatic controls for cars, portable communications equipment and more. This forecast later came to be called “Moore's Law”, a yardstick for talking about the growth of computing power, including for AI.
The popular story:People tend to take “Moore's Law” as something that continues at a steady pace like a law of nature The original source says:the 1965 article was a forecast limited to a period, , and its subject was the number of components integrated on one circuit (the number that is cheapest). Ten years later, in 1975, Moore himself eased his forecast to , and the name was given by someone else, Carver Mead (CHM and IEEE).
What happened next:In 1975, Moore himself said in a talk that the complexity of integrated circuits had roughly doubled every year since their appearance. For the 1975 point he placed a 16,384-bit memory device, alongside the calculation that a yearly doubling would give about 65,000 times. He then revised his forecast, saying growth by the end of the 1970s might be more like . The Computer History Museum writes that this prediction became a goal for the semiconductor industry and turned into (a prediction that becomes people's goal and so comes true).
What the figure shows:Doubling every year for 10 years gives about a 1000-fold increase. Even a small change in the doubling time can make the numbers after 10 years differ by dozens of times.
Cost per component and number of components (the cheapest point each year)
111965–1967 · Criticism
Narrowing Down the Moves to Read
Did chess computers read every move they could play?
In 1965, the philosopher Hubert Dreyfus argued in the RAND report that after its first successes, AI was running into a wall bigger than expected. People think using clues at the edge of their field of view and a sense of what matters, he wrote, but when that is translated into the language of computers, it drops out.
In 1967, the chess program Mac Hack VI, built by Richard Greenblatt at MIT, beat Dreyfus in a game. This program does not read all the moves. Using about 50 clues, it gives each move a plausibility score and reads only the high-scoring moves. At the tournament, it narrowed the number of moves read to 15 for the 1st and 2nd moves, 9 for the 3rd and 4th, and 7 beyond that. In the figure on the right, change how it narrows and how deep it looks.
number of possible moves (a made-up number in the figure)
d
depth
What you think you knowThe popular story:Dreyfus predicted that computers could not beat even an amateur at chess, and then lost himself The original source says:The 1965 original sentence described the situation of the time: . He himself answered that (according to Papert's memo).
People and machines together
In the same report, Dreyfus also wrote that a combination in which a person chooses what to focus on and a machine does the detailed reading is strong. In his conclusion, though, he also wrote that for what machines can do.
Mac Hack VI also used alpha-beta pruning, which cuts off the lines after clearly bad moves, and the paper estimates this reduces the computation to about one hundredth. Its tournament rating in April 1967 was 1450.
Sources & confidence (2)Narrowing Down the Moves to Read
December 1965Verified in the original
The philosopher Dreyfus's criticism,
The philosopher Hubert Dreyfus wrote a survey of AI research as a RAND Corporation report (based on what he said at RAND in August 1964). He pointed out that after the early successes, the research was running into a bigger wall than expected. Humans think using cues at the edge of their field of view (the fringe of awareness), the ability to tell what is important from what is not, and tolerance of ambiguity. He argued that these drop out in computer languages. In his conclusion he wrote that it was time to ask where we stand, so as not to be like the alchemists who kept trying to turn lead into gold. He also wrote that a combination of people and computers is strong, in which people decide where to focus and machines handle the detailed calculation.
What we still don’t know:Dreyfus wrote that big progress in areas such as understanding ambiguous situations (the only model available today being the poorly understood human brain). How far today's learning AI has achieved this is judged differently by different people and has not been settled.
The time Dreyfus lost to MIT's chess program (Mac Hack)
Seymour Papert of MIT arranged a game between Hubert Dreyfus and Richard Greenblatt's chess program (Mac Hack VI), and the program won. The newsletter SIGART printed the game record with only one line added, from Dreyfus's 1965 report: . Dreyfus replied that it was not a prediction but a statement of the facts of the time (according to Papert's memo). The program achieved a mid-level amateur result at a 1967 tournament and was made an honorary member of the United States Chess Federation.
The popular story:People tend to say that Dreyfus predicted computers could not beat even amateurs at chess and then lost The original source says:the 1965 original sentence described the situation at the time of writing: there was still no program that could play amateur-level chess, and the world championship (that Simon had predicted) was two years away. Dreyfus answered that (according to Papert's memo). On the other hand, in the conclusion of the same report he also wrote that for what machines can do, .
3 sources and how they were checked
Where sources disagree:Year of the game: 1967 (Nilsson)/Papert gives no year and says only “when the 1967 paper was in press”. The month and day are unknown.
What the figure shows:The chess program did not examine every legal move. Restricting its search to plausible moves reduced the number of positions enormously, allowing it to look further ahead.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_selective_search.py
121966–1972 · Robots
Bump, Fix the Map, Search Again
What should a robot do when it bumps into something it could not see?
From 1966 to 1972, SRI (the Stanford Research Institute) researched a wheeled robot called . It carried a TV camera, a range finder and bump sensors, and a large computer linked by radio built a map of the room from what it saw, planned routes and moved the robot. It shook when it stopped suddenly, so it was called Shakey (the shaky one).
In the working example in a 1970 report, Shakey plans a route and moves along it. When it bumps into something it did not see, it stops, takes a picture again, and plans again. The figure on the right moves automatically. Press “Stop” to pause it and "Keep going" to resume. The dotted objects are not yet on the map. The search method can be switched between breadth-first, which spreads out from nearby places, and depth-first, which goes as far as it can and then backs up.
breadth-first ← queue depth-first ← stack
queue
examine first the place found first (first in, first out)
stack
examine first the place found last (last in, first out)
What you think you knowThe popular story:Life magazine called Shakey the “first electronic person” The original source says:In the article that phrase is quoted as the words of a young researcher who showed the reporter around. In the same issue, the reporter wrote that the tasks he saw . SRI's 1984 summary describes Shakey as a robot with .
Sources & confidence (1)Bump, Fix the Map, Search Again
17 March 1966–1972Verified in the original
SRI's robot — a wheeled robot that sees with a TV camera, makes plans and moves (1966–72)
The Artificial Intelligence Center at the Stanford Research Institute (SRI) studied a wheeled robot, , from 1966 to 1972 (funded by ARPA, the Advanced Research Projects Agency, and others). The robot carried a TV camera, a range-finding device and whisker-like sensors that could tell when it bumped into something. A large computer connected by radio (an SDS-940 in 1969, a PDP-10 and PDP-15 from 1971) built a map (model) of the room from the images, planned routes to destinations and steps for pushing boxes, and made the robot move. Its body shook when it stopped suddenly, so it was called (the shaky one). This research gave rise to the path-finding algorithm A* (a separate record) and the action-planning program STRIPS.
The popular story:It is often said that “Life magazine called Shakey the ‘first electronic person’” The original source says:at the start of the article, that phrase is quoted as a remark by a young SRI researcher who showed the reporter around. In the same article the reporter described its actual performance as something that . SRI's own 1984 summary describes Shakey as a robot with .
What the figure shows:Shakey maps what it sees, plans a route, and moves along it. If it hits something missing from the map, it stops, corrects the map, and plans again.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_shakey_replan.py
131966 · Computer Vision
Separating Objects from the Background
How does a computer find the “objects” in a photograph?
In July 1966, Seymour Papert of MIT wrote a memo planning to have summer workers build . The plan was to separate objects from the background in the images from a scanner, examine the properties of their surfaces, and name the objects it knew. In July it would handle only spheres, bricks and cylinders placed without overlapping (uniform surface color and background). In August, it would extend to cigarette packs and batteries, and then to tools and cups.
The figure on the right is the first step, the mechanism for "separating objects from the background". Points darker than the threshold are treated as objects, and connected points are grouped and counted. If the background is uniform, the count is right. But if the background is uneven, even the pattern becomes an object, and objects that touch merge into one.
object point ⇔ brightness<threshold group=a set of object points connected up, down, left or right
Nilsson (2010), who wrote a book on the history of AI, writes that this plan was much harder than expected, but became the start of continuing vision research at MIT.
What you think you knowThe popular story:MIT tried to have one undergraduate solve “computer vision” in a single summer The original source says:The memo lists about 11 names as people in charge, and plans to handle only spheres, bricks and cylinders on a uniform background in July, widening step by step in August. However, later history books also write that it was much harder than expected.
Sources & confidence (1)Separating Objects from the Background
7 July 1966Verified in the original
MIT's (1966) — what “had undergraduates solve vision in one summer” really meant
Seymour Papert of MIT wrote a plan to be carried out by people working over the summer. The plan was to build a system that, from images from a vidisector (a device that reads images), separates from , examines the properties of surfaces, and identifies the names of known objects. July was to deal only with non-overlapping spheres, bricks and cylinders (with uniform color of the surfaces and background), and August to widen the targets to cigarette packs and batteries, and then to tools and cups, in a step-by-step plan, with the work assigned to about 11 people. Nilsson records that the task was far harder than expected but became the occasion for vision research to continue at MIT.
The popular story:People tend to say “MIT tried to have one undergraduate solve ‘computer vision’ in one summer” The original source says:according to the memo, the plan was for several people working over the summer (about 11 names are listed with assignments) to build . July was to cover only non-overlapping spheres, bricks and cylinders, to be widened step by step in August. However, later history books also write that the project turned out to be far harder than expected.
What the figure shows:An initial way to find objects in a photo is to connect points darker than a threshold and count the resulting groups. It works with a uniform background, but uneven backgrounds or touching objects quickly cause counting errors.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_figure_ground.py
141967 · Japan · How to Learn
Learning Little by Little from Errors
How does a machine learn from data that cannot be cleanly separated?
In 1967, Shun'ichi Amari studied learning methods mathematically in the paper . It is a method in which, each time one example is seen, the weights (how heavily each feature counts) are corrected a little according to the error. The abstract showed that the weights approach the optimum even when the two groups cannot be cleanly separated, and showed the trade-off between speed and accuracy. A 1968 explanation in Japanese calls this the “stochastic descent method”.
In the figure on the right, the points of the two groups overlap. A large learning rate η gets close quickly but keeps shaking forever. A small one settles down but takes a long time. Making η gradually smaller gets the best of both.
w ← w − η ∇L (x, t ; w)
w
weights
η
learning rate (size of the correction)
L
error on that one example
∇L
the direction showing which way to correct to reduce L
Schmidhuber (2022) cites this as probably the first paper to train a multilayer neural network with this method. The 1967 abstract, however, describes a theory of linear classifiers and their extensions.
The same person's cautious outlook
At the start of his 1968 explanation, Amari wrote: (translated). He also wrote that people's ability to recognize is not produced by logic but acquired unconsciously through long years of learning (spread 18 of this chapter).
Sources & confidence (2)Learning Little by Little from Errors
June 1967Verified in the originalSources disagreeJapan
Shun-ichi Amari's (a learning theory that updates weights bit by bit from errors)
Shun-ichi Amari mathematically analyzed a learning method that, each time it sees one data point, updates the weights (how much importance to give each feature) a little according to the error. The abstract of the English paper describes a proof that the weights approach the optimum even for data that cannot be classified cleanly, the trade-off between learning speed and accuracy, and an extension to piecewise-linear (broken-line-like) discrimination. A 1968 Japanese commentary calls this learning method the “stochastic descent method” (translated). Schmidhuber (2022) writes that it is probably the first paper to train a multilayer neural network by stochastic gradient descent (the basic learning method of today's deep learning).
The popular story:Schmidhuber (2022) writes that in 1967 Amari proposed to The original source says:the abstract of the 1967 paper itself is written as a general theory for linear classifiers and piecewise-linear discrimination. Experiments with a 5-layer network are said to be in his 1968 Japanese book and a student's master's thesis (neither has been confirmed).
What we still don’t know:How much a 5-layer network could actually learn (the 1968 book and H. Saito's master's thesis) is not known, because the original sources have not been opened.
6 sources and how they were checked
Where sources disagree:Pages: 299-307 (IEEE bibliographic record)/Schmidhuber's reference list gives “279-307”. Year of the Japanese edition: Schmidhuber writes 1965 (unconfirmed)
At the start of a 1968 explanatory article, Shun-ichi Amari wrote that pattern recognition, which identifies characters and sounds, is extremely difficult, that there is not yet any device with discrimination ability close to that of humans, and that the chance of one appearing in the near future is also slim. He also wrote that people's ability to tell characters and sounds apart is not worked out by logic but picked up unconsciously through many years of learning. What is needed, he said, is not a complete logic that covers everything but a rough, efficient one.
What happened next:No deadline is given for the (translated). As one rough yardstick, there is a paper that examined the 2014 image-recognition competition (1,000 kinds of photos). The error rate of a well-trained person was 5.1%, while the best AI at the time (GoogLeNet) had 6.8%. On a limited task, AI reached accuracy close to a person's (about 46 years later). No source settles when (translated), in the broad sense that includes characters, sounds and medical diagnosis, was achieved.
What the figure shows:Each example adjusts the weights a little according to the error, moving toward the best line even when the data cannot be separated perfectly. Large updates are fast but unstable; gradually shrinking them combines the benefits of speed and stability.
Number of examples seen and error rate (dotted line: the best line)
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_amari_online.py
151968 · Search
Shortest Path: Traveled + Estimated
When searching a maze, which square should be examined first?
In July 1968, Hart, Nilsson and Raphael of SRI published a paper on a theory for finding the minimum-cost path using a clue (a heuristic). According to Nilsson's recollection, for Shakey's path finding the idea arose of adding to an estimate of the to the goal. If the estimate is never larger than the true remaining distance, the shortest path is always found. This search method was named A* (A-star).
Compare three cases in the maze on the right. Without an estimate, it examines everything around the start in order of closeness. A* stretches toward the goal. If you triple the estimate, fewer squares are examined, but the path is no longer the shortest.
f(n)=g(n)+h(n) examine the square with the smallest f first
g(n)
distance from the start to n
h(n)
estimate from n to the goal (straight-line distance, vertical and horizontal)
In this maze, the squares examined were 133, 97 and 52, and the path lengths were 29, 29 and 39. A* is now also used in car-navigation route guidance and in games (Nilsson 2010).
The A* (A-star) pathfinding method — searching by the distance so far plus an estimate to the goal
Peter Hart, Nils Nilsson and Bertram Raphael published the paper . For the problem of finding the lowest-cost route in a graph of connected points, they built a theory that brings in hints from the problem itself (a heuristic: a shortcut based on experience or estimates) and showed that one search method is optimal (from the paper's abstract). According to Nilsson's recollection, for the path search of the robot “Shakey,” they took the idea of using the to the goal as the estimate (Nilsson) and added to it the idea of (Raphael). Hart then showed that as long as the estimate never exceeds the actual remaining distance, the shortest route is always found. This search method was named A*, and today it is used in places such as car navigation route guidance and games (Nilsson 2010).
SRI's robot — a wheeled robot that sees with a TV camera, makes plans and moves (1966–72)
The Artificial Intelligence Center at the Stanford Research Institute (SRI) studied a wheeled robot, , from 1966 to 1972 (funded by ARPA, the Advanced Research Projects Agency, and others). The robot carried a TV camera, a range-finding device and whisker-like sensors that could tell when it bumped into something. A large computer connected by radio (an SDS-940 in 1969, a PDP-10 and PDP-15 from 1971) built a map (model) of the room from the images, planned routes to destinations and steps for pushing boxes, and made the robot move. Its body shook when it stopped suddenly, so it was called (the shaky one). This research gave rise to the path-finding algorithm A* (a separate record) and the action-planning program STRIPS.
The popular story:It is often said that “Life magazine called Shakey the ‘first electronic person’” The original source says:at the start of the article, that phrase is quoted as a remark by a young SRI researcher who showed the reporter around. In the same article the reporter described its actual performance as something that . SRI's own 1984 summary describes Shakey as a robot with .
What the figure shows:A* checks squares in order of the smallest sum of the distance traveled so far and the estimated distance to the goal. The estimate steers the search toward the goal, but making it too large gives a faster route that may not be the shortest.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_astar_maze.py
161968 · Japan
0 if Negative, As Is if Positive
If you stack neurons that only add, does what they can do increase, however many layers you use?
In 1968, Kunihiko Fukushima of the NHK Science and Technical Research Laboratories used experiments on the eyes and brains of cats and monkeys as a model, and simulated a layered network on a computer. It was a first step toward a machine that reads handwritten characters. As a figure moves through the layers, it is broken into isolated points and line segments of each orientation, and then the end points of the segments are found.
The neuron he used outputs its input sum as it is if positive, and 0 if negative. This is the same form as the ReLU widely used in deep learning today. He also wrote that if everything is built from neurons that only add, the result is the same as two layers however many layers you stack. In the figure on the right, turn off "0 if negative (ReLU)", and negative values in places with no line flow into the next layer, so the ends are not found correctly.
output=max(0, Σ weight × input)
max(0, u)
0 if u is negative, u as it is if positive (today's name: ReLU)
The weights in the figure are simplified by this site (they are not the weights of the 1968 network itself). The neuron that finds ends subtracts neighbors along a line.
What you think you knowThe popular story:ReLU was an invention of 2010s deep learning The original source says:An activation function of the same form (0 if negative, as is if positive) was already used as an imitation of how the eye works in Fukushima's 1968 explanation in Japanese and his 1969 paper in English.
Sources & confidence (1)0 if Negative, As Is if Positive
November 1968–1969Verified in the originalSources disagreeJapan
Kunihiko Fukushima (NHK), (translated) — an element that gives 0 if negative and passes the value through if positive
Kunihiko Fukushima of the NHK Broadcasting Science Research Laboratories took experiments on how the eyes and brains of cats and monkeys work (by Hubel and Wiesel) as his model and, as a first step toward a machine that reads handwritten characters, recreated a multilayer circuit network on a computer. As the input shape moves through the layers, it is broken down into points, then lines in each direction, then the ends of lines. The element he used outputs the input total as it is if that total is positive, and outputs 0 if it is negative. This has the same form as the ReLU that is widely used in today's deep learning. He also wrote that if everything is built from elements that only add, the network is equivalent to two layers no matter how many layers are stacked.
The popular story:People tend to think “ReLU was an invention of 2010s deep learning” The original source says:an element of the same form (0 if negative, the value itself if positive) was already used as a model of how vision works in a 1968 Japanese-language explanatory article by Kunihiko Fukushima of NHK and in a 1969 English-language paper
3 sources and how they were checked
Where sources disagree:The year given as “the first ReLU”: 1969 (Schmidhuber, an English-language paper)/an element of the same form already appears in the 1968 Japanese-language article (this site's check)
What the figure shows:In 1968, Fukushima stacked layers of neurons that return 0 for negative inputs and leave positive inputs unchanged to find lines and edges. Without the negative-to-0 step, even stacked layers cannot find the edges correctly.
Sum of inputs and the neuron's output
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_relu_layers.py
171968–1969 · Japan
Reading Handwritten Postal Codes
How did a machine deal with characters it could not read?
On July 1, 1968, the day the postal code system began, the automatic postal-code reader and sorter that Toshiba had developed under the guidance of the Ministry of Posts and Telecommunications was shown at the Tokyo Central Post Office. Reading and sorting freely handwritten digits is said to have been . It analyzed about 300,000 handwritten digits collected from across the country, erased the red frames with an optical filter, and read the first 3 digits at a speed of about 6 items per second.
According to a 1969 paper by the developers, it examined each character in horizontal bands, sorted the features into 19 types, and compared their top-to-bottom sequence with the sequences of each digit's variations stored in a dictionary. A digit with 2 or more mismatches is removed. If one remains, that is the answer; if none or 2 or more remain, it is a “reject”, and the item was passed to a person.
The characters and features (5 types) in the figure are examples made by this site. At the first International Joint Conference on Artificial Intelligence in 1969, 3 of the 5 papers from Japan were studies of character recognition.
Sources & confidence (2)Reading Handwritten Postal Codes
1 July 1968Verified in the originalSources disagreeJapan
Automatic reader-sorter for handwritten postal codes (Toshiba and the Ministry of Posts, 1968)
On July 1, 1968, the day the postal code system began, the automatic postal code reader-sorter (TR-3 and TR-4), which Toshiba had developed under the guidance of the Ministry of Posts, was shown at the Tokyo Central Post Office. It is said to be the world's first machine to recognize digits that people had written freely by hand and sort mail with them. The developers analyzed about 300,000 handwritten digits collected from across the country to find which quirks in letter shapes appear most easily, and used an optical filter to erase the red frame so that only the strokes of the digits were extracted. It read the first 3 digits at a speed of about 6 items per second, and characters it could not recognize were marked as “reject” and passed to people.
What we still don’t know:The statement that it was (translated) comes from the accounts of Japanese academic societies, museums and companies. We have not opened any source that compares it with machines from the same period in other countries.
4 sources and how they were checked
Where sources disagree:When development began: 1965 (Showa 40) (Postal Museum, Toshiba 2022)/from 1966 (Showa 41) (a 1969 paper by the developers). Processing speed: 20,000 items per hour (Information Processing Society of Japan)/22,000 (Postal Museum)/about 6 per second (the 1969 paper; about 21,600 per hour when converted).
The first international AI conference (IJCAI-69, Washington) — five talks from Japan as well
The first international conference where artificial intelligence researchers gathered across national borders, the (IJCAI), was held for 3 days in Washington, D.C., in the United States. The proceedings' table of contents lists 63 papers, divided into sections on pattern recognition (identifying shapes), problem solving by search, theorem proving, question-answering systems, self-organizing systems, robots, language, and more. Nilsson of SRI presented a vehicle-type robot carrying a TV camera (later Shakey; see another record). From Japan, five papers were included, by researchers at the Institute of Industrial Science of the University of Tokyo, the Research Institute of Electrical Communication of Tohoku University, the central research laboratories of NEC (Nippon Electric Company) and Tokyo Shibaura Electric (Toshiba), and Chuo University; three of them were studies on recognizing characters (such as handwritten digits). The conference then continued with London in 1971, Stanford in 1973 and so on, and was held in Tokyo in 1979 (another record).
What the figure shows:The postal code reader turned each character into a sequence of features from horizontal strips and compared it with a dictionary; if it could not settle on 1 answer, it marked it “reject” and passed it to a person. It chose human review over making a mistake.
Amount of noise and the reading results
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_postal_digits.py
181967–1970 · Predictions
A Generation, 3–8 Years, Unlikely Soon
Where were the forecasts of those years actually written?
In 1967, Marvin Minsky wrote in a book: . In 1970, an article by a Life magazine reporter said that Minsky had told him would be in a machine, and also reported another researcher saying . The 1968 film features HAL 9000, a computer that talks in a human voice.
In the same year, 1968, Japan's Shun'ichi Amari wrote: (translated). The figure on the right connects, with lines, the year each prediction was made to its deadline. The color and the shape of the tags show where the words came from. Move the "current year" to see what happened to the predictions whose deadlines had come.
The book does not say how many years is. A historian's book says Minsky later wrote a rebuttal denying the quotation, but I could not confirm whether what he denied was the statement.
What you think you knowThe popular story:Minsky stated flatly in 1970 that a machine with human-level intelligence would come in 3–8 years The original source says:The words were originally in a magazine reporter's article (Life, issue of November 20, 1970, page 58D). The wording in his own book (1967) was (the book's pages could not be seen; this was confirmed through 2 books that cite it).
The film and the researcher
MIT's 1997 news item describes Minsky as a science expert on the film set. But the person credited in the film as science consultant was someone else (Ordway).
According to a book on the history of AI (Nilsson 2010), Dreyfus writes that Minsky said in the film's press materials that within 30 years a machine as intelligent as a person would exist (this is secondhand, and the press material itself was not seen).
Sources & confidence (5)A Generation, 3–8 Years, Unlikely Soon
1967Confirmed in summaries
Minsky's prediction (1967 book):
Marvin Minsky made a prediction on page 2 of his book . He wrote that within a generation few areas of the intellect would remain out of reach of machines, and that the problems of creating would be nearly solved.
What happened next:Nilsson (2010) writes that even as of 2000, AI surpassed people in many intellectual tasks but was still far from them in most respects. There is no criterion for judging at what point it could be said that the problems were , so this has not been settled.
What we still don’t know:The book does not say how many years means.
Words that spread as Minsky's prediction (Life magazine, 1970):
Brad Darrach, a reporter for Life magazine, covered SRI's robot . In his article the reporter wrote that Minsky said the following. In 3 to 8 years there will be a machine with the general intelligence of an average person. After that, the machine will teach itself and rapidly rise above the level of a genius. In the same paragraph, the reporter writes that other researchers said , but agreed that such a machine would be built.
The popular story:It is often quoted that in 1970 Minsky flatly declared “a machine with human-level intelligence in 3 to 8 years” The original source says:the words come from a magazine reporter's article. According to the historian McCorduck's book, Minsky wrote a long rebuttal denying the quotations attributed to him. However, we could not confirm whether the statements he denied included the 3 to 8 years. In Frank Rose's 1985 book (as reported by Quote Investigator), what he denied was the line in the same article about the machines deciding to .
What happened next:We found no record that, by the deadline given in the article (1973–1978), a machine with the general intelligence of an average person had been built. Nilsson (2010) writes that even as of 2000, AI surpassed people in many intellectual tasks but was still far from them in most respects.
What we still don’t know:We could not find the text of Minsky's rebuttal itself, which is said to have been handed out to researchers. So we do not know whether he denied the words .
At the start of a 1968 explanatory article, Shun-ichi Amari wrote that pattern recognition, which identifies characters and sounds, is extremely difficult, that there is not yet any device with discrimination ability close to that of humans, and that the chance of one appearing in the near future is also slim. He also wrote that people's ability to tell characters and sounds apart is not worked out by logic but picked up unconsciously through many years of learning. What is needed, he said, is not a complete logic that covers everything but a rough, efficient one.
What happened next:No deadline is given for the (translated). As one rough yardstick, there is a paper that examined the 2014 image-recognition competition (1,000 kinds of photos). The error rate of a well-trained person was 5.1%, while the best AI at the time (GoogLeNet) had 6.8%. On a limited task, AI reached accuracy close to a person's (about 46 years later). No source settles when (translated), in the broad sense that includes characters, sounds and medical diagnosis, was achieved.
Minsky's forecast in the press materials for the film (1968) (cited secondhand)
According to Nilsson (2010), Dreyfus wrote the following. In 1968, in the press materials for the film (directed by Kubrick), Minsky said that within 30 years there should be machines with intelligence about equal to a human's.
What we still don’t know:We could not confirm in the original sources how Minsky was involved in making the film (whether he gave advice). We have not seen the press materials themselves.
2 April 1968Confirmed in summariesSources disagree
The film and HAL 9000, the computer that talks and thinks — Minsky was on the set
The film “2001: A Space Odyssey,” which director Stanley Kubrick wrote with the author Arthur C. Clarke, was released (distributed by MGM). HAL 9000, the computer on the spaceship Discovery bound for Jupiter, was shown as a machine that speaks in a human voice (the voice was finally done by Douglas Rain). In the story it predicts the failure of an antenna part but is wrong, and then acts in a way that kills crew members (from the AFI Catalog synopsis). In a 1997 news item, MIT introduced its professor Marvin Minsky as “2001: A Space Odyssey,” and in 1998 the MIT Press included a chapter titled in a book. On the other hand, the person credited as in the film's record (AFI Catalog) is Frederick Ordway, and the designs of the future, such as the computers, were offered free of charge by companies such as IBM.
What we still don’t know:From the sources gathered for this site, we cannot tell how far Minsky was involved in how HAL was portrayed (the text of the interview is behind a paywall). Nilsson (2010, p.163) cites Dreyfus and writes that Minsky said, in the press materials for the 1968 film, that within 30 years there should be machines with intelligence about equal to a human's (record → Minsky's forecast in the press material…, weakly confirmed).
4 sources and how they were checked
Where sources disagree:Minsky's role: MIT (1997) says “scientific expert on the set,” and the MIT Press book (1998) uses “Scientist on the Set” in a chapter title. But the science adviser listed in the AFI Catalog credits is Ordway, and Minsky's name does not appear among the sources we looked at. We have not checked what he advised on, because we could not read the text of the interview.
What the figure shows:Predictions from the same period about how many years something would take came from different sources: the person's own book, a reporter's article, or a secondhand quotation. What happened by each deadline is viewed alongside its source.
191971 · Language
In a World of Blocks, Conversation Works
If you narrow the range, does the computer “understand” language?
In January 1971, Terry Winograd of MIT presented in his doctoral thesis a program (later called SHRDLU) that answers questions and carries out commands in English conversation. Its subject was a world in which a simple robot with a hand and an eye moves blocks and pyramids on a table. To it first moved the block on top out of the way, picked it up, and answered To it asked back because there were three in the scene.
Press “Give a command” in the figure on the right. It narrows the objects by words for shape, color and size: if exactly one is left, it acts; if two or more, it asks back; if none, it says so. The thesis's vocabulary was 200 words, and one sentence took 5 to 20 seconds.
1 match → act 2 or more → ask back 0 → answer that none exists
matching objects
objects whose shape, color and size fit the words of the command
What you think you knowThe popular story:SHRDLU moved real blocks with a robot arm and understood English The original source says:In the thesis, the robot and the blocks were a simulation on a screen. The scene data was given from the start, assuming , and the conversation was done by typing text. In 2004 he himself wrote that there is a fundamental gap between how SHRDLU worked and how the brain works (according to Nilsson).
Sources & confidence (1)In a World of Blocks, Conversation Works
January 1971Verified in the original
SHRDLU, which talks in English about a world of blocks (Winograd) — the robot and the blocks were inside the screen
Terry Winograd, a graduate student at MIT, developed in his doctoral thesis a program that can hold a dialogue in English (later called SHRDLU). It can answer questions, carry out commands and take in new information. Its subject is a world in which a simple robot with a hand and an eye moves toy blocks and pyramids on a table. When he typed the program moved away the green block on top, picked the red one up and answered To the instruction it asked back because there were three. It could also decide that referred to something that had come up earlier in the conversation, and explain the reasons for its own actions. However, the robot and the blocks were simulated on a computer screen. The scene data was supplied from the start, assuming ; the dialogue was done by typing text at a terminal. The vocabulary was 200 words, and it took 5 to 20 seconds to process one sentence.
The popular story:People tend to think “SHRDLU moved real blocks with a robot arm and understood English” The original source says:in the thesis, the robot and the blocks are simulated on the screen. The scene data was supplied from the start, assuming ; the dialogue was done by typing text (speech was not tried). Later (2004) Winograd himself wrote that there is a fundamental gap between how SHRDLU works and how the human brain works (Nilsson).
What we still don’t know:Winograd's words from 2004 (according to Nilsson): he thought that the gap between a program like SHRDLU and the brain had not been closed much even by the research of the time. How far today's text-writing AI has crossed this gap is a matter on which evaluations differ (this site's view).
What the figure shows:In a limited world such as a set of blocks, words can narrow down an object, allowing action when there is 1 match and a follow-up question when there are 2 or more. But words outside that world are unknown.
This is the same mechanism as "Try it", written as Python code. Press ▶ Run and the result appears below. Change the numbers and run it again.
u04_blocks_world.py
Closing
What We Still Don't Know in This Chapter
Whether Minsky himself denied the “3–8 years” statement is not known, because the rebuttal text itself could not be found. What span of years refers to is not written in the book either. The source that first put ELIZA's “1964” into the teaching-material table is also still unknown.
Dreyfus wrote that big advances, such as understanding ambiguous scenes, . How far that ability has been achieved by today's learning AI is judged differently by different people. Whether getting answers right is the same as understanding the meaning is also a question that has run on since the days of STUDENT and SHRDLU.
The AI of this chapter ran on rules, estimates and templates that people had thought up and written in. It could do astonishing things within a narrow range, but did not work outside it. In Japan around the same time, seeds that connect to today's deep learning were also growing, such as Amari's learning method that corrects weights little by little, and Fukushima's layered network of “0 if negative” neurons. The next chapter looks at the era of the , when expectations went unmet, and the seeds that grew during it.