Last night was the thrilling conclusion to the man-machine matchup on Jeopardy!, pitting two of the best contestants ever against IBM’s contestant, Watson. As soon as I saw that one category was “One Buck or Less,” and another had a theme of keys on a keyboard, I thought, “Now this is a matchup!”
Both were categories that dealt with “common sense”—the kind of thing that nobody bothers to write down, because everybody knows it. Had Watson managed to figure out, somewhere in its trolling through piles of information, what the names of the keys on a keyboard are? It’s the kind of thing you don’t point out on your webpage—“Look down at your fingers, right now! Let me now explain in excruciating detail what you can see in front of you!” Similarly, Watson has probably never had the experience of going to the local convenience store, and any items that cost less than a dollar would be generally not worth shipping via the internet. So it’s not clear that this information appeared anywhere that Watson could have “read” in its preparation; and even if it did, it might not have been marked as Important in the way that 19th century novelists, say, are Important.
The funny thing about AI is that it makes you appreciate just how amazing we really are as a species. The things we think are “hard” or “smart” are often computationally very tractable, such as thinking logically instead of intuitively, or remembering specific facts, or looking ahead several moves in a chess game. They’re hard because they’re not exactly what we evolved to do; and in some cases, they also require a fair amount of practice. But when it comes to everyday life—recognizing objects, understanding a joke, telling a story—we’re amazing. Unbeatable. Show me the worst student in my class, and I’ll still show you a fantastically complex, marvelous individual, performing feats of recognition and planning and practical thinking every day that trump any machine. The final game wasn’t quite as biased against machines as one webcomic artist predicted, but it gave Watson a run for its money.
So what did Watson do? It basically flubbed those categories, as predicted; and it even fell behind on Actors and Directors, as the humans realized that they needed to just go for it and buzz in before knowing the answer. “Know Thyself” had been a key component of Watson’s success all along, being able to analyze not just what the most probable answer, but its confidence in that answer, to decide whether to buzz in. But the humans did one better—buzzing in before they had any answer or confidence in it, believing through past experience that they would get the answer. Of course, it’s possible Watson would have done the same thing, had it been behind, since acting with more risk makes more sense when you need to catch up. Still, to all those wags who complained that it was just a who-hits-the-buzzer-fastest contest, I say: What a piece of work is man! (And in fact, one IBM researcher has reported that Jeopardy! winners buzz in within zero to a hundred milliseconds of being allowed to. Human and machine alike must anticipate the end of the question.)
Watson caught up on the other categories, though, again displaying fantastic skill at understanding questions and retrieving answers. As soon as I saw the Final Jeopardy question, I knew it would have the thing locked down:
William Wilkinson’s “An account of the principalities of Wallachia and Modavia” inspired this author’s most famous novel
Watson is all over this kind of thing—just look at all those unique proper nouns! There was no question that Watson would ace it—the question was whether Watson knew it knew this stuff. Its bets, which had seemed decidedly inhuman because they were the result of optimizing various equations, had appeared erratic over the past three nights; had it bet enough?
It bet a ton—over $17K—as it should for a very proper-noun-heavy category like “19th century novelists.” And this itself was a big triumph for the night—Watson showed that when push comes to shove, it knows what it knows. And on the whole, Watson’s assessment of its own answers was really consistent over each night, only being incorrectly confident a few times. It won the game, but not before Jeopardy! champ Ken Jennings had a final quip on his Final Jeopardy screen—“I for one welcome our new computer overlords.”
The reaction to Watson’s win tends to be divided between two extremes. One is Ken Jennings’ reaction—“C’mon, IBM! You invented SkyNet! Own it!” It’s a funny recurring theme, this idea that machines will take over the world if we let them learn for themselves, except that we ultimately have control over Watson’s goals, sensors, and actuators. So it would be a kind of funny story in which an AI decided that the most effective way it could answer a question would be to take over the world first, and then it managed to take over the world by just subtly answering questions in a way that makes us want to cede control of the world to it. Humanity: don’t do this. Watson itself doesn’t really have that ability to think outside the box in terms of its own actions—it just really wants to answer questions—so the worst a system like Watson could do would be to inspire lazy thinking. In which case, it’s no worse than just about any commentator or blogger (hi).
Another extreme is the dismissive one, an example being this Washington Post article asking whether trivia is now irrelevant. It’s just a database, or being fast with the buzzer, or trivia, claims the skeptic. Alan Turing, in his famous essay in which he argued that machines will one day be able to think, wrote, “Usually if one maintains that a machine can do [something impressive], and describes the kind of method that the machine could use, one will not make much of an impression. It is thought that the method (whatever it may be, for it must be mechanical) is really rather base.” This is our perpetual curse in AI… as soon as we achieve something, well, I guess it must not be all that special, right? (Actually, this was the conclusion I came to when I got a robot to learn to recognize its own motion in a mirror, but that’s another story.)
The truth is somewhere in between, but closer to the spectacular. What Watson represents is a triumph in information retrieval, in that it has the ability to not just provide relevant sources, as Google does, but answers. The real winners last night were the IBM team, and AI researchers in general. I really wanted to see the team on stage hugging each other last night, the way the contestants on Wheel of Fortune had in the time slot previous. Alas, all we got was a brief flash of the many names that contributed to the system’s success. Like a corporation, Watson ends up being the personification of the many individuals that made it happen. But since Watson was produced by a corporation, we also have a pretty good chance of seeing the technology come to market.
Put yourself in Alex Trebek’s shoes—ask Watson anything at all, and with amazing accuracy, Watson will give you the answer. Isn’t that neat? The Library of Alexandria had nothing on this.
I, for one, welcome our new mastery over the world’s knowledge.
Kevin Gold is an Assistant Professor in the Department of Interactive Games and Media at RIT. He received his Ph.D. in Computer Science from Yale University in 2008, and his B.A. from Harvard in 2001. When he is not thinking up new ideas for his research, he enjoys reading really good novels, playing geeky games, listening to funny, clever music, and reading the webcomics xkcd and Dresden Codak.
