ImageNet helped trigger the modern AI breakthrough

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 9:47

   so that we can empower the machine to read or to see. A group of us, computer vision scientists, were struggling with these algorithms. And I was a very young faculty, first-year faculty, 2006, at Princeton. And my students and I are looking at these algorithms and how little data were fed into these algorithms to learn. So I turned to cognitive neuroscience literature, namely vision literature. I started to study how much humans learn, how much humans can see.

2. 10:30

   And the numbers were incredible. Humans were, by age six, can learn tens of thousands of different object categories. And the exposure to visual world is also massive, right? Babies can see most of the time the moment they're born. So they're inundated with this big data. So we conjectured that the lack of data was a huge part of the reason that's the lack of progress in AI. So we took a departure from everybody else who are really focusing only on algorithm and said that we need data.

3. 11:11

   We need data to drive these algorithms. So long story short, we led this ImageNet project that collected the first ever internet scale large data set for the field of artificial intelligence, but really through the field of vision, because ImageNet is a collection of 15 million images. And the goal of ImageNet was to drive machines to recognize everyday objects, you know, microphones, cups, chairs.

4. 11:46

   And that work converged with the advances in neural network algorithm as well as in GPU computing. And by 2012, that work, that convergence of the three elements of modern AI became the defining moment of what modern AI is. I recall somewhere around 2012, it seems, there was this debate at this vision course at Cold Spring Harbor that was held every other summer. Like, could a computer learn to recognize specific faces as well as humans?

5. 12:27

   Now, I think most people would say computers are... actually much better at it than humans are, even though you have these super, super recognizer people who are exceptional at this. Could you tell us how is it that this technology went from a state basically where it would confuse you and maybe a cousin or even someone that looks somewhat like you, or to the point where now it is exquisitely precise?

6. 12:56

   How do we get here? I want to definitely double, triple click on the convergence of this technology. I think around the second decade of 21st century, so like you said, around 2012, the huge convergence was the capability of GPU computing, which basically accelerated or parallelized computing so that you can have more flops going through algorithms, right? You need that speed. Then you also have, after many decades of research, neural network algorithm is getting more mature.

7. 13:42

   You know, starting, as we said, 1950s, people start to create these very simple algorithms that behave similarly to neurons, but much simpler. Neurons, as you know, are very complex. But here the idea is that you have one unit of node that takes some input and outputs another input. And within it, it's just a function, a very simple function. So you stack them together. That's what neural network is.

8. 14:11

   But by the time it's in the, after, you know, around 2010-ish, the maturity of these algorithms have gotten to a level that it's becoming really good. But also, last but not the least, the recognition of big data. Internet definitely fueled that. It made data more available. But the reckoning moment of, wow, big data needs to be part of that equation. We need to use big data to drive this algorithm, to learn these patterns.

9. 14:48

   So this convergence of these three things really set off the revolution of AI. The specific moment is also worth mentioning, because you mentioned face recognition, is this ImageNet challenge my lab put forward. That starting 2010, after we collected this humongous data set, at that point, GPU was not yet mature. And we put out a public challenge for the research community for multiple years in a role and invited people to solve this major computer vision problem called object recognition.

10. 15:32

    The task was very easy. We have a data set of a thousand different categories of objects, and this data set is more than a million images large. It's what we call the testing data set. And the task for the algorithm is, I'll show you a picture. You have to name the main objects inside. And if you guess right, you get a point. If you guess wrong, you don't get a point. So that image, that challenge, we later, a couple of years later, benchmarked human performance by a very smart graduate student at Stanford.

11. 16:13

    And that was roughly 4%. So random chance will be 1 over 1,000, right? So 4% for humans is not that bad. The first few years, machines were not as good as humans. The turning point was 2012, the convergence of neural network, image net, data set, and GPU. Even that year, even though the error rate was cut to, By the way, the human performance error rate was 4%. Sorry, I need to correct that. The error rate was cut down to the teens.

12. 16:54

    It wasn't where human performance was. So this is looking at images and assigning a- One out of a thousand labels. Got it. Yeah. But 2012 was so momentous that year because the error rate from previous algorithm dropped a lot by this neural network algorithm. And we know in the research community, when something this drastic happens, it means an inflection point. But it still took another three years, I remember.

13. 17:30

    2012, 2016, for the algorithm to beat humans in naming 1,000 objects.