Rural internet scarcity led Gomez into hands-on AI research

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 8:23

   journey really starts there yeah I grew up in the middle of the woods in Canada without like any internet at some point we got like a satellite dish because before that it was a dial-up but this was like pre-starlink and so it was like these like geostationary like if there's a cloud because it has to be like super high up if there's a cloud you're you're screwed like goodbye internet and i think that scarcity from technology like it was just so hard to use my computer and like i couldn't watch youtube it was impossible i could like watch pixel load pixels load one by one on an image that scarcity made it um super appealing to me it was like just out of reach and like you know my friends were gaming online and like doing all this cool cool stuff and when i went home you know we had a computer in the house of course but it wasn't the the experience everyone

2. 9:19

   else was getting so that made me deeply appreciate computers and be very excited about them invested in them I was always trying to make my computer faster at home or like let me access the things that other people were accessing through my my limited was it working were you hacking a faster computer i mean i mean it got me into like web dev and that type of i wanted to understand why this fucking web page wouldn't load faster and so you know i just inspect element blah blah but yeah that's what got me into coding like first it was just like going through websites and trying to understand all this text on my screen and then i took a course because i was interested in that then i started a little small business in my tiny little town of brighton Ontario building websites for the little businesses like there was a really nice

3. 10:09

   lady who had a knitting store and you know yarn and patterns and stuff and so I helped her build like an online catalog for her customers and You know, she's been an incredible customer of mine, but I started that little business and then I just decided this is what I want to do. So I went into computer science at U of T. That was my local university, right? Like Toronto was the nearest big city and I just happened to go to university at like the epicenter of deep learning.

4. 10:43

   Like Jeff Hinton was there, the guy who won the Nobel Prize and the Turing Award for his work creating deep learning, which is like the thing that worked. It was like the breakthrough system that let us reach where we are today. And were you immediately like, this is an area that I want to throw myself into or was it more grounded? I was obsessed. I, like, I just remember every waking minute of my day thinking about it.

5. 11:15

   I would walk around with, like, a stack of papers. And initially, to read one paper in the field would take me, like, a month, actually. Because I would, like, be going through and I'd run into, like, a word that I don't understand. And then I would have to go start the process of, you know, researching what the hell is that? You know, like, what's Bayesian or, you know, what's a Gaussian? You didn't have LLMs to short cut for you.

6. 11:39

   To ask, no, no. It's just Wikipedia pages and research papers. And then gradually I just got more and more familiar and comfortable with the material. And I went to a startup in Vancouver where I worked on, it's called polyphonic transcription, which is like listening to music and transcribing music into sheet music. So you listen to someone playing and you write down the notes that you're hearing.

7. 12:06

   super difficult problem at the time but we used neural networks to approach it and and got some pretty good results and i had to this was before yes before jacks before tensorflow before pytorch before any of these frameworks really existed and i had to write kernels in metal which was like apple's gpu language for forward and backward propagation so i just had to learn everything at like the lowest level imaginable you're like you know basically writing machine code not really but you know just super low level stupidly low level and then i came back to toronto and i messaged uh hinton and i was just like hey i have this idea this activation function that you've made i think there's a better one and he actually responded uh and then he sent me in the direction of uh roger gross uh and others

8. 13:01

   inside the uft ml group and from there started doing work with them eventually ended up at google uh down here in mountain view and was on the transformer paper and the rest is kind of history Then I went off to Oxford.