AI can improve doctors when their expertise complements its data

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:39:06

   for his research i said oh wow how do you use it and he said we use it as a an idea pump what do you mean idea pump well we you know we give it all of the experiments that we've done and uh and we have you know the the literature it's access to the literature and so forth and we ask it for ideas for new experiments oh i love it i love it i was on a plane where i sat next to a guy that worked at works at Google and he he's one of the main people there in terms of voice to to text and text to voice software and he showed me something I'll provide a link to it because it's another one of these open resource things and I'm not super techie I'm not like the I don't get an F in technology I don't get an A plus I'm kind of in the middle so I think I'm pretty representative of the average listener for this podcast presumably what he showed me is that you

2. 1:39:59

   You can take. You open up this website and you can take PDFs or you take URLs, so websites, website addresses, and you just place them in the margin. You literally just drag and drop them there. And then you can ask questions and the AI will generate answers that are based on the content of whatever you put into this margin, those PDFs, those websites. And the cool thing is it references them so you know which article it came from.

3. 1:40:31

   And then you can start asking it more sophisticated questions. Like in the two examples of the effects of a drug, one being very strong and one being very weak, which of these papers do you think is more rigorous based on subject number but also kind of the strength of the findings? You know, a pretty vague thing. Strength of findings is pretty vague, right? Anyone that argues those are weak findings, those aren't enough subjects. Well, we know a hell of a lot about human memory from one patient, HM. So strength of findings when people is a subjective thing.

4. 1:41:07

   Right. You really have to be an expert in a field to understand strength of findings and even that. And what's amazing is it starts giving back answers like, well, if you're concerned about number of subjects, this paper, but that's a pretty obvious one, which one had more subjects. But it can start critiquing these statistics that they used in these papers in very sophisticated ways and explain back to you why certain papers may not be interesting and others are more interesting.

5. 1:41:36

   And it starts to weight the evidence. Oh, my God. And then you say, well, with that weighted evidence, can you hypothesize what would happen if – and so I've done a little bit of this where it starts trying to predict the future based on 10 papers that you gave it five minutes ago. Amazing. I don't think any professor could do that, except in their very specific area of interest. And if they were already familiar with the papers, and it would take them many hours, if not days, to read all those papers in detail. And they might not actually come up with the same answers, right? Right. Yeah. So actually, this is something that is happening in medicine, by the way, for doctors who are using AI as an assistant, this is really interesting.

6. 1:42:23

   So, and this is dermatology, it was a paper in Nature, you know, skin lesions, there's several, 2,000 skin lesions, and some of them are, you know, cancerous and others are benign. And so, in any case, they tested the expert doctors, and then they tested an AI, and they were both doing about, you know, 90%, right? However, if you let the doctor use the AI, it boosts the doctor to 98%. 98% accuracy. Yes. And what's going on there? It's very interesting. So it turns out that although they got the same 90%, they had different expertise that the AI had access to more data. And so it could look at the lesions that were rare, that the doctor may never have seen, okay?

7. 1:43:10

   But the doctor has more in-depth knowledge of the most common ones that he's seen over and over again and knows the subtleties and so forth. But so putting them together, it makes so much sense that they're going to improve if they work together. And I think that now what you're saying is that using AI as a tool for discovery with the expert who's interpreting and looking at the arguments, the statistical arguments, and also looking at the paper maybe in a new way, maybe that's the future of science.

8. 1:43:47

   Maybe that's what's going to happen. Everybody is worried about, oh, AI is going to replace us. It's going to be much better than we are everything, and humans are obsolete. Nothing can be further from the case. Our strengths and weaknesses. are different and we by working together it's going to strengthen it's uh you know both