0:15 Welcome to our second episode of the Stanford healthc care AI uh water cooler um discussions and uh I'm I'm Justin nordan Stanford faculty uh and co-founder of qualified Health uh we Matthew lungren are their co-host Who's chief scientific officer for Microsoft health and also on Stanford faculty and we're thrilled to be joined by Dr Graham Walker who is also a Stanford medicine former Stanford faculty former we'll get you former Stanford faculty perfect uh

0:47 who also co-directs the Advanced Technology Group for Kaiser Permanente which is all the advanced technology Ai and new things coming uh and he's also the founder of MD cal um who I'm sure many people have used a new company called off call and and many other things so thank you Graham so much for for joining us thanks for having me guys yeah good to be back on uh on the virtual Farm as it were this counts um yeah so for today I mean so

1:15 from our first episode we had you know a ton of a ton of great responses some comments about uh you know some technical things some things that people wanted to see us see us cover and as you know this is really meant to be kind of a conversation that reflects the ones that we have when we're not you know on camera and recording um and we have a few topics today um and then we'll kind of get into just some just some general

1:36 discussion about about what we're seeing today so uh we're going to talk a about GPT 4.5 um uh kind of one of the newer models that have hit the scene although we have a a bunch and then we'll talk about the bunch as well we we kind of have a paralysis by choice at this point with so many different models to choose from and then we'll talk about some new data that's coming out around um Physicians and their use of both the

1:59 public models and even some of the solutions that um have been have been deployed into practice uh and get get F to take and there's just a few odds in end some headlines that have come our way that have raised a lot of discussions one around uh some proposals U around regulation and maybe even new capabilities for AI models um prescribing capabilities which is kind of an interesting idea um and we'll get everyone's reaction sounds good uh well should

2:29 should we start with some of the new capabilities yeah let's do it I think maybe it's worth it and Graham feel free to chime in on this I I just wanted to just a level set well uh so for those of you aren't familiar with the the the lore behind the ti Z unicorn this is Sebastian Buck this is a Harkin back to a paper he wrote SP Sparks of AGI basically paper uh the original gbd4

2:51 which you know surprised a lot of people that the model was able to put together as you can see in the middle something that resembles a unicorn um again this was a language model there was this was a this was a shocking result back in 2022 um and obviously a leap from 3.5 so to to in the recent 4.5 uh sort of evaluations he posted the new kind of updated uh impression of a tixie unicorn just to give a visual sense but if you

3:18 actually look at some of the data um uh around the model's performance you know again we're not seeing this massive leap but remember that some of these benchmarks are you know relatively close to saturation certain areas I think for some of these simple ones um some important things to take away uh we're used to these huge jumps right between three to four and and and we're seeing you know some maybe somewhat incremental or slowing Improvement and this is

3:42 really reflective I think of um of the pre-training uh the traditional approach to pre-training the scaling laws of you know exponential input for somewhat linear Improvement so it gets these models get bigger and bigger they become harder and harder to make uh quote unquote smarter but um but we have some other things to to work with in tes time compute but on the right hand side which I'm really excited about which is that you know the hallucination rate which

4:06 does tend to to limit or really force you to build a lot of scaffolding around applications um off the shelf is starting to look quite a bit better and I think a lot of people would agree when they when they're using the models I don't know Gram or Justin if you've been playing with any of these newer models and noticing that you know the stuff that you used to pick up all the time this thing was wrong or this seemed off

4:26 it's starting to look a little bit better I mean Matthew I'm a little little surprised that it still is 37% I I don't I'm not super familiar with the simple QA um data set but I mean that that still seems way higher I think there's probably two things going on people are more comfortable with the llm responses and so they're probably checking them less to be completely honest like they you know these things have have met some sort of

4:55 acceptable criteria for humans to start using and trusting I mean I think you know Google's partly um partly to blame for that too right I mean they've they've just been been embedding them in all your Search terms I mean often I'm now seeing like if I'm like getting in an argument with a friend a friend will send me the Google answer but it's not the Google answer it's the Google AI answer and that's not really the same

5:19 thing um you know it used to be oh yeah here's the screenshot of the fact from the web page or from the from census.gov nope it's now the screenshot and I don't know if my friend know that they're sending an AI version or if they are just implicitly trusting them but that that's the thing like we've reached some level where people are just implicitly trusting these things I'm I'm not sure that Humanity should be trusting them just yet but um

5:46 it seems like it seems like we're there yeah there's almost a yeah there's a trust but verify I think too like at least for those of us that use them all the time um and in this metacognition thing right where you're sort of like okay I know it's it's going to get me like 80% of the way there but it's still going to save me some some time but the the you know the the worst part of this

6:06 is what we would call sort of the the GPT slop right I think which which by the way I think was in the running for the term of the year last year um Webster dictionary new new term but but that's where like I mean I've gotten to the point I think there was there's been some data on this where I can read you know some some of these posts and kind of say ah that looks like a Claud or

6:25 that looks like a gbd4 to me and and there you could tell there was not a lot of effort put into the editing well this is the thing I want to pull up that kind of brings both of those things together which is one there are so many of these models out now or like what's changed in two years right versus oh my gosh look at these capabilities they've come out you know one there's so many models out there now

6:48 and competition for that that Graham to your point the companies are just embedding the AI capabilities in their native applications which lots of people thought would come and the reason to do that is to keep people in the platform and it's a way to compete with with others and so there are so many of these models out there you know for I guess most of us and people listening it's reminiscent of the old uh search Pages

7:13 where you'd use altta Vista and ask geves and just a long list of companies where you'd ask multiple times to try to get to your answer before you know Google really kind of one out um and as as we're looking at these models a question that like I get all the time is like well what model do I use for what and the challenging part of that is it's changing every few weeks right it's not just GPT 4.5 that came out it's Cloud

7:38 3.7 just came out and there's new models every couple weeks and so that's changing um but the interesting thing to me at least is it seems like this is moving to a world where there's going to be multiple models to choose from both for a consumer both for companies both for healthcare organizations and so you know when I think about it you know how do you set yourselves up to use multiple different things and to be able to

8:01 access multiple different things is something that you know I think is super important but curious math gram what what do you guys think about that what are you using for yourself what you're showing on the screen is I was just doing the back of the napkin that's about $300 a month of of subscriptions right tools yeah assuming you're not paying the $200 a month for for open AI Pro for the pro yeah I I'm I'm mostly using um chat

8:31 chat gbt and Claude um chat gbt tends to give me um answers that are a bit more um uh unique um and then I'm using Claude really around more editing capability when I want feedback on my writing but I don't want my writing to change as much chat BT is more than happy to like be to tell me that oh I love this article that you're that you just wrote and then rewrite it in its own

9:00 um its own words and you know it's like both complimenting me but also it's like a backhanded compliment that it's redone the whole thing because it didn't it wasn't that good versus Claude typically will stick with my own words and then try to clean them up make them a little bit more cohesive without changes changing as much so those are the two I'm using the most I've played around with deepsea cafer amount I found that um deep seek like when I've just been

9:27 testing deep seek deep seek will like let you get away with more stuff and it'll be a little bit more allow a little bit more devious behavior um which is probably a fact because it's you know open source and not as um you know the weights are all open so I imagine they probably have to allow more of that and I think that's the testing that I've seen as well like you can get deep seek to tell you how to rob a bank

9:49 much easier than you can Claude or chat chbt or some of the other kind of Clos models have you tried them all like I mean that you still practice right and and so do ever um you know throw throw in and just see if there if it comes up with a differential that seems reasonable do you ever you know do that or talk the model even yeah yeah the differentials are always um reasonable they so far they've and I'm I'm excited

10:13 to try out 4.5 Matthew if you want to give me access um the um the models all have some Gap that still makes me very leery to rely on them I think they're they're you know they're content generation so they're great for generating a differential but I'm not yet convinced that they're going to be a comprehensive differential or that the number one thing is always going to be the correct one at the top um you know

10:39 like if you uh you know and then you add one more complicating Factor like oh it's a kid or it's a pregnant patient or something like that and then they start to show a little bit more of their um their challenges I think but again growth mindset these things are every year these things are you know getting better and better and it's harder and harder to find those CH find those areas where they they're not as good yeah I

11:08 remember when 3.5 came out like you'd see these viral threads like super simple you know medical questions it' be you know wildly off base and I like I said I I mean obviously I use it all the time I you know part of some red teaming work and things and it's increasingly hard um in fact it actually stretches some of your your own medical knowledge and then you have to go back to the like that sounds right but I need to double

11:31 check and that kind of gets me to deep research and I don't know if you played with those uh gramar Justin but like you know I've had moments of just like almost just like the initial gbt moment with with deep research um where I'm like this is a game changer and then they'll be times when like ah it's a pretty superficial read of of some of the content but you know where this is headed I me to your point this is the

11:51 worst it'll ever be that kind of thing you hear said a lot but if you talk about access to all of your journals and all the you know all the things that we subscribe to in like our medical libraries now you're talking right I mean that could be pretty pretty powerful and I you know um I don't know if you've used these and kind of tested out with topics that you know really well to kind of get a sense of how close

12:11 it is yeah well speaking of medical journals and access there's a recent announcement a headline open open evidence just raised a boatload of money yeah um and you know they claimed I think 25% of of Physicians were Now using the tool kind of in in my head you know I think about it as an up-to-date you know competitor of being able to ask open-ended questions cite this with real research you know reduce that hallucination rate um which was

12:40 something that came up before you know drop that down where you know initially when people were asking questions with gbd4 you I was like oh my gosh look at all these citations you look at the citations they were all made up yeah um but you know as the scaffolding I think that you know Matt talked about before around these models where you constrain what they're B to look through to only be able to produce certain things um

13:03 it's not just that the models are getting better basically as software developers and I feel like this is actually something a lot of people miss when they're talking about at least in healthcare of people who aren't maybe familiar with the technology it's like oh did the models just get better it's like that is one aspect of how these tools are getting better we're also getting a lot better as you know Engineers for how to use them and use

13:25 them in a way that actually produces meaningful results and so that's just an example Le um recently of something that has has really kind of gained a lot of traction in the last few months and years I was going to say gram have you ever tried to throw in prompt to like just say CLA or something and just say create MD Cal from scratch like have you ever tried that totally yeah um and it's uh there are some calculators that it's

13:50 been able to do really well and there's others that it's like oh this is dangerously bad cuz it's going to you know I mean that that's the other thing that I think about a lot is um it's that level of trust I mean it's it's the Google embedding it in search results thing that um you know I do worry that even with hallucination rates going down and Scaffolding getting better and you know you could feed the model back to

14:15 itself and verify that the answer is that the model thinks the model is accurate um I still do worry that like uh somebody um is going to make a you know make a decision based on um an llm piece of information that is maybe it's not wrong but it's just not comprehensive and um they are they do not have the training or the experience to know that oh yeah usually you can give that drug in this particular

14:46 circumstances that was a really bad idea and um you know either the model gave you the wrong information or you didn't give the model enough information for the model to give you the right answer you know I mean mean take pregnant patients take for some reason I'm thinking about um uh dexin toxicity and like giving them calcium and stone heart which I think has mostly been debunked but still like you could imagine a human doctor is going to have

15:15 it's going to try to take in all the pieces of information and then and then take an action um but if you're just giving it the two liner from your h&p that's often not sufficient information to give a right answer too yeah and I think the the usage is interesting but one thing actually that has come up before and through a ton of conversations Graham to argue the other side as compared to what so are there

15:42 going to be AI misses yes yes and yes you know often it'll come up and usually they're overblown but you always hear the media headlines of look how many mistakes Physicians make you know and you know now we're starting to see the articles of patients working with AI to get the diagnosis that the doctor missed um H how do you think uh we what should we compare you know that AI against you know or even for yourself how how are

16:10 you comparing it for yourself for when when and where you should use these tools yeah it's uh I think of the the Tesla full full driving capability thing where I don't really know that um it it feels ethically because it's like the loss of control that these tools need to be not just as good not just one point better than the average human driver because they you can always argue that the average human driver may have hit

16:39 The Brak fast enough or something like that like it does feel because you're losing the control Factor it feels like they need to be at the 99.9th percenti of the best driver in in the world um you know and then the the other thing I always think about is we to your point Justin we can't think about risk um of tools in a vacuum we have to think of total risk right uh I mean it's like

17:03 when I give somebody a blood transfusion I tell them your risk of walking across the street after I get you feeling better and you get hit by a bus is way higher than your risk of HIV or hepatitis C which are of course the two most common things that people worry about but they're like one in million one in 10 million um and so you have to think about all the risk and and certainly the the other risk is you know

17:23 patients waiting six months to see a cardiologist because there's a backlog to get seen um so and the patients are going to get sicker in that time so I think we do have to think about um consider all the risk and not just think about the risk of AI or just think about the risks of humans but like think about the risk risk of being a patient in American Healthcare you know yeah uh the the interesting thing that I want to

17:48 make sure we get to here on people using these tools and how and and and why you brought this up a little bit but this was a recent survey uh the published you know a second generation of kind of AI use um and they call AI use interestingly not artificial intelligence they call augmented uh intelligence but across uh clinicians and this number was was shocking to me you know before and it seemed to be you know around a third of you know

18:20 clinicians were using uh AI in their practice now that jumped to two3 and the the context I'm always still you know shocked by this is two3 of clinicians are using this most work settings haven't given people access to these tools and so it's just this amazing uh it's amazing things in gram I know we've we've talked about this before yeah but it's like people are going around and using phones using other things uh but what what do you

18:47 make of this I would love to see if this trend is true in other countries I tend to assign a lot of the things that we see in the US as due to our very dysfunctional Healthcare System um and so like I see the the the rapid adoption here as a sign of like doctors in the US are really struggling to keep up with all of the stuff whether it's clinical or administrative or PRI author or anything

19:15 like that um so I would be fascinated to know if this trend is an international trend is the UK I mean every actually every country's Physicians and nurses and you know healthcare workers in the whole world are struggling many are quitting many med students across the world don't want to practice at the bedside so it could be that this is you know a a wide Trend or it could be like American medicine is particularly bad and so doctors are way more likely to

19:44 use these in the US and other places I mean I I feel like we talked a little bit about this last time Justin too which was the sort of British medical journal had a paper on surveying GPS and the NHS and you know they they weren't quite as um you know granular as maybe what I'm seeing here and and they were actually focusing on just the public API like basically the public models that you'd have you know put on your phone

20:04 and and I think the surprising result you that we talked about which I think is echoed here but just I can't quite match the results there to he as well because I'm assuming some of these are you know software that that was sold you know from from you know vendors that have vetted the the capabilities but you know I guess almost a third of these docks were using it for what you would consider a medical device right

20:28 something that would be a clinical decision support device and that you know that that's surprising but also at the same time not right like of course they are like you know at mean at some level to your point they're stressed they're they're working through things and maybe they're finding these tools useful um you know I'm hearing stories of folks um are literally just kind of using the voice mode um uh around uh you know in ER settings urgent cares and

20:51 basically kind of almost doing a like you would when you were an intern or resident presenting the case and then you know getting getting feedback and um um I've heard other stories of folks I'm hoping to have one of them on the on the Pod because they're uh they're doing a study on this but remember in the old when we were in med school at least when I was it was kind of a novel idea to

21:10 bring an infectious disease or Farm you know a pharmacist actually with you on rounds particularly in ICU to you know look at the huge list of meds we we're putting patients on and and give feedback right and um now a similar thing is happening but they're literally Wheeling around uh you know GPT basically right and and sort of having it listen into the discussion and then make make comments I I think it's a fascinating idea um and again just kind of brings to

21:36 the point that maybe we don't know how to practice medicine and take advantage of these tools in the best way yet you know yeah it's still very very early days yeah what's interesting about that though is this this one other slide and then I I'll I promise I'll put away some of the the AMA data here but the definite Advantage group to me is what what I'm tracking in in in my head um you know obviously scribing

22:05 has gained a ton of momentum you know over the past couple years as these tools have gotten better and there's still you know a mixed you know people have different opinions for how helpful or where it's helpful and things like that but but there are certain people who who swear by it now and there are certain people right who starting to use these other AI tools that are absolutely seen advantaged you know I've talked to former students who taken you know

22:29 generative medicine course who are using these things all the time and they think they save just from the public tools where they're trying not to put in any patient information uh a couple hours a day you know when they were an intern because it helped with discharge summaries planning tough conversations with patients you know uh looking up current evidence and and things like this and so the the reason I track this kind of definite Advantage group is it's

22:56 kind of a bell weather for kind of what's to come there's always this kind of curve of adoption of Technology but when you track that group and people are absolutely seeing the benefits to me it just says it's a matter of time before this gets more wde widespread um and and and that's one of the things that yeah again this is the worst it will be um and so we're really already seeing benefits that if you scale it to the

23:18 issues that Grandma was talking about around burnout issues people leaving medicine um it it has started to at least give give me some hope I think we need to have more people sharing how they're using them I mean Matthew I'd never heard of that kind like a virtual AI pharmacist um idea love it um you know I think there's so many ways that these tools can be used that people just aren't thinking about I mean I I was

23:46 showing my mom like oh you could like talk to it in Spanish like you want to practice your Spanish just ask it to do super beginner Spanish with you and it'll just do it and you know she hadn't even thought of that as a possibility um and you know I I had like a I had a challenging pal of care discussion with a patient in in the Ed and I like you know the the day later it was still

24:08 bugging me I was like God could I have done that better and so I kind of described without of course Phi or details I described like how the patient and their family was feeling and what I was trying to convey and and I said hey can you pretend to be this family member be the daughter of my patient and then I'm going to talk to you and then when I say you know that's a wrap or something

24:31 let's pause and like give me feedback how did I do um and so there are just you know I I shout out Sanford did my sim simulation medicine training there but like there's so many ways we could use these tools outside of just clinical medicine think about simulation think about difficult you know conversations with patients Breaking Bad News AI pharmacist um I I heard somebody a guy at Yale is using a bridge with Med students and having the med student

25:00 write their own note having a bridge write the note and then they have to critique what they liked about their Note versus a Bridges and why and why did a bridge include that here in the HPI but you put it in Social history like there are ways we can use these tools not to replace us but to teach and educate and and help us for sure yeah the Education topic is one that you know I'm surprised I don't hear more often

25:22 and you I you brought up the really really important news case which is the empathetic coaching there's been a few papers on I think that's a phenomenal use case right we don't first of all we don't get that many terms about thankfully in a lot of our you know Specialties except for obviously paliative care or something where you know we we would get seminars from folks in paliative care who do this as experts and they would come and give us some you

25:44 know some tips on how we can have these hard discussions with patients but you know hearing it a couple times maybe reading some but then actually when you're standing there having to have that discussion man I would almost take any sort of life preserver to help me you know for of discussions and but on the Education topic there's been some phenomenal papers that just aren't really getting a lot of attention and I'm surprised by one um I don't know if

26:08 Justin you have that one that was it was a news article that recently talked about um a school in Nigeria that used GPT forges as a tutor and two grade levels of advancement within like a six week Intervention which is insane to me and then obviously just like you I mean you talked about your mom my my daughter takes Japanese and her you know we obviously live in PA Alto so everyone's thinking about gbt anyway but her

26:33 teacher just said to the students straight up you need to be practicing your Japanese conversation with um you know the advanced voice mode of of one of these models and I think that in you know it's it's been just tremendous I wonder whether there's an medical equivalent you mentioned a couple you Cas I remember back in the day we used to just go through question banks for the step yeah um but these models as I think Nick Shaw's group showed can can

26:57 actually write um you know step questions that are indistinguishable from real ones um and actually when you score the people on them it's equivalent right so it's really is a a phenomenal study a I again I feel like there's a lot maybe this is going on again under the covers and we're just not hearing as much about it but um we're just it's it's early I feel like is what what I end up concluding the the other thing I'll just

27:20 call out a few from students have come up over the years from learning just explaining Concepts yeah hey explain CB cycle to me no no no explain it like I'm I'm an eighth grader no no no explain it like I'm six yeah and you get these different reps and versions of these Concepts that these models right have you know an inherent ability to kind of tackle from different perspectives like that's what an expert teacher does they

27:48 have such a good grasp of material they can get six different ways to explain it and like that's what these models can do and uh to to Graham your point before people are still just learning these different ways to interact and what's scary to me kind of seeing this is if I when I hear students use of these models they're all across the Spectrum some are using these models 10 hours a day in their learning it's

28:18 always on transcribing lectures summarizing lectures asking questions translating teaching others are barely using them at all and the gaps of just what people can do and understand uh with these models that alarms me a little bit especially when we see how much more some people can get done with them and so just and that's part of the reason we're doing this is hey great maybe Graham's gonna try advanced voice mode on on on rounds with a pharmacist

28:48 next time or people are going to kind of get ideas for ways to do this but hopefully people will do so safely I'll just that disclaimer I hope the models get smart enough to tell the student you don't need to know that cycle it's all a scam you'll never use it again I mean Matthew that's that's part of the um that you know that that's part of this I I have this idea that these models will

29:08 be um if we train them appropriately with without misinformation like these models will be kind of more objective Arbiters of the truth and they may actually drive us to reform the medical education curriculum um I mean I still why do why why were we all taught the brachial plexus not just that it exists but like the chords and the I don't even know right I remember memorizing those and I remember a bunch of pneumonics memorize them but why do I need to know

29:41 how those lines connect and then overlap and cross I don't know I don't think either of you could answer either but it's part of tradition but you know as I mean have you guys seen a a uh one of the the um you know first aid for step one now it is at least twice as thick as when I bought it in 2004 or something like that I mean it's insane how much more they have to to memorize and I mean

30:09 I thought it was bad enough to have to answer like stuff about cyclic GMP but now it's so much worse that like it may these tools may Force humans to kind of disrupt some of our tradition and be like oh what well it's important that people know that where the nerves are on the body but do they need to know the way that the breako plexus connects maybe not the kreb cycle it's important to know that that thing exists I think

30:34 it does like oxidative phosphorilation or is that the other one but anyway yeah anyway like we need to know what that is because you know aspirin overdoses block that but besides that do you need to have memorized the whole thing it's like a running joke you know totally well I I will say that well there's two I I feel like I have two minds on this one is like I uh I feel like we we are llms in some

30:57 like if we want to use this technology and we're do we have to do some pre-training right so there's some just like yeah exercising the muscle like that we do I I sort of still and you know happy with the way that we at least attempt to to get as much information it forces you to become someone that can absorb vast amounts of information process it and still be able to you know manipulate the concepts there's

31:18 something to that right I think in terms of just as being able to functionally keep up to your point with the acceleration but there's also this idea of skill atrophy which is kind of real I mean that you know um I I haven't used a map in 20 years um I probably could barely find my way outside of my neighborhood anymore without a GPS and and so that's real too how much does that impact me not maybe

31:38 as much am I more efficient sure so there's got to be some balance here but um but where that is I don't know but I I do agree though to to your point that the medical education system is just like all the rest of I would say the traditional education system is kind of staring into this uh and wondering what does this really mean for them yeah and and and with that said there have been people people have been asking

32:03 this question about medical education for a long time and you know I'll I'll give a shout out to Dr Charles prober who flipped the classroom did a lot at Stanford and really Across the Nation and we've had the internet and ways to get access to Raw memorization let's have people apply this information through cases learn in the classroom together to do that and so to me this is just a super supercharging of those Concepts which is

32:30 let's get people more into you know real simulated situations whether simulated with AI and use AI to kind of Advance it it just kind of really accelerates that um maybe the the one last topic uh to call out before we wrap is you know there was there was a bill proposed on AI being allowed to prescribe um Matt I'll I'll start with you as you wanted to to go through this like walk us through what it was and what you

33:00 think is going to happen well I mean it I think that so this isn't the first time so but this one did catch headlines I think just given the kind of current conversation around Ai and Healthcare and all these things that I think are starting to make make be more mainstream but I guess the general thought is that a a model um it could legally prescribe a medication and I don't you know know exactly how you'd even really Implement

33:24 that per se just given how things would work but I can imagine a scenario where there's some confined scope of certain medications where almost like you're starting to see with some of these apps right that are kind of willing and fairly I guess they kind of loosen the rules in terms of how how deep you need to go with your metac and your physician to actually get a medication that's common right but I think it's something

33:47 to that effect but now imagine you could have your app also has this ability to then literally prescribe a medication after you know interaction again I don't I don't see this passing um but but I always look for these kinds of signals that are kind of like that what you know Ethan M calls the Jagged Edge the jagged Frontier right of of where the capabilities are starting to meet some real world impact that that has pretty

34:11 big ramifications if um you know if if we' let it continue I do think the um the regulations that require uh I mean just getting back to I mean the same traditions of medical education are the the same Legacies of like medical lure I mean the this the fact that we all need a California license and if we drive to Nevada we need a Nevada license um I think those those structures are starting to break down I would I would love to have a

34:41 hospital be able to prescribe like have a hospital owning the the risk and yeah if the hospital wants to use an AI to do that give the hospital a a license um to prescribe medicines or to dispense medicines or something and and I agree Matthew the um if if you look at um many of the telea health providers what are those prescribers doing they are as fast as possible clicking boxes that have already been um evaluated and checked by

35:12 the patient which then have routed through an algorithm to say this person's appropriate to receive pryc or something I mean and then they're writing a prescription and then they're charging their insurance a bunch of money for it and then they're filling the prescription um how much work how much labor is the human doing in that piece not much often those medicines are over the counter and so it's a little bit of a shell game anyway um I you know

35:35 I don't support AI prescribing anytime soon but when you start to again looking at the whole picture when you start to compare it to what we're allowing companies to make billions of dollars doing um for prescribers that are sitting at home uh how how different is that I don't know well well well said uh well anyway with with with with that very optimistic note uh always that's what that's all you get from me is optimism uh we we

36:05 we'll wrap it there but uh thank you so much Gr for for joining us we'll have to have you back soon as we're all trying to figure out what's going to happen next with with Healthcare and AI anytime thanks for having me guys thanks Graham