AI Health Apps: The “Almost True” Problem

Open AI has acquired Torch, a health care startup that unifies lab results, medications, and visit recordings. Bringing this together with chatbt health opens up a new way to understand and manage your health. You've really really got to be careful with uh your health. There are plenty of people not really realizing that when they give their health to AI that it could in fact one hallucinate, two be composed of answers from losers on Reddit who don't know what the [ __ ] they're talking about. Yeah. Yet AI has learned from that. Yeah. What I don't like about this new world that we are in it, uh whether it's with health or software development, it feels like nobody knows [ __ ] and it feels like there is less and less care for knowing like specifics. It seems like everything is an approximity of truth but not truth. But truth cannot be just approximately true. It has to be definitely true. And this is what AI is doing. It is like almost true most of the time. It it's not just hallucination. It is some sort of like a soup mix of everything together. And it it might sound right. It might be mostly right. But I there are so many cases where you don't want you you don't want a general soup of like truth. You want a definite maybe even cited you know by a human let's say expert uh view on something like somebody who has made up their mind on something. Sometimes you need that much more than, you know, something that is just fuzzy and sort of true. This is the lamest example, but I was on I was on Instagram and I flick through and I I get this story that someone's telling me about the what the basis of my favorite band. And he sends a story about how he came up with the song. And he's talking and he's just saying this story and then just ends the thing. And I'm like like why are we in this world where this person just feels no like desire to say Mark the basist told this story in an interview with um you know Rolling Stone or why they feel no need to site their sources is wild to me. Like Mark didn't tell you this story. It's just like trust me bro this is this is it. to say that. Trust me, bro. That's trust me, Bo. And by the way, I think this story is true. And you know, people hammering me in the in the comments and I'm like, it might be true. I'm not doubting that it isn't true. Well, I am doubting it's a a little bit untrue, but my point is just site your sources. Like, we go through our articles here. We show the articles. It's really important for us to show the articles. It's not he hearsay. It's not this person said this, this person said that. Of course, you're allowed your own, you know, um, interpretations of things and, you know, we're not going to get everything perfect all the time, but I like to think that we show you the source and we talk about that. We're not we're not just like, "Trust me, bro." And similarly, um, there's a great um, annoyingly there's a a a YouTube channel called Better Stack, and they released a video on the tool search thing. Their video was titled uh this recent release made anthrop claude 10x faster. Now there are two problems with that title. One, the tool search tool does not inherently make MCPS faster. It doesn't make it f in fact I questioned in my video I said does it now make it slower because it's got to do that extra search to to whatever. Um, maybe in some ways you could squint a little bit and say, "Well, there's less context, so it's it's it's got less information to process." Maybe there's a um something to be said about that. But what I do have a problem with is 10x. And I know we say it in conversation when we're, oh man, I just I was like 100 times faster this morning on my run. I know we say it in a what's the word? Colloquial way. We say this sort of stuff, but when you're putting information out there and you present no facts or no statistics or no testing to say 10x, then it becomes a problem, especially if you're a news source. Do you know what I mean? We just throw stuff out there. And I'm more for me at the end of the day because I'm the one getting pissed off with it because the the youth of today Yeah. the folks at home, they're not picking up on the on the absurdity and the the destruction of what this actually means because it it means that when you say stats or when actual figures are put out there, you then ignore them and you you don't really process them in the same sort of way. Cancer rates have dropped by 10x. Like, yeah, they're just saying that. No, no, no. This is a big deal. You know what I mean? Like, it's I I know exactly what you mean. I don't want to get political. I just say that if you look in the world what's happening you you see that everywhere it seems like we're moving towards where truth doesn't mean [ __ ] like saying you just say things and you don't site you know your sources. you you don't care about like the the the the truth anymore essentially. You Mhm. you don't care about like the scientific method. You don't care about any of that. you just make up things and you you you can't be you can't basee everything uh on your feelings like how you feel today and you just go on about you know make decisions based on that um and I think we we were on a different path. I'm I tend to not think like that society goes like has a downfall. I tend to not think that way, but it's getting really really difficult to not see it that way because more and more uh misinformation is being shared and people are getting more and more lazy about it or like they don't just straight up don't care that because I do I do care. I I don't want anyone to come to me and give me some [ __ ] explanation of something. I Yeah. Yeah. You know, if if somebody in your like personal life or like in your circle gives you [ __ ] you shouldn't take it. You you're you know, you shouldn't allow people to [ __ ] you. So why do we allow that on a bigger circle that we have which could be social media, it could be geopolitics, it could be companies, uh you know with our data, whatever it is, we we should be a bit more should just care a little bit more about that. Another message to everyone is like don't put humans on such a high pedestal either. Like yeah AI is good. It makes mistakes. We talk about the easiest example I can give is hallucinations, right? Right. Hallucination rate. Humans hallucinate all the frigin time. We're hallucinating like probably more. So we get on our high horse about like what did I see today or yesterday? AI made some silly. Oh, I was watching like an historic video which is really cool, really interesting way to um showcase um AI in that someone does historical videos and he was doing one about Lisbon and he's taking photos and not photos, sorry, uh paintings and sketches of Lisbon way back in the 1800s and then animating them with AI. And I think this is a really nice way to like visualize Lisbon back in the 1800s or probably even earlier than that, probably 1500s, right? You know, he's very obvious about the fact there's AI. And then um there was one scene where the flag was like a different flag or it was the wrong flag that the AI had obviously whatever. And then there was this big onslaught of people laughing and they think AI is going to like, you know, rule the world or this and that. just like Jesus Christ. Like when if you're going to use that as as reason why AI isn't going to rule the world or or this or that. I bet you're completely unaware about how many paintings, sketches or stories or written scripts or whatever that humans have just got totally wrong. Yeah. You know, but you just don't know about it. But you're then like getting on your high horse pointing your finger. So [ __ ] Oh, I'm perfect. It was just so funny to me like obviously I see both sides. I understand what they're saying. But but we have methods for that. You know, we have the scientific method to to basically use tools to to replicate and then make sure that you know that we we know that we hallucinate. We know that we get things wrong, but we have ways to uh get better at it and do better. And I all I'm saying we should question it. Like would you absolutely absolutely question it and it's fun I like it is fun to question stuff and go and learn. This is part of a larger conversation on my show Command AI which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter.