Google showed off an array of new artificial intelligence (AI)-driven health care tools on Tuesday, from a souped-up chatbot that can shed light on your medical symptoms to enhanced search features that tell you if a doctor takes Medicaid.
Why it matters: There’s an arms race among big tech companies to infuse their products with AI — but the results, particularly in health care, can have unwanted consequences or pitfalls, like racial bias, privacy concerns and ethical problems.
Driving the news: The « large language model » that Google has been building for the medical world — an AI chatbot called Med-PaLM 2 — now consistently passes medical exam questions with a score of 85%, placing it at « expert » doctor level, the company said.
- That’s an 18% improvement from the system’s previous performance, per the company, and « far surpasses similar AI models. »
- A rival generative AI tool, ChatGPT, also passed the medical exams — but just barely. (ChatGPT’s creator, OpenAI, just released a new, more powerful version of its underlying tech.)
- Google’s system is being trained to « retrieve medical knowledge, accurately answer medical questions, and provide reasoning, » the company says.
Yes, but: Google acknowledges AI’s shortcomings in the medical realm.
- « There’s still a lot of work to be done to make sure [Med-PaLM 2] can work in real-world settings, » reads a blog post from Yossi Matias, a Google vice president of engineering and research, and Greg Corrado, its head of health AI.
- Google found « significant gaps » when the tool was « tested against 14 criteria — including scientific factuality, precision, medical consensus, reasoning, bias and harm, » per the post.
- « We look forward to working with researchers and the global medical community to close these gaps and understand how this technology can help improve health delivery. »