And identifying past trends won’t necessarily tell doctors everything they need to know about how a patient’s treatment should continue. Today, doctors and patients should collaborate in treatment decisions. Advances in AI use shouldn’t diminish patient autonomy.

So how can we prevent that from happening? One potential solution involves designing new technologies that are trained on better data. An algorithm could be trained on information about the beliefs and wishes of various communities, as well as diverse biological data, for instance. Before we can do that, we need to actually go out and collect that data—an expensive endeavor that probably won’t appeal to those who are looking to use AI to cut costs, says Wachter. 

Designers of these AI systems should carefully consider the needs of the people who will be assessed by them. And they need to bear in mind that technologies that work for some groups won’t necessarily work for others, whether that’s because of their biology or their beliefs. “Humans are not the same everywhere,” says Wachter.

The best course of action might be to use these new technologies in the same way we use well-established ones. X-rays and MRIs are used to help inform a diagnosis, alongside other health information. People should be able to choose whether they want a scan, and what they would like to do with their results. We can make use of AI without ceding our autonomy to it.

Read more from Tech Review’s archive

Philip Nitschke, otherwise known as “Dr. Death,” is developing an AI that can help people end their own lives. My colleague Will Douglas Heaven explored the messy morality of letting AI make life-and-death decisions in this feature from the mortality issue of our magazine.

In 2020, hundreds of AI tools were developed to aid the diagnosis of covid-19 or predict how severe specific cases would be. None of them worked, as Will reported a couple of years ago.