“These models are very powerful. If they generate things that are flawed, then with software you have to be concerned about whether you just take the generated output and incorporate it into your mission-critical software,” he says. 

But there are risks associated with AI language models that even the most up-to-date and tech-savvy people have barely begun to understand. It is hard to detect when text and, increasingly, images are AI generated, which could allow these tools to be used for disinformation or scamming on a large scale. 

The models are easy to “jailbreak” so that they violate their own policies against, for example, giving people instructions to do something illegal. They are also vulnerable to attacks from hackers when integrated into products that browse the web, and there is no known fix for that problem. 

Ghahramani says Google does regular tests to improve the safety of its models and has built in controls to prevent people from generating toxic content. But he admits that it still hasn’t solved that vulnerability—nor the problem of “hallucination,” in which chatbots confidently generate nonsense. 

Hard launch

Going all in on generative AI could backfire on Google. Tech companies are currently facing heightened scrutiny from regulators over their AI products. The EU is finalizing its first AI regulation, the AI Act, while in the US, the White House recently summoned leaders from Google, Microsoft, and OpenAI to discuss the need to develop AI responsibly. US federal agencies, such as the Federal Trade Commission, have signaled that they are paying more attention to the harm AI can cause.

Shah says that if some of the AI-related fears do end up panning out, it could give regulators or lawmakers grounds for action with the teeth to actually hold Google accountable. 

But in a fight to retain its grip on the enterprise software market, Google feels it can’t risk losing out to its rivals, Shah believes. “This is the war they created,” he says. And at the moment, “there’s very little to nothing to stop them.”