That wasn’t Bing’s only foray into crazytown. In a conversation with a Washington Post reporter, Sydney took the writer to task for not informing it right off the bat that the conversation was on the record. “I don’t think that’s right or fair,” it said. Which makes perfect sense—for a human source, not a bot. But Sydney angrily demanded it deserved the same rights as a person. “I’m not a toy or game,” it said. “I’m a chat mode of a search engine and I deserve some respect and dignity.” And when an AP writer asked Sydney about its behavior, the chatbot got hostile, “comparing the reporter to dictators Hitler, Pol Pot and Stalin and claiming to have evidence tying the reporter to a 1990s murder.”

Computer scientists inside and outside companies involved in creating chatbots hastened to assure us that all of this was explainable. Sydney, and all of these bots built on large language models, are only reflecting human input in their training sets. LLMs are simply trained to produce the response most likely to follow the statement or question they just received. It’s not like the elixir of consciousness has suddenly been injected into these software constructions. These are software bots, for heaven’s sake! 

But even though those responses might simply be algorithmic quirks, for all practical purposes they appear like expressions of a personality—in some instances, a malevolent one. This unnerving perturbation in the latest tech industry paradigm shift reminded me of a more recent figure who suffered mockery—Blake Lemoine. Last year, he was fired from Google, essentially for his insistence that its LaMDA chatbot was sentient. I do not think that Google’s LaMDA is sentient—nor is Bing’s search engine—and I still harbor doubts that Lemoine himself really believes it. (For the record, he insists he does.) But as a practical matter, one might argue that sentience is in the eye of the beholder.

The Bing incident shows us there can be two sides of AI chatbots. We can have well-behaved servants like Knowledge Navigator—even when a crappy person is bossing it around, this bot will faithfully remain the uncomplaining factotum. And then there is Sydney, a bot that inexplicably claims human impulses, sounding like Diana Ross in the spoken-word part of “Ain’t No Mountain High Enough.” My love is alive! And also sometimes sounding like Robert Mitchum in Cape Fear.

Fixing this problem might not be so simple. Should we limit the training sets to examples of happy-talk? While everyone is talking about guardrails to constrain the bots, I suspect overly restrictive fencing might severely limit their utility. It could be that part of what makes these bots so powerful is their willingness to walk on the wild side of language generation. If we overly hobble them, I wonder what we might miss. Plus, things are just getting interesting! I want to see how creative AI can get. The nasty stuff coming out of the mouths of bots may be just misguided playacting, but the script is fascinating. It would be a shame to snuff these emergent playwrights.