Why the AI industry could stand to slow down a little

I.

What a difference four months can make.

If you had asked in November how I thought AI systems were progressing, I might have shrugged. Sure, by then OpenAI had released DALL-E, and I found myself enthralled with the creative possibilities it presented. On the whole, though, after years watching the big platforms hype up artificial intelligence, few products on the market seemed to live up to the more grandiose visions that have been described for us over the years.

Synthetic text is rapidly making its way into the workflows of students, copywriters, and anyone else engaged in knowledge work; this week BuzzFeed became the latest publisher to begin experimenting with AI-written posts.

Elsewhere, OpenAI released plug-ins for GPT-4, allowing the language model to access APIs and interface more directly with the internet, sparking fears that it would create unpredictable new avenues for harm. (I asked OpenAI about that one directly; the company didn’t respond to me.)

If nothing else, the letter strikes me as a milestone in the march of existential AI dread toward mainstream awareness. Critics and academics have been warning about the dangers posed by these technologies for years. But as recently as last fall, few people playing around with DALL-E or Midjourney worried about “an out-of-control race to develop and deploy ever more digital minds.” And yet here we are.

There are some worthwhile critiques of the technologists’ letter. Emily M. Bender, a professor of linguistics at the University of Washington and AI critic, called it a “hot mess,” arguing in part that doomer-ism like this winds up benefiting AI companies by making them seem much more powerful than they are. (See also Max Read on that subject.)

In an embarrassment for a group nominally worried about AI-powered deception, a number of the people initially presented as signatories to the letter turned out not to have signed it. And Forbes noted that the institute that organized the letter campaign is primarily funded by Musk, who has AI ambitions of his own.

There are also arguments that speed should not be our primary concern here. Last month Ezra Klein argued that our real focus should be on these system’s business models. The fear is that ad-supported AI systems prove to be more powerful at manipulating our behavior than we are currently contemplating — and that will be dangerous no matter how fast or slow we choose to go here. “Society is going to have to figure out what it’s comfortable having A.I. doing, and what A.I. should not be permitted to try, before it is too late to make those decisions,” Klein wrote.

These are good and necessary criticisms. And yet whatever flaws we might identify in the open letter — I apply a pretty steep discount to anything Musk in particular has to say these days — in the end I’m persuaded of their collective argument. The pace of change in AI does feel as if it could soon overtake our collective ability to process it. And the change signatories are asking for — a brief pause in the development of language models larger than the ones that have already been released — feels like a minor request in the grand scheme of things.

Tech coverage tends to focus on innovation and the immediate disruptions that stem from it. It’s typically less adept at thinking through how new technologies might cause society-level change. And yet the potential for AI to dramatically affect the job market, the information environment, cybersecurity and geopolitics — to name just four concerns — should gives us all reason to think bigger.

II.

Aviv Ovadya, who studies the information environment and whose work I have covered here before, served on a red team for OpenAI prior to the launch of GPT-4. Red-teaming is essentially a role-playing exercise in which participants act as adversaries to a system in order to identify its weak points. The GPT-4 red team discovered that if left unchecked, the language model would do all sorts of things we wish it wouldn’t, like hire an unwitting TaskRabbit to solve a CAPTCHA. OpenAI was then able to fix that and other issues before releasing the model.

In a new piece in Wired, though, Ovadya argues that red-teaming alone isn’t sufficient. It’s not enough to know what material the model spits out, he writes. We also need to know what effect the model’s release might have on society at large. How will it affect schools, or journalism, or military operations? Ovadya proposes that experts in these fields be brought in prior to a model’s release to help build resilience in public goods and institutions, and to see whether the tool itself might be modified to defend against misuse.

If adopted by companies like OpenAI and Google, either voluntarily or at the insistence of a new federal agency, violet teaming could better prepare us for how more powerful models will affect the world around us.

At best, though, violet teams would only be part of the regulation we need here. There are so many basic issues we have to work through. Should models as big as GPT-4 be allowed to run on laptops? Should we limit the degree to which these models can access the wider internet, the way OpenAI’s plug-ins now do? Will a current government agency regulate these technologies, or do we need to create a new one? If so, how quickly can we do that?

I don’t think you have to have fallen for AI hype to believe that we will need an answer to these questions — if not now, then soon. It will take time for our sclerotic government to come up with answers. And if the technology continues to advance faster than the government’s ability to understand it, we will likely regret letting it accelerate.

Either way, the next several months will let us observe the real-world effects of GPT-4 and its rivals, and help us understand how and where we should act. But the knowledge that no larger models will be released during that time would, I think, give comfort to those who believe AI could be as harmful as some believe.

If I took one lesson away from covering the backlash to social media, it’s that the speed of the internet often works against us. Lies travel faster than anyone can moderate them; hate speech inspires violence more quickly than tempers can be calmed. Putting brakes on social media posts as they go viral, or annotating them with extra context, have made those networks more resilient to bad actors who would otherwise use them for harm.

I don’t know if AI will ultimately wreak the havoc that some alarmists are now predicting. But I believe those harms are more likely to come to pass if the industry keeps moving at full speed.

Slowing down the release of larger language models isn’t a complete answer to the problems ahead. But it could give us a chance to develop one.