What Is Google LaMDA & Why Did Someone Believe It’s Sentient? via @sejournal, @martinibuster

LaMDA has been in the news after a Google engineer claimed it was sentient because its answers allegedly hint that it understands what it is.

The engineer also suggested that LaMDA communicates that it has fears, much like a human does.

What is LaMDA, and why are some under the impression that it can achieve consciousness?

Language Models

LaMDA is a language model. In natural language processing, a language model analyzes the use of language.

Fundamentally, it’s a mathematical function (or a statistical tool) that describes a possible outcome related to predicting what the next words are in a sequence.

It can also predict the next word occurrence, and even what the following sequence of paragraphs might be.

OpenAI’s GPT-3 language generator is an example of a language model.

With GPT-3, you can input the topic and instructions to write in the style of a particular author, and it will generate a short story or essay, for instance.

LaMDA is different from other language models because it was trained on dialogue, not text.

As GPT-3 is focused on generating language text, LaMDA is focused on generating dialogue.

Why It’s A Big Deal

What makes LaMDA a notable breakthrough is that it can generate conversation in a freeform manner that the parameters of task-based responses don’t constrain.

A conversational language model must understand things like Multimodal user intent, reinforcement learning, and recommendations so that the conversation can jump around between unrelated topics.

Built On Transformer Technology

Similar to other language models (like MUM and GPT-3), LaMDA is built on top of the Transformer neural network architecture for language understanding.

Google writes about Transformer:

“That architecture produces a model that can be trained to read many words (a sentence or paragraph, for example), pay attention to how those words relate to one another and then predict what words it thinks will come next.”

Why LaMDA Seems To Understand Conversation

BERT is a model that is trained to understand what vague phrases mean.

LaMDA is a model trained to understand the context of the dialogue.

This quality of understanding the context allows LaMDA to keep up with the flow of conversation and provide the feeling that it’s listening and responding precisely to what is being said.

It’s trained to understand if a response makes sense for the context, or if the response is specific to that context.

Google explains it like this:

“…unlike most other language models, LaMDA was trained on dialogue. During its training, it picked up on several of the nuances that distinguish open-ended conversation from other forms of language. One of those nuances is sensibleness. Basically: Does the response to a given conversational context make sense?

Satisfying responses also tend to be specific, by relating clearly to the context of the conversation.”

LaMDA is Based on Algorithms

Google published its announcement of LaMDA in May 2021.

The official research paper was published later, in February 2022 (LaMDA: Language Models for Dialog Applications PDF).

The research paper documents how LaMDA was trained to learn how to produce dialogue using three metrics:

Quality

The Quality metric is itself arrived at by three metrics:

The research paper states:

“We collect annotated data that describes how sensible, specific, and interesting a response is for a multiturn context. We then use these annotations to fine-tune a discriminator to re-rank candidate responses.”

Safety

The Google researchers used crowd workers of diverse backgrounds to help label responses when they were unsafe.

That labeled data was used to train LaMDA:

“We then use these labels to fine-tune a discriminator to detect and remove unsafe responses.”

Groundedness

Groundedness was a training process for teaching LaMDA to research for factual validity, which means that answers can be verified through “known sources.”

That’s important because, according to the research paper, neural language models produce statements that appear correct, but are actually incorrect and lack support from facts from known sources of information.

The human crowd workers used tools like a search engine (information retrieval system) to fact-check answers so that the AI could also learn to do it.

The researchers write:

“We find that augmenting model outputs with the ability to use external tools, such as an information retrieval system, is a promising approach to achieve this goal.

Therefore, we collect data from a setting where crowdworkers can use external tools to research factual claims, and train the model to mimic their behavior.”

LaMDA Was Trained Using Human Examples and Raters

Section 3 of the research paper describes how LaMDA was trained using a set of documents, dialogs, and utterances numbering in the billions, consisting of 1.56 trillion words.

Section 4.2 documents how humans rated the responses by LaMDA. The ratings are feedback that teaches LaMDA when it’s doing well and when it is not.

The human raters use an informational retrieval system (search engine) to verify the answers, ranking them as helpful, correct, and factual.

LaMDA Training Used A Search Engine

Section 6.2 describes how LaMDA receives a question and then generates an answer. After the response is generated, it then performs a search query to verify the accuracy and revises the answer if it is incorrect.

The research paper mentioned above illustrates the process of receiving a question, generating a draft response, researching the response, and then updating it with correct facts:

The problem with the answer is that it is factually incorrect. So LaMDA performs a search query and picks facts from the top results.

It then responds with the factually updated response:

“Oh wow, her life course is so inspiring. Did you know she was a practitioner of Japanese flower arrangement before turning to sculpture?”

Note the “Oh wow” part of the answer; that’s a form of speaking learned how humans talk.

It seems like a human is speaking, but it merely mimics a speech pattern.

Language Models Emulate Human Responses

I asked Jeff Coyle, Co-founder of MarketMuse and an expert on AI, for his opinion on the claim that LaMDA is sentient.

Jeff shared:

“The most advanced language models will continue to get better at emulating sentience.

Talented operators can drive chatbot technology to have a conversation that models text that could be sent by a living individual.

That creates a confusing situation where something feels human and the model can ‘lie’ and say things that emulate sentience.

It can tell lies. It can believably say, I feel sad, happy. Or I feel pain.

But it’s copying, imitating.”

LaMDA is designed to do one thing: provide conversational responses that make sense and are specific to the context of the dialogue. That can give it the appearance of being sentient, but as Jeff says, it’s essentially lying.

So, although the responses that LaMDA provides feel like a conversation with a sentient being, LaMDA is just doing what it was trained to do: give responses to answers that are sensible to the context of the dialogue and are highly specific to that context.

Section 9.6 of the research paper, “Impersonation and anthropomorphization,” explicitly states that LaMDA is impersonating a human.

That level of impersonation may lead some people to anthropomorphize LaMDA.

They write:

“Finally, it is important to acknowledge that LaMDA’s learning is based on imitating human performance in conversation, similar to many other dialog systems… A path towards high quality, engaging conversation with artificial systems that may eventually be indistinguishable in some aspects from conversation with a human is now quite likely.

Humans may interact with systems without knowing that they are artificial, or anthropomorphizing the system by ascribing some form of personality to it.”

The Question of Sentience

Google aims to build an AI model that can understand text and languages, identify images, and generate conversations, stories, or images.

Google is working toward this AI model, called the Pathways AI Architecture, which it describes in “The Keyword“:

“Today’s AI systems are often trained from scratch for each new problem… Rather than extending existing models to learn new tasks, we train each new model from nothing to do one thing and one thing only…

The result is that we end up developing thousands of models for thousands of individual tasks.

Instead, we’d like to train one model that can not only handle many separate tasks, but also draw upon and combine its existing skills to learn new tasks faster and more effectively.

That way what a model learns by training on one task – say, learning how aerial images can predict the elevation of a landscape – could help it learn another task — say, predicting how flood waters will flow through that terrain.”

Pathways AI aims to learn concepts and tasks that it hasn’t previously been trained on, just like a human can, regardless of the modality (vision, audio, text, dialogue, etc.).

Language models, neural networks, and language model generators typically specialize in one thing, like translating text, generating text, or identifying what is in images.

A system like BERT can identify meaning in a vague sentence.

Similarly, GPT-3 only does one thing, which is to generate text. It can create a story in the style of Stephen King or Ernest Hemingway, and it can create a story as a combination of both authorial styles.

Some models can do two things, like process both text and images simultaneously (LIMoE). There are also multimodal models like MUM that can provide answers from different kinds of information across languages.

But none of them is quite at the level of Pathways.

LaMDA Impersonates Human Dialogue

The engineer who claimed that LaMDA is sentient has stated in a tweet that he cannot support those claims, and that his statements about personhood and sentience are based on religious beliefs.

In other words: These claims aren’t supported by any proof.

The proof we do have is stated plainly in the research paper, which explicitly states that impersonation skill is so high that people may anthropomorphize it.

The researchers also write that bad actors could use this system to impersonate an actual human and deceive someone into thinking they are speaking to a specific individual.

“…adversaries could potentially attempt to tarnish another person’s reputation, leverage their status, or sow misinformation by using this technology to impersonate specific individuals’ conversational style.”

As the research paper makes clear: LaMDA is trained to impersonate human dialogue, and that’s pretty much it.

More resources:

Image by Shutterstock/SvetaZi