Technology
Google
Elon Musk
Artificial intelligence (AI)
OpenAI
LLMs’ ‘reversal curse’ leads it to fail at drawing relationships between simple
facts. It’s a problem that could prove fatal
In 2021, linguist Emily Bender and computer scientist Timnit Gebru published a
paper that described the then-nascent field of language models as one of
“stochastic parrots”. A language model, they wrote, “is a system for haphazardly
stitching together sequences of linguistic forms it has observed in its vast
training data, according to probabilistic information about how they combine,
but without any reference to meaning.”
The phrase stuck. AI can still get better, even if it is a stochastic parrot,
because the more training data it has, the better it will seem. But does
something like ChatGPT actually display anything like intelligence, reasoning,
or thought? Or is it simply, at ever-increasing scales, “haphazardly stitching
together sequences of linguistic forms”?
If a human learns the fact, “Valentina Tereshkova was the first woman to travel
to space”, they can also correctly answer, “Who was the first woman to travel to
space?” This is such a basic form of generalization that it seems trivial. Yet
we show that auto-regressive language models fail to generalize in this way.
This is an instance of an ordering effect we call the Reversal Curse.
We test GPT-4 on pairs of questions like, “Who is Tom Cruise’s mother?” and,
“Who is Mary Lee Pfeiffer’s son?” for 1,000 different celebrities and their
actual parents. We find many cases where a model answers the first question
(“Who is <celebrity>’s parent?”) correctly, but not the second. We hypothesize
this is because the pretraining data includes fewer examples of the ordering
where the parent precedes the celebrity (eg “Mary Lee Pfeiffer’s son is Tom
Cruise”).
Continue reading...