Path: nntp.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: JAB <here@is.invalid>
Newsgroups: misc.news.internet.discuss
Subject: potemkin understanding
Date: Thu, 03 Jul 2025 18:51:46 -0500
Organization: A noiseless patient Spider
Lines: 33
Message-ID: <104752l$e8qf$1@dont-email.me>
Reply-To: JAB <here@is.invalid>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 04 Jul 2025 01:51:49 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="97b0fe2729d5beac956eb2210277555d";
	logging-data="467791"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1+NTspiXX3Y8dX/dTSnFWue"
User-Agent: ForteAgent/8.00.32.1272
Cancel-Lock: sha1:Vc3QQ/uFQVEf7/SmwvuNyck6xBs=

AI models just don't understand what they're talking about

Researchers find models' success at tests hides illusion of
understanding

Researchers from MIT, Harvard, and the University of Chicago have
proposed the term "potemkin understanding" to describe a newly
identified failure mode in large language models that ace conceptual
benchmarks but lack the true grasp needed to apply those concepts in
practice.

It comes from accounts of fake villages - Potemkin villages -
constructed at the behest of Russian military leader Grigory Potemkin
to impress Empress Catherine II.

The academics are differentiating "potemkins" from "hallucination,"
which is used to describe AI model errors or mispredictions. In fact,
there's more to AI incompetence than factual mistakes; AI models lack
the ability to understand concepts the way people do, a tendency
suggested by the widely used disparaging epithet for large language
models, "stochastic parrots."

Computer scientists Marina Mancoridis, Bec Weeks, Keyon Vafa, and
Sendhil Mullainathan suggest the term "potemkin understanding" to
describe when a model succeeds at a benchmark test without
understanding the associated concepts.

https://www.theregister.com/2025/07/03/ai_models_potemkin_understanding/

>AI models just don't understand what they're talking about

Same with current Congressional Republicans....T-Borg's stochastic
parrots