Highly-recommended overview of metacognition in LLMs.
(bookmark it)
Interesting behaviors in LLMs like confidence calibration, self-verification, knowing when to stop, and knowing what you do not know have mostly been studied in isolation.
This survey argues they are facets of one thing, metacognition, and proposes a comprehensive map of it.
The authors taxonomize methods and benchmarks for measuring and evaluating metacognitive abilities in LLMs, then connect those abilities to capability, reliability, and transparency.
As agents take on longer horizons, the ability to monitor and regulate their own reasoning becomes an important way to measure reliability.
Paper: https://arxiv.org/abs/2607.11881
Learn to build effective AI agents in our academy: https://academy.dair.ai/