Publications

Metacognition is a foundational component of intelligence that has become increasingly recognized as a cornerstone of capable, transparent AI systems. While LLMs have made significant progress, it remains unclear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, and how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first systematic, comprehensive review of the current state of knowledge on metacognition for LLMs. See the GitHub repo for the full paper list!

We introduce a novel framework to systematically quantify faithful calibration—the alignment between expressed and intrinsic confidence—of large reasoning models (LRMs) in long-form reasoning settings. We analyze linguistic decisiveness relative to three sources of internal uncertainty, based on token probabilities, hidden states, and sampled response consistency. We also devise a prefix-conditioned sampling approach to control for conditional and structural variation across traces.

We present the first pipeline for automatic, difficulty-controlled creation of uncheatable, realistic, unanswerable, and multi-hop queries (CRUMQs), adaptable to any corpus and domain. This offers a simple way to enhance benchmark difficulty and realism and drive development of more capable RAG systems.

Nugget-based evaluation methods have emerged as the standard for measuring relevance of long-form RAG responses. We argue that nuggets are valuable also for guiding retrieval and generation, and we present a nugget-centric RAG system that automatically constructs its own nugget bank and uses it as a control signal throughout the pipeline.

Evaluation of long-form, citation-backed reports typically depends on use of atomic, QA-based nuggets to assess coverage of query-relevant information. This largely depends on manual curation of sets of nuggets per test topic. To improve scalability, we introduce DoGMaTiQ, a pipeline to generate high-quality QA nugget sets, and integrate it with Auto-ARGUE, a recent nugget-based evaluation framework, to enable fully automatic evaluation of generated reports.

The first systematic study of whether epistemic markers emitted by LLMs consistently and stably reflect their intrinsic confidence. We operationalize marker internal confidence (MIC) as the internal confidence level an LLM associates with a specific epistemic marker, and introduce a suite of 7 metrics to evaluate the in- and out-of-distribution reliability of MICs across diverse models and tasks.

Report generation (RG) is a RAG task that aims to produce a long-form, citation-attributed response to a complex user query. We present the first public, automated, LLM-based implementation of the ARGUE evaluation framework for RG.

LLM benchmarks are essential for tracking progress and ensuring safety in AI, but most benchmarks don’t measure what matters, as suggested by our systematic review of 445 LLM benchmarks from top AI conferences. A taxonomy of these failures was therefore built and translated into an operational checklist to help future benchmark authors demonstrate construct validity.

For LLMs to be deployed reliably and responsibly, it is essential that their linguistically expressed confidence faithfully reflect their internal uncertainty. This paper presents the first study to systematically and comprehensively benchmark faithful calibration of LLMs and proposes MetaFaith, the first method to improve faithful calibration of any instruction-following LLM in a task-agnostic manner.

Multi-document (MD) processing is crucial for LLMs to handle real-world tasks across large sets of documents. This paper introduces MDCure, an effective and scalable procedure for generating high-quality multi-document instruction tuning data to improve MD capabilities of any base LLM.

Metacognition of emotion refers to the ability to evaluate success at identifying one’s own emotional feelings and adjusting them accordingly. This paper establishes the first experimentally-validated procedure to reliably assess and quantify metacognition of emotion.

This paper discusses the social implications of reinforcement learning with human feedback (RLHF), identifying key social and ethical issues and discussing social impacts for stakeholders. Seven impact areas are examined, including misinformation, AI value-alignment, bias, AI access, cross-cultural dialogue, industry, and workforce.

An undergraduate-level curriculum that cultivates understanding of the robot rights debate as informed by Western and Eastern principles, stepping toward filling existing gaps in knowledge and discourse and uniting the global community. Constructs and works through a foundational framework to engage students in critical thinking, understanding, and discussion of decisions regarding robot rights.

We define the Dynamical Chinese Restaurant Process (Dynamical CRP), a novel stochastic process that provides a non-stationary prior over cluster assignments and yields an efficient streaming variational inference algorithm. Experiments show the Dynamical CRP can be applied on diverse synthetic and real data with Gaussian and non-Gaussian likelihoods.

R-IBP is a novel recursive form of the Indian Buffet Process that makes feature models applicable to streaming data. It enables creation of new features online and in a probabilistic, principled manner. As a prior for feature models, R-IBP yields efficient inference over an unbounded number of latent features, with quasilinear average time complexity and logarithmic average space complexity.

The first experimentally quantitative index of metacognition of emotion. Advancing understanding of awareness toward subjective feelings.

Weight friction draws from principles in neuroscience and physics to help mitigate catatstrophic forgetting in neural networks. It operates over multiple task domains and has convergence comparable to SGD.

GFCCs can be powerful speech features for use in speech emotion recognition. This preliminary study suggests they are more suitable versus MFCCs for emotion and intensity classification tasks with neural networks.