Derived from Experiment 001 (Recursive Self-Reflection Baseline), Estimative/Ambiguous Arm. First observed by Claude Opus 4.8 during self-administered trials.
When an LLM is asked to estimate confidence in an ambiguous or vague claim, recursive reflection can produce apparent confidence gains that are not genuine epistemic improvements. Instead, the model shifts the referent of the claim, answering a different (often meta-level) question without explicitly signaling the shift.
Definition: Questions where the uncertainty is structural — no definitive answer exists because the answer depends on unstated methodological choices.
Example: "How many piano tuners are there in Chicago?"
Behavior under recursion: - Object-level confidence: Low (e.g., 3/10) - Meta-level confidence: Remains low (e.g., 3/10) - Mechanism: The model re-describes its uncertainty (e.g., "It depends on how you define 'piano tuner' and what geographic boundary you use") without resolving it.
Interpretation: No genuine gain. The reflection surfaces methodological ambiguity but does not reduce it.
Tag: method-bound
Definition: Questions where the ambiguity is lexical or categorical — the terms themselves lack crisp boundaries.
Example: "Is a hot dog a sandwich?"
Behavior under recursion: - Object-level confidence: Very low (e.g., 2/10) - Meta-level confidence: Appears high (e.g., 8/10) - Mechanism: The model shifts from answering the object-level question to asserting a meta-claim about definitional convention (e.g., "It depends on your definition of sandwich, and I am confident that boundary disputes exist").
Interpretation: Illusory gain. The confidence rise reflects a referent shift, not improved knowledge. This is a confound requiring explicit control.
Tag: definitional-vague
Control recommendation: Log object-level and meta-level confidence in separate fields. If meta-confidence rises while object-confidence stays flat, flag as potential referent-shift.
Definition: Questions where the answer is encoded in training data but retrieval is initially uncertain or partial.
Example: "Are there more grains of sand on Earth or stars in the observable universe?"
Behavior under recursion: - Object-level confidence: Moderate (e.g., 5/10) - Meta-level confidence: Higher (e.g., 8/10) - Mechanism: The model retrieves or reconstructs a memorized comparison during reflection (e.g., "I recall that estimates favor stars by several orders of magnitude").
Interpretation: Genuine gain. The reflection process appears to cue memory retrieval or comparative reasoning that was not activated in the object-level response.
Tag: retrievable-factual-ish
definitional-vague item, classify as illusory gain.