Mislearning and Overconfidence
Modeling how agents learn the wrong lessons and trust them too much#
Mislearning occurs when agents update beliefs or behaviors in ways that increase confidence while decreasing accuracy.
Overconfidence emerges when partial success, social reinforcement, or identity protection masks underlying error.
Together, they explain why intelligent agents persist in failure.
Purpose#
This module exists to:
- model false learning trajectories
- explain confidence divorced from competence
- capture institutional and cultural blind spots
- support delayed collapse and sudden failure
- prevent agents from converging on truth by default
Mislearning is not a bug.
It is a structural feature of bounded cognition.
Mislearning as Substrate Expression (S / E / R)#
Structure (S)#
- flawed mental models
- brittle heuristics
- institutional dogma
- narrative shortcuts
Activation (E)#
- success reinforcement
- stress‑induced simplification
- social validation
- threat‑driven certainty
Relational Time (R)#
- reinforcement lag
- delayed feedback
- generational transmission
- error accumulation
Mislearning compounds quietly over time.
Common Mislearning Pathways#
1. Success‑Based Mislearning#
Mechanism: early success reinforces incorrect causal models
Outcome: confidence grows faster than understanding
Winning for the wrong reason is dangerous.
2. Overgeneralization#
Mechanism: narrow lessons applied too broadly
Outcome: brittle strategies fail under novelty
What worked once becomes doctrine.
3. Identity‑Protected Error#
Mechanism: beliefs shielded from correction by identity
Outcome: evidence is reinterpreted or ignored
Error becomes loyalty.
4. Socially Reinforced Falsehood#
Mechanism: group validation outweighs feedback
Outcome: synchronized mislearning
Groups can learn together — incorrectly.
5. Institutional Lock‑In#
Mechanism: procedures persist despite mismatch
Outcome: adaptation lags reality
Institutions remember success longer than relevance.
Overconfidence Dynamics#
Overconfidence increases when:
- feedback is delayed
- failure is externalized
- dissent is punished
- narratives explain away anomalies
Confidence is cheaper than correction.
Mislearning and Collapse#
Mislearning contributes to collapse by:
- masking early warning signals
- narrowing perceived option space
- accelerating commitment to failing paths
- delaying corrective action
Collapse often arrives after peak confidence.
Mislearning Metrics (Simulation Hooks)#
Trackable indicators include:
- confidence‑accuracy divergence
- error persistence rate
- dissent suppression index
- narrative rigidity
- correction latency
These metrics predict fragility under shock.
Failure Modes#
Mislearning modeling fails when:
- agents always self‑correct
- confidence tracks accuracy
- dissent is always effective
- institutions abandon doctrine easily
Error must be sticky and rewarded.
Integration Notes#
Mislearning and overconfidence:
- distort learning curves
- harden identity
- polarize social interaction
- delay regime transition
This module explains why systems fail loudly after succeeding quietly.
Status#
Canonical mislearning and overconfidence framework for cognitive agent simulation.
Designed for individual, institutional, and civilizational agents.