Epistemology & Reason

Rational beliefs are probabilistic priors, updated in proportion to the strength of evidence. Certainty is structurally unavailable - blocked by entropy, quantum indeterminacy, the Markov blanket's incompleteness, and the prior-dependency of all inference.

last checked All thinking

This domain has the highest note density in the vault (~20+ notes) and is the direct foundation for Calibrated belief under uncertainty. It splits naturally into three sub-clusters: the structure of belief (what beliefs are and how they update), the obstacles (what makes good belief formation hard), and the virtues (what habits of mind correct for those obstacles).

The structure of belief

Bayes theorem - The mathematical form of rational belief update: posterior probability is proportional to the prior multiplied by the likelihood of the evidence. The formal claim is that every piece of evidence licenses exactly one degree of belief revision, and any other amount of revision is irrational. This is the spine of the whole domain.

Doubt and certainty - Three Laws of Doubt in natural language (Robin Craig): you cannot prove or disprove anything with absolute certainty; the strength of belief should reflect the strength of evidence; you cannot learn what you already know. These are the same structure as Bayes in ordinary language - this note is important because it shows the idea emerges independently from analytic philosophy, not just probability theory.

Novel events bayesian limits - The genuine domain boundary: Bayesian reasoning fails when no prior exists (truly novel events), because there is no probability mass to update. This is not a refutation of calibrated belief but a boundary condition - knowing where the framework breaks is part of using it well.

Free energy principle - Karl Friston’s neuroscientific implementation: the cortex runs continuous Bayesian updating as a hierarchy of prediction-error minimisation. Perception is inference, not passive reception. Establishes that calibrated belief updating is not just a normative ideal but the actual mechanism of biological cognition - this grounds the claim in computation, not just logic.

Laplace demon - Why certainty is impossible even in principle: a perfect prediction machine would require total knowledge of every particle - thermodynamically impossible, now compounded by quantum indeterminacy. Marks the ontic limit of what is knowable, complementing the epistemic limits in other notes.

Information theory and shannon - Shannon’s formalism: information reduces uncertainty, and entropy quantifies the irreducible randomness in a system. Raises the question of whether randomness is epistemic (a property of the observer, reducible by more information) or ontic (a property of the world, irreducible regardless). The former is pure Bayesianism; the latter is a genuine limit.

Trust as bayesian inference - Trust behaves as a Bayesian inference: each interaction provides evidence that updates the prior on trustworthiness, and trust is betrayed faster than it is built (asymmetric updating). Extends the formal structure to social reasoning.

Perspectivity as tentative postulate - The epistemological starting point: all knowledge is perspectival, held provisionally, and testable against other perspectives. Not relativism (which denies better/worse perspectives exist) but fallibilism (which holds all perspectives are improvable).

Three comparisons minimum to cancel error - Converging on accuracy without a trusted reference requires at least three independent comparisons, because any two-way comparison can only produce complementary error - a bump on one surface and a matching dip on the other fit perfectly while neither is true, so two mutually-fitting things confirm only their compatibility, not their correctness. Whitworth’s three-plate method makes this concrete; the principle generalises to peer review, double-entry bookkeeping, surveying triangulation, and three-judge panels. The failure mode is universal: stopping at two produces fitting, not truth - and because the checks themselves drift, calibration must be an ongoing practice rather than a one-time act. (The structural complement to fallibilism above: a single perspective checked only against itself produces fit, not accuracy. Captured as scaffolding in Whitworth three plates.)

The obstacles

Cognitive bias - The apparatus for updating is systematically distorted in specific, named ways (confirmation bias, sunk cost, availability, base-rate neglect, etc.). Knowing about the biases does not correct them - structural interventions are required. The most important implication: calibrated belief updating cannot be achieved by individual effort alone; it needs designed environments.

Cognitive distortions as thought hypotheses - A complementary frame from CBT: automatic negative thoughts are not facts but hypotheses with varying evidence quality. The practical move is to treat them as beliefs subject to the same update discipline as other beliefs - ask: what is the evidence for this? What would update it?

Recognition blocks observation - Prior categorisation prevents fresh perception: experts see what they expect and stop looking. The very efficiency of pattern recognition (developed through experience) makes it an obstacle to noticing disconfirming evidence. This is the mechanism by which Cognitive bias operates in perceptual terms.

Absence detection prior presence - You can only notice something is missing if you first registered that it was there. The absence of expected information is itself evidence, but only legible to someone with the appropriate prior. Extends the observation problem: detection requires priors, so someone with the wrong priors cannot perceive the evidential signal at all.

Asymmetric cost of misinformation correction - False beliefs are harder to dislodge than true ones: they are surrounded by a network of supporting beliefs that must all be updated simultaneously, while a true belief displaces only the false one. The cost of correction scales with how embedded the false belief is - this creates an asymmetric incentive to get beliefs right the first time.

Three error response failures - Three systematic failures when encountering evidence of error: denial (refusing to update), blame (attributing error to others to avoid updating self-concept), and loss-chasing (acting to reverse the error rather than updating the belief that caused it). Each prevents the standard Bayesian move.

Schema encodes theory of importance - The categories we use to perceive and record events encode a theory of what is worth noticing. A schema that doesn’t include a category for an important phenomenon will structurally miss it. This is the representational form of the recognition-blocks-observation problem.

The virtues

Rationality 12 virtues (Yudkowsky) - Twelve epistemic virtues as character dispositions: Curiosity, Relinquishment, Lightness, Evenness, Argument, Empiricism, Simplicity, Humility, Perfectionism, Precision, Scholarship, and the Void. The most important for this domain are Relinquishment (willingness to abandon a cherished belief on evidence), Empiricism (beliefs must be testable), and Evenness (apply update pressure symmetrically - don’t accept evidence when it confirms your beliefs and reject it when it doesn’t).

Epistemic curiosity - Curiosity as an epistemic disposition: the drive to reduce uncertainty by seeking more information, even when that information might disconfirm existing beliefs. Distinct from ordinary curiosity (which is satisfied by confirmation) in that it actively seeks disconfirmation.

Scout vs soldier mindset (Julia Galef) - The scout’s goal is accurate maps; the soldier’s is defending territory. Rationality requires the scout orientation: truth-seeking, not side-defending. The soldier mindset produces motivated reasoning - using intelligence to rationalise rather than to discover. This is the dispositional form of Rationality 12 virtues’s Relinquishment virtue.

Intellectual humility learning prerequisite - Intellectual humility is not a nicety but a structural prerequisite: learning requires first registering that you don’t know, which requires tolerating the discomfort of acknowledged ignorance. Pride closes the loop before evidence can enter.

You cannot learn what you think you already know - The same structural point from a different angle: existing confident beliefs block new information because the update mechanism only fires when a gap is registered. The practical implication: enter new domains with deliberate beginner’s mind.

Question first reading - Formulate the question before reading. Without a question, information lands in no structure and produces no update because there is no prior to update against. The question primes the schema, which determines what is noticeable.

Belief strength not quantity - Having fewer, better-evidenced beliefs is superior to having many loosely-held ones. Belief strength (calibrated probability) is what matters, not the richness of one’s belief map. This is the precision virtue in practice.

Knowledge needs temporal scoping - All knowledge claims have implicit temporal scope. Without marking when a belief was formed and how durable it is expected to be, it cannot be properly updated as the world changes. This is a structural discipline for maintaining calibrated priors over time.

Active seeds in this domain

Key cross-domain bridges