Congruence bias
Testing only your favored hypothesis, never the alternative.
What it means
The tendency to test a hypothesis exclusively through experiments or queries that would confirm it if true, while neglecting tests that could discriminate it from rival explanations. Unlike confirmation bias broadly, congruence bias is specifically about the structure of the test people choose: they ask whether the expected result occurs under their pet theory, not whether some other theory predicts the same result. As a consequence, even a 'successful' test provides little evidence, because it was never set up to fail. The classic demonstration is people's strategy on rule-discovery tasks, where they propose only instances consistent with their guessed rule and so rarely uncover it. The bias matters in science, debugging, medical diagnosis, and self-assessment, where genuinely informative inquiry requires designing tests that the favored hypothesis could plausibly flunk.
Positive testing and when it misleads
The behavior looks less strange once you see it as a positive test strategy: people probe cases they expect to have the target property rather than cases they expect to lack it. Klayman and Ha argued this default is often sensible, since in most everyday environments a hypothesis that keeps surviving positive tests is probably close to right. It misleads under specific conditions, chiefly when the favored hypothesis is narrower than the truth, or the target property is common. Wason's 2-4-6 task is rigged to hit exactly that condition: the real rule, any increasing sequence, is broader than any tidy guess, so every confirming sequence a subject tries slips through and teaches nothing. The problem is not curiosity about your theory; it is the mismatch between test and world.
What the diagnostic studies measured
Baron, Beattie and Hershey put numbers on the effect by asking people which yes-or-no question to pose in diagnostic scenarios, such as a patient or a faulty device, where one leading hypothesis was already in mind. Respondents favored questions whose 'yes' answer fit that hypothesis, even when a different question would better separate it from the rival, and even when the favored question's answer could not change what they would do. The authors named three departures from the normative benchmark of expected information gain: congruence bias, testing the leading hypothesis rather than discriminating between hypotheses; information bias, valuing answers that cannot alter the decision; and certainty bias, overvaluing tests that promise a definite yes or no. Congruence is the structural one: it concerns which test you select, before any data arrive.
Related but distinct
Congruence bias is easy to fold into confirmation bias, but they sit at different stages. Confirmation bias, broadly, is about how you weigh evidence you already hold: reading ambiguous data as support, discounting what cuts against you. Congruence bias is upstream of that. It is a flaw in choosing the test, in posing a question whose likely answer fits your theory regardless of whether rivals predict the same. A perfectly even-handed interpreter can still fall into it by never running the experiment that could split the hypotheses. It is also distinct from the positive test strategy, which merely describes the tendency; congruence bias names the cases where that tendency yields uninformative tests.
Designing tests that can fail
The correction is not to test your hypothesis less, but to test it against something. Before running an experiment, ask what result each rival explanation predicts; if they all predict the same outcome, the test is wasted no matter how it comes out. The strongest move is a diagnostic test, one your favored theory forbids, or one whose answer would point to a different cause. In debugging, that means reproducing the bug under a condition your suspected cause rules out, not only under the one it predicts. In medicine, it means ordering the test that separates the two leading diagnoses rather than the one that merely fits the front-runner. Prompts to consider the alternative work because they force at least one disconfirming question into the plan.
Examples
In Wason's 2-4-6 task, people test their guessed rule ('even numbers ascending') only with sequences that fit it, never trying a case that would expose the simpler true rule ('any increasing sequence').
Convinced that complaints come from slow loading, a product team tests only page-speed fixes and cheers every small uptick, never running the test that would reveal the checkout form is the real problem.
A doctor suspecting an allergy orders the allergy panel, gets a mild positive, and stops — skipping the test that would have distinguished it from the infection producing the very same rash.
A hiring manager sure a candidate is strong asks only questions the candidate will ace, takes the smooth answers as proof, and never poses the problem that would expose a weakness.
A fraud analyst who suspects one seller runs only the queries that would flag that account, finds a few matches, and closes the case without the check that would implicate a whole ring.
First described in Related to Peter Wason's rule-discovery work (1960).
Key references
- Evans, J. St. B. T. (2016). Reasoning, biases and dual processes: The lasting impact of Wason (1960). The Quarterly Journal of Experimental Psychology, 69(10), 2076-2092. doi.org/10.1080/17470218.2014.914547
- Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220. doi.org/10.1037/1089-2680.2.2.175
- Baron, J., Beattie, J., & Hershey, J. C. (1988). Heuristics and biases in diagnostic reasoning: II. Congruence, information, and certainty. Organizational Behavior and Human Decision Processes, 42(1), 88-110. doi.org/10.1016/0749-5978(88)90021-0
- Klayman, J., & Ha, Y.-W. (1987). Confirmation, disconfirmation, and information in hypothesis testing. Psychological Review, 94(2), 211-228. doi.org/10.1037/0033-295X.94.2.211
- Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology, 12(3), 129-140. doi.org/10.1080/17470216008416717