How I Know What Is True

Trying Hard to Break My Ideas

Also called: Falsifiability and adversarial testing

Established idea
Philosophical question
Working interpretation

I trust an idea more when I have actively looked for the situations that would break it. Testing an idea I like against the cases most likely to break it is a way of caring about getting it right.

What experiment would make me give up this explanation?

Why it attracts me

Finding evidence for an idea I already like is easy. The harder habit, and the more useful one, is to go looking for the case that would break it. I trust ideas more after they have survived that search.

The idea

A claim can only be tested if some result could count against it. Adversarial testing turns that into a practice: write down what would prove you wrong, then go and look for it. Often a second person or a separate program plays the attacker, because it is hard to attack your own work with full effort.

An example

The Kryptos K4 side quest has a clear case. A publicly proposed decoder for the unsolved K4 passage produced a readable message, which looked like strong evidence. An independent rebuild found that the decoder used more adjustable table entries than the 97-letter passage could pin down. With those tables left free, it could fit the proposed message, two other English messages and 200 out of 200 random strings built around the known clue words. A method that can fit almost anything tells you very little when it fits the answer you hoped for. The project now runs this test on any claimed solution before spending real effort on it.

Where it connects

Hostile testing and formal proof checking (Proofs a Computer Can Check) strengthen claims in different ways. One hunts for cases that break the claim; the other checks every logical step. Reproducible experiments (Results That Hold Up When Rerun) invite other people to try to break a result. Together they give Proof Versus Evidence its teeth.

Questions I'm still exploring

Which of my favorite ideas have I never tried to break?
How do I test beliefs about meaning that no experiment can reach?
Who could attack my reasoning more honestly than I can?

Sources and further reading

Stephen Thornton, "Karl Popper", Stanford Encyclopedia of Philosophy
Karl Popper, The Logic of Scientific Discovery (1959)

Working interpretation: drafted from my notes and interests for review. It is not a direct quotation, and I may still change it.

In short

An idea is falsifiable if some possible observation could show it is wrong. The philosopher Karl Popper argued that this is what separates scientific claims from other kinds. Adversarial testing puts the idea into practice: instead of collecting examples that agree with an idea, you go looking for the cases most likely to break it. An idea that survives serious attempts to break it earns more trust than one that was only ever confirmed.

Why this matters to me

It is easy to find support for an idea I already like. I would rather find the weak point myself than have it found later by someone else, or by reality. In AI-assisted research this matters even more, because a system can produce convincing output very quickly. I see hostile testing as a form of respect for an idea. If it is good, it can take the pressure.

Connections

Proofs a Computer Can CheckChecks

Trying to break a claim and checking a proof step by step both strengthen it, in different ways: one hunts for failures, the other confirms every step.

Results That Hold Up When RerunPractice

Making results easy to rerun is one way of inviting other people to try, independently, to break them.

Reasoning From the Ground UpPart of

A conclusion rebuilt from the ground up should then be attacked. Looking hard for the case that breaks it is how I check the rebuild.