The Clever Hans Effect

• Models learn whatever pattern predicts the label, including patterns nobody intended.
• These shortcuts look like skill on a test built the same way as the training data.
• They fall apart the moment the shortcut isn’t there.
• Test with cases designed to break the likely shortcut.

Clever Hans was a horse in early 1900s Germany that seemed to do arithmetic by tapping its hoof. An investigation showed Hans was reading tiny, unconscious cues from the people asking questions. Hans was smart, just not at math.

Models do the same thing. If every photo of a boat has water in it, a model can learn “water” instead of “boat.” Geirhos et al. (2020) call this shortcut learning: decision rules that perform well on standard benchmarks and fail to transfer to harder conditions, like the real world.

The watermark detector. Lapuschkin et al. (2019) used explanation methods on an image classifier trained on the PASCAL VOC dataset and found it identified horses partly by a copyright tag that appeared on many horse photos.

Answers without the question. Gururangan et al. (2018) showed that in natural language inference datasets, a model given only the hypothesis sentence, without the premise, could predict the label far above chance. Annotators had left telltale word patterns, like negation words in contradictions.

Heuristics, not understanding. McCoy, Pavlick, and Linzen (2019) built the HANS test set and found that inference models relied on shallow heuristics like word overlap, and failed badly on examples where those heuristics give the wrong answer.

Test behavior, not just accuracy. Ribeiro et al. (2020) introduced CheckList, a behavioral testing approach borrowed from software engineering, and found critical failures in commercial and research models that had high benchmark accuracy.

1. Ask, “What’s the laziest way to score well on this test?” Then build items where that way fails.
2. Run a partial-input baseline. Remove the part of the input that should matter and see if accuracy stays high.
3. Create contrast pairs: minimal edits that should flip the answer, and edits that shouldn’t change it.
4. Look at explanations or reasoning for a sample of correct answers, not just wrong ones.

For vendor reviews, model cards, and launch reviews.

• Was a partial-input baseline run to see whether accuracy stays high without the part of the input that should matter?
• Do contrast pairs or behavioral tests show the model gets the right answers for the right reasons?

A model that uses a shortcut may still be good enough for your use. Shortcuts matter when conditions change or when the claim is about understanding. Behavioral tests help you tell which case you are in.

The horse was investigated by psychologist Oskar Pfungst, and “Clever Hans” became shorthand in psychology for experimenter cues. Lapuschkin et al. (2019) brought the term into machine learning with “Clever Hans predictors.” Geirhos et al. (2020) unified related findings under “shortcut learning.”

1. [1]
   Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut learning in deep neural networks.Nature Machine Intelligence, 2, 665-673
   Open ↗ (opens in a new tab)
2. [2]
   Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S. R., & Smith, N. A. (2018). Annotation artifacts in natural language inference data.Proceedings of NAACL-HLT 2018
   Open ↗ (opens in a new tab)
3. [3]
   Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn.Nature Communications, 10, 1096
   Open ↗ (opens in a new tab)
4. [4]
   McCoy, R. T., Pavlick, E., & Linzen, T. (2019). Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference.Proceedings of ACL 2019
   Open ↗ (opens in a new tab)
5. [5]
   Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020). Beyond accuracy: Behavioral testing of NLP models with CheckList.Proceedings of ACL 2020
   Open ↗ (opens in a new tab)

• v1.05 Oct 2026Published.

Full changelog · Spot a mistake or a better source? Report it. Corrections are logged in the changelog.