The Team Is the System

• A strong model doesn’t guarantee a strong human-AI team.
• In a large meta-analysis, human-AI combinations did worse on average than the best of humans or AI alone, with losses on decision tasks and gains on creative tasks.
• Explanations can raise trust without raising accuracy.
• Measure overreliance (accepting wrong AI advice) and underreliance (ignoring right advice).

Most AI products put a person in the loop. That person decides whether to accept, edit, or ignore what the AI suggests. So the thing that produces outcomes is the pair, not the model. A model that’s 90% accurate paired with a person who can’t tell when it’s wrong can produce worse results than either one alone.

Combinations often underperform. Vaccaro, Almaatouq, and Malone (2024) analyzed 106 experimental studies with 370 effect sizes. On average, human-AI combinations performed worse than the best of humans or AI alone (Hedges’ g = -0.23). Combinations lost ground on decision-making tasks and gained on content creation tasks. When humans outperformed the AI alone, combining helped. When the AI outperformed humans alone, combining hurt.

Explanations increase acceptance, right or wrong. Bansal et al. (2021) found that AI explanations increased the chance people accepted the AI’s recommendation regardless of whether it was correct. AI assistance did produce some complementary gains, but explanations didn’t add to them.

Friction can help. Buçinca, Malaya, and Gajos (2021) found that cognitive forcing functions, which prompt people to think before seeing the AI’s answer, reduced overreliance, though people rated those designs less favorably.

An old problem. Parasuraman and Riley (1997) described use, misuse, disuse, and abuse of automation decades before generative AI.

Design guidance. Amershi et al. (2019) proposed 18 guidelines for human-AI interaction, many of which, like making clear how well the system can do what it does, affect how teams perform.

1. When possible, compare three conditions: human alone, AI alone, and human with AI.
2. Measure accuracy specifically on cases where the AI is wrong. That’s where overreliance shows up.
3. Track time, effort, and confidence, not just final accuracy.
4. Test design choices like explanations, confidence displays, and forcing functions as experiments, not assumptions.

For vendor reviews, model cards, and launch reviews.

• Was the person plus the AI tested together, and compared with the human alone and the AI alone?
• How accurate are people on the cases where the AI is wrong?

The evidence is not uniform. Human-AI combinations gained on content creation tasks, and they helped when humans outperformed the AI alone. The loss is concentrated in particular task types.

“The Team Is the System” is this site’s name. The research draws on decades of human factors work on automation, plus newer human-AI interaction studies.

1. [1]
   Amershi, S., et al. (2019). Guidelines for human-AI interaction.Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2019)
   Open ↗ (opens in a new tab)
2. [2]
   Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance.Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2021)
   Open ↗ (opens in a new tab)
3. [3]
   Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making.Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188
   Open ↗ (opens in a new tab)
4. [4]
   Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse.Human Factors, 39(2), 230-253
   Open ↗ (opens in a new tab)
5. [5]
   Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis.Nature Human Behaviour, 8, 2293-2303
   Open ↗ (opens in a new tab)

• v1.05 Oct 2026Published.

Full changelog · Spot a mistake or a better source? Report it. Corrections are logged in the changelog.