Bringing Back Rule Induction to Fluid Intelligence Research? An Initial Validation of the ARC-AGI Benchmark in Humans

bringing-back-rule-induction-to-fluid-intelligence-research?-an-initial-validation-of-the-arc-agi-benchmark-in-humans

Bringing Back Rule Induction to Fluid Intelligence Research? An Initial Validation of the ARC-AGI Benchmark in Humans

[Submitted on 13 Jul 2026 (v1), last revised 17 Nov 2026 (this version, v4)]

View PDF

Abstract:Two competing perspectives on fluid intelligence (gf) measures propose that performance is primarily constrained either by working memory capacity or by the ability to induce novel relations. The first perspective is currently dominant in measurement, as evident from the use of a limited set of recurring rules, whereas the second perspective is reflected in many definitions but rarely present in measurement. The ARC-AGI benchmark predominantly requires rule induction and was proposed as a measure of gf for both humans and artificial systems. However, its psychometric properties have not yet been examined in human samples. We therefore investigated the psychometric characteristics and nomological network of ARC-AGI in a first study with 100 participants. A compilation of ARC-AGI items showed good psychometric properties and correlated substantially with figural fluid intelligence as measured by a figural reasoning test ($rho$ = .63). Associations with figural originality were weak. These findings provide initial support for the validity of ARC-AGI as a measure of human fluid intelligence. Future research should include more rule induction tasks as well as additional multivariate covariates. This study is unusual by studying a task in humans that was initially designed for machines. We suggest systematically embedding AI benchmarks into the nomological network of human cognitive abilities to enable more systematic evaluation and interdisciplinary cooperation.

Submission history

From: Jasmin Thelen [view email]
[v1] Mon, 13 Jul 2026 08:45:45 UTC (753 KB)
[v2] Tue, 14 Jul 2026 17:47:10 UTC (753 KB)
[v3] Wed, 15 Jul 2026 12:09:51 UTC (753 KB)
[v4] Tue, 17 Nov 2026 19:50:53 UTC (885 KB)