Overall-IQ reliability and measurement error
The Japanese version of BrainTypeIQ analyzed reliability using 841 eligible first-time completers. They were selected from more than 2,000 completed test records after checks for response quality and repeat attempts.
| Statistic | Estimate | 95% confidence interval | What it mainly indicates |
|---|---|---|---|
| Composite reliability for overall IQ | .929 | .924–.932 | Precision when information from all nine tasks is combined into overall IQ |
| Omega hierarchical (ωH) for the general factor | .834 | .815–.852 | The common general cognitive factor across the nine tasks |
| Standardized Cronbach's alpha across nine tasks | .845 | .828–.860 | Consistency across the nine-task set |
| Standard error of measurement for overall IQ | About 4.0 points | — | Expected score imprecision on the IQ scale |
Overall IQ combines information from all nine tasks, so it provides a more stable estimate than an individual task or a domain based on only one or two tasks. SEM expresses imprecision on the same scale as the reported IQ; it does not mean that every score moves by that amount or that a person's “true IQ” must fall within that distance of the observed result.
The result was reproduced in the earlier and later halves
Participants were divided into earlier and later groups by testing period to check whether consistency across the nine tasks depended on one particular period of data collection.
| Data set | Standardized alpha across nine tasks |
|---|---|
| Earlier 420 participants | .844 |
| Later 421 participants | .846 |
The earlier and later estimates were nearly identical to the full-sample estimate in the opening table. A sensitivity analysis using a different treatment of repeat attempts also left the central conclusion unchanged.
The Japanese-version norming analysis uses the same quality-checked participants, so the comparison basis and reliability estimates come from the same sample.
Reliability estimates are also available for the five domains
The current task structure and scoring model produced the following estimated domain reliabilities.
| Domain | Contributing tasks | Estimated reliability |
|---|---|---|
| Gc | Vocabulary and Analogies | .852 |
| Gf | Matrix Reasoning and Figure Weights | .834 |
| Gv | Paper Folding and Visual Puzzles | .789 |
| Gwm | Computation Span and Visual Reversal | .726 |
| Gs | Symbol Search | .734 |
The domain estimates are lower than the overall estimate because overall IQ combines nine sources of information, while each domain is based on one or two tasks.
Domain scores are best used to see broad patterns in a cognitive profile. A difference of only a few points should not, by itself, define a strength or weakness. The size of the difference, the precision of each domain estimate, and whether the pattern matches everyday experience all matter.
How to interpret these estimates
Some tasks stop when performance meets a stopping rule, while others measure how much a person processes within a time limit. The analysis therefore used task-level models suited to those administration methods and combined their results through the current scoring method. The overall coefficient is model-based composite reliability, not test-retest reliability over time or agreement with WAIS.
Use overall IQ as the most stable estimate, allow for its reported SEM, and expect domain scores to contain more error. Substantial patterns across domains are more informative than small point differences, especially when they match the conditions in which strengths and demands appear in daily life. Reliability describes consistency; norming defines the comparison group, and validity addresses whether a score supports a particular meaning and use.
For English-language results, BrainTypeIQ currently applies a provisional scoring model and is building a dedicated English dataset.