Reviving Sampling Theories of Intelligence

In my last post, I examined developmental and network models of intelligence as alternatives to ‘g‘ theory as explanations for the positive manifold. However, there are other alternatives as well: the sampling model proposed by Thomson (Thomson 1916) has long been pointed out by critics of ‘g‘ theory as something to fill the void (Shalizi 2007). Essentially, Thomson’s model proposes that there are many different uncorrelated abilities, and the fact that a positive manifold is observed is not reflective of an underlying “general” ability or anything of the sort, but because tests share/tap into these particular neural/cognitive resources/abilities.

The usual arguments brought up against sampling theory come from Jensen (1998):

But there are other facts the overlapping elements theory cannot adequately explain. One such question is why a small number of certain kinds of nonverbal tests with minimal informational content, such as the Raven matrices, tend to have the highest g loadings, and why they correlate so highly with content-loaded tests such as vocabulary, which surely would seem to tap a largely different pool of neural elements. Another puzzle in terms of sampling theory is that tests such as forward and backward digit span memory, which must tap many common elements, are not as highly correlated as are, for instance, vocabulary and block designs, which would seem to have few elements in common. Of course, one could argue trivially in a circular fashion that a higher correlation means more elements in common, even though the theory can’t tell us why seemingly very different tests have many elements in common and seemingly similar tests have relatively few.

Let’s break these parts down one by one:

One such question is why a small number of certain kinds of nonverbal tests with minimal informational content, such as the Raven matrices, tend to have the highest g loadings

Despite the claim that “Ravens matrices … tend to have the highest g loadings”, recent research has shown to be false (Gignac 2015). It actually ends up to be one of Jensen’s many falsehoods.

and why they correlate so highly with content-loaded tests such as vocabulary, which surely would seem to tap a largely different pool of neural elements

This claim is also false. The correlation between Raven’s matrices and tests like vocabulary are only around 0.48, indicating about 23% of shared variance per Johnson & Bouchard (2011). The tests that correlated highest with Raven’s matrices are the tests with the most similar content, such as block design, paper folding and hidden patterns tests. However, the correlations are broadly similar for Raven’s, making it difficult to make specific claims. Regardless, the fact is that we can’t presume the neural elements drawn upon by particular tests are reflected in the actual content or form of the test.

Another puzzle in terms of sampling theory is that tests such as forward and backward digit span memory, which must tap many common elements, are not as highly correlated as are, for instance, vocabulary and block designs, which would seem to have few elements in common

Johnson and Bouchard report correlations of vocabulary and block design ranging from .39 to .43, indicating that they are quite small. As for the forward and backward digit span memory, this is actually quite an interesting area of research. Factor analytic methods suggest that the abilities are broadly similar (Colom et. al 2005; Engle et. al 1999), but experimental research suggests that the difference is that backwards recall employs the visuospatial resources, while forwards recall does not (Clair-Thompson & Allen 2013). Moreover, the small correlation observed (which I cannot confirm at this time) may be the result of the extremely restricted range of the digit span tests (Miller 1956).

Of course, one could argue trivially in a circular fashion that a higher correlation means more elements in common, even though the theory can’t tell us why seemingly very different tests have many elements in common and seemingly similar tests have relatively few.

The issue with Jensen’s critique here is that the explanation that sampling theorists would offer to respond to this is not the argument he proposes they would. Sampling theorists might argue that there isn’t a need for an explanation of any set of particular correlations between tests, or that the proper mode of investigation isn’t to employ subjective and ad hoc interpretations of the elemental similarity based on content similarity, but rather to employ factor analytic methods and/or neurocognitive research.

And how would sampling theory explain the finding that choice reaction time is more highly correlated with scores on a nonspeeded vocabulary test than with scores on a test of clerical checking speed

It is unclear where Jensen is getting his sources from, but the choice reaction time research is quite supportive of a sampling model (Neubauer & Knorr 1997). However, we should be cautious in interpreting the correlations of choice RT tasks to other IQ subtests, as choice RT tasks can vary significantly (Liewald 2013). Given other issues like age differences (Der et. al 2017), and task complexity (Proctor & Schneider 2018), we should be cautious in interpreting the alleged finding.

When people actually do end up testing the actual predictions of the sampling theory, they end up being empirically confirmed (Rabaglia 2012). Modern versions have reworked the mathematical and empirical foundations of the theory to demonstrate that the alleged evidence mounted against it in the past are not so much disconfirmations as non-sequiturs (Bartholomew et. al 2009). Other research has extended the sampling theory into the neurocognitive domain (Kovacs & Conway 2016), while Detterman’s older theories (Detterman 1987, 2000) has been suggested to be similar to sampling and the new process overlap theory (Detterman et. al 2016).

Conclusion

Whether or not you’re an advocate of ‘g‘ theory, dynamic mutualism, or reject ‘g‘ altogether, the sampling model is a model to contend with. It does seem to have some empirical support, particularly with the widespread notion of different cognitive processes in cognitive psychology. While it may have some things to work out, it certainly has not been “refuted”, much less to the extent that Spearman’s g theory has.