Introduction

In this post I will examine the data on beliefs about climate change, conspiracy theories and free market ideology collected by Stephan Lewandowsky. Lewandowsky conducted an exploratory factor analysis (EFA) followed by Structural Equation Modeling (SEM) on the assumption that established theories of conspiracy theory ideation obtained from prior research could be applied to research in online communities of skeptics and warmists. Lewandowsky collected data from those who accept the science of anthropogenic global warming (AGW), hereafter referred to as warmists, and those who reject it, hereafter referred to as skeptics. Lewandowsky constructed exploratory factors by applying factor analysis to separate sets of variables; I think exploratory factor analysis should be more data driven and, since a dataset of individuals who are active on online communities cannot be assumed to follow the same cognitive patterns as communities from which prior results on cognitive models and conspiracy theories were derived, the data should be examined without constraints on which groups of variables should be associated with factors.

In this post I will show that this simple assumption leads to significant differences in outcome, to very different results about the cognitive framework of skeptics vs. warmists, and to different conclusions about the type of communication strategies that warmists should use to persuade skeptics to change their minds.

This is a long post, but if you skip to the “conclusion” section you will also be able to read a BMJ-style “what is known already” and “what this study adds” section that may help to digest the essential points of the post (and the conclusion should be meaningful to non-statsy people).

Methods

(You can skip this if factor analysis makes your eyes bleed).

The data set was obtained using code available from Steve Mcintyre at Climate Audit. Principal Components Analysis (PCA) was used to extract eigenvectors and eigenvalues from the correlation matrix for all 32 variables, and the values of the first eigenvector (loadings) were examined for information about possible relationships between the variables on this variable. The Kaiser Criterion was applied to eigenvalues extracted from PCA to determine how many factors to retain in factor analysis. Factor analysis was conducted using a varimax rotation (the default in R) with the number of retained factors determined by this Kaiser Criterion. To be clear, this means the core analysis proceeds according to the following stages:

  1. Use PCA to extract eigenvectors and eigenvalues
  2. Examine the loadings of the first eigenvector for descriptive purposes, because they are usually informative
  3. Apply the Kaiser criterion to the eigenvalues obtained from PCA: that is, the number of eigenvalues >1 will determine the number of factors to be extracted from the data
  4. Use factor analysis to extract this number of factors, based on maximum likelihood estimation with a varimax rotation

A variable was considered to load onto a given factor in an explanatory sense if the absolute value of its loading was greater than 0.4. That is, if a variable j loads onto factor r with absolute value <0.4, that variable is not considered to be associated with that factor. The actual values of the factors (the so-called scores) were calculated based on actual loadings, so would include linear combinations of the variables not considered to load onto the factor in an explanatory sense. Some factor analysis techniques reduce the final values of the factors to a straight sum of only those variables that loaded onto the given factor, but for this article I have chosen to use the full linear combination of all variables as identified in the factor analysis. I think this is consistent with Lewandowsky’s approach to calculating factors.

Subjects were defined as warmist or skeptic on the basis of their responses to variables 7 to 10, the global warming questions. Those individuals who scored greater than or equal to 12 on the sum of these questions were considered warmist. That is, skeptics were those who refused to agree with all of questions 7 to 10. Obtained factors were then regressed against this variable to see the relationship between the factors and the AGW allegiance of the respondent.

Factors were interpreted based on their variable loadings, and further exploratory analyses conducted as necessary to explore the difference between factors obtained using this method and those of Lewandowsky.

For sensitivity analysis, factor analysis was repeated based on two possible results of visual inspection of a scree plot of eigenvalues (not shown). Differences between the values of the loadings on factor 1 were checked for all three methods (the Kaiser method and the two possible results of the visual inspection).

All analysis was conducted in R. Code with some descriptive information is linked in the appendix.

Results

(If you don’t understand factor analysis, you can skip reading most of this section. I have tried to include a layperson’s explanation, but it’s very difficult to give a layperson’s explanation of factor analysis so it may not be adequate).

(If you do want to skip the minutiae of the results, there is also a section here comparing Lewandowsky’s factors and mine, which is potentially informative).

There were 32 variables in the dataset and 1145 observations. PCA identified five eigenvalues with values greater than one, and the associated eigenvectors explained 60% of the variance in the data (which is not really very good for physical sciences, but pretty good for a psychological survey). The eigenvector of the first principal component contained large positive values (ranging from about 0.1 to 0.3) for the global warming and science variables, and negative values (ranging from about -0.1 to about -0.3) for the free market variables. That is, the first principal component measures a contrast between endorsement of global warming and other science-related variables, and endorsement of free market variables.

Based on these results, factor analysis was conducted with five factors and a varimax rotation. The values of the loadings for each variable on each factor are shown in table 1. Variables whose loading has an absolute value larger than 0.4 are shown in bold. Variables with negligible loadings after rotation are given a blank value in the table, consistent with output from R.

Table 1: Loadings for a five factor exploratory factor analysis of the Lewandowsky data

Variable

Loadings

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

FMUnresBest

0.67

 

 

 

0.37

FMNotEvQual

0.24

0.10

 

 

 

FMLimitSocial

0.54

 

-0.13

 

0.19

FMMoreImp

0.68

 

 

 

0.34

FMThreatEnv

0.74

 

-0.11

 

0.42

FMUnsustain

0.74

 

-0.10

 

0.39

CO2TempUp

-0.86

 

0.21

 

0.14

CO2AtmosUp

-0.92

 

0.16

 

0.14

CO2WillNegChange

-0.93

 

0.11

 

 

CO2HasNegChange

-0.87

 

 

 

 

CFCNowOK

0.41

 

 

 

 

AcidRainNowOK

0.49

 

 

 

 

CYNewWorldOrder

0.35

0.51

-0.19

 

 

CYSARS

 

0.59

 

0.13

 

CYPearlHarbor

 

0.63

 

 

 

CYAIDS

 

0.64

-0.22

 

 

CYMLK

 

0.73

 

 

 

CYMoon

 

0.42

 

 

 

CYArea51

 

0.53

-0.14

0.67

 

CYJFK

 

0.68

 

 

 

CY911

 

0.73

-0.10

 

 

CYRoswell

 

0.54

-0.13

0.73

 

CYDiana

 

0.65

-0.11

0.14

 

CYOkla

-0.13

0.41

 

 

 

CYClimChange

0.85

0.11

-0.14

 

 

CYCoke

 

0.51

 

 

 

CauseHIV

-0.26

-0.27

0.54

 

 

CauseSmoke

-0.24

-0.18

0.55

 

 

CauseCO2

-0.86

 

0.27

 

0.10

ConsensHIV

-0.12

 

0.55

 

 

ConsensSmoke

 

 

0.55

 

 

ConsensCO2

-0.69

 

0.27

 

0.11

The factors can be interpreted approximately as follows, based on those variables that load onto a factor with absolute value greater than 0.4:

  • Factor 1 (Free Market-AGW axis): measures conflict between endorsement of global warming and endorsement of free market ideas. Those who agree with one disagree with the other. Note that this factor has a very strong loading from the climate change conspiracy theory variable CYClimChange – this is a crucial point. Those who endorse the free market questions generally disagree with the climate change questions and see AGW theory as a conspiracy, but they do not endorse any other conspiracy theories.
  • Factor 2 (conspiracy theory): a factor measuring endorsement of conspiracy theories, that loads strongly onto all conspiracy theory variables except the climate change conspiracy.
  • Factor 3 (Cause and consensus): a variable measuring agreement with measures of cause and consensus on smoking and HIV. This measures the strength of subjects’ beliefs about HIV and smoking issues, and could probably be seen as an endorsement of broader scientific ideals. Note it is uncorrelated with both the AGW/free market axis and the conspiracy theory factor, but explains only about 4% of the variance in the data
  • Factor 4 (Space Aliens!): this variable measures an additional dimension of conspiracy theory, and marks out those individuals who have a really strong belief in alien conspiracy theories
  • Factor 5 (meaningless): this factor is a meaningless factor with no interpretation, which does not need to be included in the model but was kept because the Kaiser criterion tends to retain too many factors. We ignore it.

The variance explained by each factor is shown in table 2.

Table 2: Variance Explained by Factors

Factor Variance Explained Cumulative Variance
Free market- AGW axis 0.26 0.26
Conspiracy theory 0.15 0.40
Cause and Consensus 0.05 0.46
Space Aliens! 0.04 0.49
Meaningless 0.02 0.51

Comparison of Factors with Lewandowsky’s Constructs

Note that the free market-AGW axis (factor 1) essentially contrasts Lewandowsky’s free market factor and his (separately generated) climate change factor. However, it also includes the climate change conspiracy theory variable, which tends to be endorsed by those who oppose climate change theory and support free market ideals. This climate change conspiracy theory is approximately the fourth most popular (endorsed by 134 individuals) and its inclusion in the free market-AGW axis factor is important. The combination of Lewandowsky’s free market and AGW factors into one factor is important too – in Lewandowsky’s model they were correlated with each other, whereas in this model they form a single factor.

The conspiracy theory factor (factor 2) measures the strength of a respondent’s beliefs about a range of conspiracy theories except the global warming conspiracy, which does not load strongly on this factor. Note that, by design in factor analysis, this factor is uncorrelated with the free market-AGW axis factor. That is, it is unrelated to the factor that measures conflict between free market ideals and global warming theory.

Layperson’s Explanation

When you allow factor analysis to apply to all the variables, rather than applying it to predetermined groups of variables, you get a different relationship between factors to that obtained by Lewandowsky. One factor measures conflict between free market ideology and AGW theory, and one factor measures conspiracies, though just as with Lewandowsky’s work the conspiracy factor does not include AGW conspiracy theory. In fact, those who reject AGW are only more likely to endorse a single conspiracy theory: the AGW one. They are not otherwise more likely to be conspiracy theorists. Note that a simple logistic regression of “AGW rejection” against the various conspiracy theory variables might have identified this.

However, because Lewandowsky has separated the free market and AGW-rejection factors, and hasn’t included the AGW conspiracy theory variable in either of them, both of these factors are now likely to be correlated with the conspiracy theory variable. There is a correlation of -0.15 between the AGW factor and the conspiracy factor in Lewandowsky’s model, and of -0.77 between the free market ideals and AGW factors. I think these correlations work together in the SEM to give Lewandowsky’s results.

Further minor implications of the factor structure

It also appears that HIV denialism (HIVCause) and the cancer/smoking relationship (SmokeCause) are more likely to be endorsed by warmists, but only very weakly (they have weak negative loadings on factor 1). It also appears that those who endorse free market ideals are associated with the new world order conspiracy (it has a loading of 0.35 on factor 1). This is consistent with my personal experience of hanging out with anarchists (who commenter Paul tells me were defined as “left wing” at his university and who did include a few HIV denialists and had kooky ideas about smoking) and also of reading WUWT, where I have read quite a few people endorsing the new world order conspiracy. I’ve never seen a moon landing denialist, but the new world order and AGW denial conspiracies do get an airing over there. I did once see a guy deny special relativity because “you can’t square a speed, man” but I guess he was just commenting while very, very stoned.

Nonetheless, while these implications might be fun for poking fun at our political enemies, they don’t meet our loading criteria (0.4 or above) so we don’t include them in our interpretation of the final factors.

Further exploratory analysis

In this sample, 18% of respondents were found to be skeptics on the basis of their responses to questions 7 to 10. Factor 2 measures conspiracy theories. A linear regression model of factor 2 by whether subjects were skeptics found no relationship between skeptic/warmist position and the conspiracy factor. However, a linear regression of responses to the climate change conspiracy question was highly significant, with those who were warmists having an average value of 1.04 for this variable, while on average the skeptics’ value was higher by 2.76 (t statistic 45.66, p<0.001). Thus, skeptics were highly likely to endorse this conspiracy theory, but showed no significant difference on the conspiracy theory factor.

The two main factors were largely unchanged if factor analysis was conducted with only two or three factors instead of five.

Conclusion

When factor analysis was applied to the entire Lewandowsky dataset without prior assumption about the structure of the underlying constructs, only two important factors were identified. The most important factor measured the contrast between free market ideals and AGW endorsement, and the second measured endorsement of conspiracy theories in general. A fourth factor recorded space alien conspiracy theories. Thus factor 1 represented a contrast between two factors identified as separate by Lewandowsky.

Skeptics are no more likely to endorse conspiracy theories than warmists, except for the AGW conspiracy theory, which they were highly likely to endorse. Because this conspiracy theory is common, if it is included in a factor that measures conspiracy theories it will completely change the factor, and this factor will become statistically significantly different between warmists and skeptics.

Lewandowsky’s exploratory factor analysis assumed three separate factors measuring, separately, AGW endorsement, free market ideology, and conspiracy theories. This has two important implications:

  1. It forces factors to be generated according to pre-existing conceptions of which variables load onto which factors
  2. It does not require factors to be uncorrelated with each other

However, a more data-driven form of exploratory factor analysis that does not make prior assumptions about the structure of the data does not force this association, and leads to a completely different set of conclusions about conspiracy theories and AGW rejection. Specifically, that they are unrelated.

Factor 1, which measures the Free market-AGW axis, provides interesting confirmation of what we all know about the history and state of play of the debate over AGW. We all know that rejection of AGW theory has been driven primarily by some elements of the Republican party and free market think tanks and institutions (like Heartland). It’s therefore not unreasonable to expect that this historical development of the debate has constructed the present context, and leads to a division between free market believers and AGW believers. It’s possible to imagine an alternative universe in which AGW was rejected by unions and socialists on the grounds that it was a capitalist plot to undermine the development of the poor, and in this case we would see the opposite relationship, with those who reject free market ideals also rejecting AGW. Factor 1 in this data is a measurement of the state of play in the current debate, and gives statistical confirmation of what we all know: that those who reject AGW tend to come from a free market perspective (see e.g. Tony Watts on PBS recently whinging about taxes and regulation).

It is also unsurprising that the AGW conspiracy theory is endorsed by those who reject AGW theory. They know a lot of people and scientists agree with the theory, so obviously in rejecting it they will be likely to see it as a conspiracy.

This data also provides information for warmists who want to convince skeptics that they are wrong. Skeptic beliefs on AGW are closely linked to free market ideology, which is currently suffering stresses under the collapse of the neo-liberal global order after the global financial collapse. If warmists want to change skeptics’ minds, they need to target this nexus of free market ideology and AGW rejection, find models of response to AGW that can use free market ideals (such as carbon pricing mechanisms), more closely attack the failings of the free market ideology, point to past successes of free markets in dealing with crises, and separate the issue of the economic response from the science of the problem.

The factor analysis presented here suggests that AGW rejection is associated with a specific ideological landscape, and contrary to Lewandowsky’s findings, that it has no particular relationship with any conspiracy theory except the obvious: those respondents who thought AGW was not true were very likely to believe the science of AGW was part of a conspiracy theory or a hoax. They did not see this conspiracy theory in the same way as other conspiracy theories. Lewandowsky’s findings of a free market/ AGW rejection effect are robust, but his conspiracy findings only arise from the placement of restrictions on the initial factor selection method, and do not represent the true subtlety of skeptic ideology, which can be strongly pro-science in other areas but strongly paranoid about AGW. It is better to attack this specific thinking, which is often ideologically driven by factors external to the science, than to view skeptics as having a conspiratorial mindset in general.

What is already known on this topic

Lewandowsky’s research has shown an association between rejection of AGW theory and free market ideals. His finding that AGW skeptics also endorse conspiracy theories (or have a conspiratorial mindset) is controversial, and has been disputed on the basis of his data selection methods and data sources. His analysis has also been questioned because it assumes prior theory about cognitive models in exploratory factor analysis.

What this analysis adds

This analysis shows that making assumptions about which factors to construct generates a spurious association between conspiracy theory ideation and AGW rejection. This analysis shows that a more data-driven exploratory factor analysis does not associate conspiracy theory ideation with AGW rejection, but does confirm that those who reject AGW are likely to see AGW science as a conspiracy theory.

A note on comments about the analysis

I don’t believe that Lewandowsky’s decision to use pre-determined variables to construct factors was a deliberate attempt to smear skeptics. I think it’s a defensible decision to construct a factor set based on previous research and theory. I’m also not an expert on the underlying theory and philosophy of factor analysis, but I think the online skeptic/warmist community should be seen as sufficiently unique that it deserves its own, data-driven exploratory factor analysis. In this sense I think Lewandowsky made a mistake but I don’t think this should be read as meaning I am competent and he is not (or vice versa). It simply represents a disagreement about the framework from which to approach factor analysis. I think that this situation could potentially make a nice teaching example of the effect of forcing data to fit a pre-existing model when it is actually from a sample with a different underlying structural relationship between variables. Unfortunately, factor analysis includes a lot of art and these decisions can never be said to be wrong or right in any strict sense. So in comments here I am not interested in entertaining accusations of incompetence or deliberate manipulation of the analysis, either about me or Lewandowsky.

Appendix: Code

I don’t usually make my programming available when I report analyses on this blog, but in this case I will. It should not be necessary to read the code to reconstruct the methods as described in this post, but in case readers want to the code is available here. The code is divided into sections, which can be selected by changing the value of the variable contr.var. Set this variable to 1 to import the data, 2 to do basic pca, 3 to repeat pca on centred variables (this is completely irrelevant), 4 to do factor analysis with 2, 3 or 5 retained factors, and 5 to explore the differences between Lewandowsky’s factors and mine. In asking questions about code, please put the code indented in a comment, but first please try and read around it and read the comments – I have commented the code extensively so hopefully you can figure it out if you read and infer.

Most of this analysis was done while offline and I couldn’t check other people’s methods or results, so if you see obvious mistakes about variable choices or definitions, please tell me.

Update

It has been pointed out to me in comments that Lewandowsky excluded the AIDS and AGW conspiracy theories from his conspiracy theory factor. I’ve changed the post to reflect that, and also added a brief comparison of the correlations between his three factors and mine. It doesn’t change the overall story, but IMO makes his results more robust.

Advertisements