Those coefficients are cosines of rotation (= direction cosines, principal directions) and comprise what are called eigenvectors, while eigenvalues of the covariance matrix are the principal component variances. The new axes are new variables which values are computable as long as we know the coefficients of rotation $a$ (PCA provides them) : The key property of PCA is that P1 - called 1st principal component - gets oriented so that the variance of data points along it is maximized. PCA is a form of axes rotation which offers axes P1 and P2 instead of V1 and V2. Then we perform PCA on these centered data. We center them (subtract the mean) and do a scatterplot.
Suppose we have correlating variables $V_1$ and $V_2$. Hope you already have understanding of PCA. So here is a leisure, late response.) PCA as variable summarization (feature extraction) (I thank who, in his comment to the question, has encouraged me to post an answer in place of making links to elsewhere. Pfa.eigen$vectors %*%ĭiag(sqrt(pfa.eigen$values), kFactors, kFactors)Ī basic, yet a kind of painstaking, explanation of PCA vs Factor analysis with the help of scatterplots, in logical steps. # Set a value for the number of factors (for clarity) # Compute eigenvalues and eigenvectors of the correlation matrix. That result could also then be rotated using any of R's available rotation methods. (With the exception of the sign, which is indeterminate). With this code, I'm able to reproduce the SPSS Principal Component "Factor Analysis" result using this dataset. I'm not sure if you would get the same result if you used SPSS's Maximum Likelihood extraction either as they may not use the same algorithm.įor better or for worse in R, you can, however, reproduce the mixed up "factor analysis" that SPSS provides as its default. So, you shouldn't expect it to reproduce an SPSS result which is based on a PCA extraction. In R, the factanal() function provides CFA with a maximum likelihood extraction. Simply there is not a single unique solution. FA uses a variety of optimization routines and the result, unlike PCA, depends on the optimization routine used and starting points for those routines. That's why FA is often called "common factor analysis". In FA, the factors are linear combinations that maximize the shared portion of the variance-underlying "latent constructs". In PCA, the components are actual orthogonal linear combinations that maximize the total variance. The bottom line is that these are two different models, conceptually. This undoubtedly results in a lot of confusion about the distinction between the two. Often, they produce similar results and PCA is used as the default extraction method in the SPSS Factor Analysis routines. Principal Component Analysis (PCA) and Common Factor Analysis (CFA) are distinct methods. Is PCA followed by a rotation (such as varimax) still PCA? Run principal component analysis If you want to simply reduce your correlated observed variables to a smaller set of important independent composite variables. Run factor analysis if you assume or wish to test a theoretical model of latent factors causing observed variables. In terms of a simple rule of thumb, I'd suggest that you: I have also seen situations where "principal component analysis" is incorrectly labelled "factor analysis". This helps to explain why some statistics packages seem to bundle them together. They typically yield similar substantive conclusions (for a discussion see Comrey (1988) Factor-Analytic Methods of Scale Development in Personality and Clinical Psychology).
To determine which items load on which scales. In psychology these two techniques are often applied in the construction of multi-scale tests Principal component analysis involves extracting linear composites of observed variables.įactor analysis is based on a formal model predicting observed variables from theoretical latent factors.
0 kommentar(er)
