We talk to the Principal Investigator and at this point, we still prefer the two-factor solution. We will talk about interpreting the factor loadings when we talk about factor rotation to further guide us in choosing the correct number of factors. F, the two use the same starting communalities but a different estimation process to obtain extraction loadings, 3.

F, only Maximum Likelihood gives you chi-square values, 4. F, greater than 0. T, we are taking away degrees of freedom but extracting more factors. As we mentioned before, the main difference between common factor analysis and principal components is that factor analysis assumes total variance can be partitioned into common and unique variance, whereas principal components assumes common variance takes up all of total variance i.

For both methods, when you assume total variance is 1, the common variance becomes the communality. The communality is unique to each item, so if you have 8 items, you will obtain 8 communalities; and it represents the common variance explained by the factors or components. However in the case of principal components, the communality is the total variance of each item, and summing all 8 communalities gives you the total variance across all items.

In contrast, common factor analysis assumes that the communality is a portion of the total variance, so that summing up the communalities represents the total common variance and not the total variance. In summary, for PCA, total common variance is equal to total variance explained , which in turn is equal to the total variance, but in common factor analysis, total common variance is equal to total variance explained but does not equal total variance.

The following applies to the SAQ-8 when theoretically extracting 8 components or factors for 8 items:. F, the total variance for each item, 3. F, communality is unique to each item shared across components or factors , 5.

After deciding on the number of factors to extract and with analysis model to use, the next step is to interpret the factor loadings. Factor rotations help us interpret factor loadings. There are two general types of rotations, orthogonal and oblique. The goal of factor rotation is to improve the interpretability of the factor solution by reaching simple structure.

Without rotation, the first factor is the most general factor onto which most items load and explains the largest amount of variance. This may not be desired in all cases. Suppose you wanted to know how well a set of items load on each factor; simple structure helps us to achieve this. For the following factor matrix, explain why it does not conform to simple structure using both the conventional and Pedhazur test. We know that the goal of factor rotation is to rotate the factor matrix so that it can approach simple structure in order to improve interpretability.

Orthogonal rotation assumes that the factors are not correlated. The benefit of doing an orthogonal rotation is that loadings are simple correlations of items with factors, and standardized solutions can estimate unique contribution of each factor. The most common type of orthogonal rotation is Varimax rotation.

We will walk through how to do this in SPSS. First, we know that the unrotated factor matrix Factor Matrix table should be the same. Additionally, since the common variance explained by both factors should be the same, the Communalities table should be the same. The main difference is that we ran a rotation, so we should get the rotated solution Rotated Factor Matrix as well as the transformation used to obtain the rotation Factor Transformation Matrix. Finally, although the total variance explained by all factors stays the same, the total variance explained by each factor will be different.

The Rotated Factor Matrix table tells us what the factor loadings look like after rotation in this case Varimax. Kaiser normalization is a method to obtain stability of solutions across samples. After rotation, the loadings are rescaled back to the proper size. This means that equal weight is given to all items when performing the rotation. The only drawback is if the communality is low for a particular item, Kaiser normalization will weight these items equally with items with high communality.

As such, Kaiser normalization is preferred when communalities are high across all items. You can turn off Kaiser normalization by specifying. Here is what the Varimax rotated loadings look like without Kaiser normalization. Compared to the rotated factor matrix with Kaiser normalization the patterns look similar if you flip Factors 1 and 2; this may be an artifact of the rescaling.

Another possible reasoning for the stark differences may be due to the low communalities for Item 2 0. Kaiser normalization weights these items equally with the other high communality items. In the table above, the absolute loadings that are higher than 0. We can see that Items 6 and 7 load highly onto Factor 1 and Items 1, 3, 4, 5, and 8 load highly onto Factor 2.

Item 2 does not seem to load highly on any factor. In SPSS, you will see a matrix with two rows and two columns because we have two factors. How do we interpret this matrix? How do we obtain this new transformed pair of values? The steps are essentially to start with one column of the Factor Transformation matrix, view it as another ordered pair and multiply matching ordered pairs.

We have obtained the new transformed pair with some rounding error. The figure below summarizes the steps we used to perform the transformation. The Factor Transformation Matrix can also tell us angle of rotation if we take the inverse cosine of the diagonal element. The points do not move in relation to the axis but rotate with it. This makes sense because if our rotated Factor Matrix is different, the square of the loadings should be different, and hence the Sum of Squared loadings will be different for each factor.

However, if you sum the Sums of Squared Loadings across all factors for the Rotation solution,. This is because rotation does not change the total common variance. Looking at the Rotation Sums of Squared Loadings for Factor 1, it still has the largest total variance, but now that shared variance is split more evenly. Varimax rotation is the most popular but one among other orthogonal rotations.

The benefit of Varimax rotation is that it maximizes the variances of the loadings within the factors while maximizing differences between high and low loadings on a particular factor. Higher loadings are made higher while lower loadings are made lower. This makes Varimax rotation good for achieving simple structure but not as good for detecting an overall factor because it splits up variance of major factors among lesser ones. Quartimax may be a better choice for detecting an overall factor.

It maximizes the squared loadings so that each item loads most strongly onto a single factor. Here is the output of the Total Variance Explained table juxtaposed side-by-side for Varimax versus Quartimax rotation. You will see that whereas Varimax distributes the variances evenly across both factors, Quartimax tries to consolidate more variance into the first factor. Equamax is a hybrid of Varimax and Quartimax, but because of this may behave erratically and according to Pett et al.

Like orthogonal rotation, the goal is rotation of the reference axes about the origin to achieve a simpler and more meaningful factor solution compared to the unrotated solution. In oblique rotation, you will see three unique tables in the SPSS output:. Suppose the Principal Investigator hypothesizes that the two factors are correlated, and wishes to test this assumption. The other parameter we have to put in is delta , which defaults to zero. Larger positive values for delta increases the correlation among factors.

In fact, SPSS caps the delta value at 0. Negative delta factors may lead to orthogonal factor solutions. F, larger delta values, 3. The factor pattern matrix represent partial standardized regression coefficients of each item with a particular factor. Just as in orthogonal rotation, the square of the loadings represent the contribution of the factor to the variance of the item, but excluding the overlap between correlated factors.

The more correlated the factors, the more difference between pattern and structure matrix and the more difficult to interpret the factor loadings. From this we can see that Items 1, 3, 4, 5, and 8 load highly onto Factor 1 and Items 6, and 7 load highly onto Factor 2.

Additionally, we can look at the variance explained by each factor not controlling for the other factors. Recall that the more correlated the factors, the more difference between pattern and structure matrix and the more difficult to interpret the factor loadings.

In our case, Factor 1 and Factor 2 are pretty highly correlated, which is why there is such a big difference between the factor pattern and factor structure matrices. The difference between an orthogonal versus oblique rotation is that the factors in an oblique rotation are correlated. The angle of axis rotation is defined as the angle between the rotated and unrotated axes blue and black axes.

The structure matrix is in fact a derivative of the pattern matrix. If you multiply the pattern matrix by the factor correlation matrix, you will get back the factor structure matrix. Performing matrix multiplication for the first column of the Factor Correlation Matrix we get.

Similarly, we multiple the ordered factor pair with the second column of the Factor Correlation Matrix to get:. This neat fact can be depicted with the following figure:. Decrease the delta values so that the correlation between factors approaches zero. T, the correlations will become more orthogonal and hence the pattern and structure matrix will be closer. The column Extraction Sums of Squared Loadings is the same as the unrotated solution, but we have an additional column known as Rotation Sums of Squared Loadings.

This is because unlike orthogonal rotation, this is no longer the unique contribution of Factor 1 and Factor 2. How do we obtain the Rotation Sums of Squared Loadings?

This means that the Rotation Sums of Squared Loadings represent the non- unique contribution of each factor to total common variance, and summing these squared loadings for all factors can lead to estimates that are greater than total variance.

First we highlight absolute loadings that are higher than 0. We see that the absolute loadings in the Pattern Matrix are in general higher in Factor 1 compared to the Structure Matrix and lower for Factor 2. This makes sense because the Pattern Matrix partials out the effect of the other factor.

Looking at the Pattern Matrix, Items 1, 3, 4, 5, and 8 load highly on Factor 1, and Items 6 and 7 load highly on Factor 2. Looking at the Structure Matrix, Items 1, 3, 4, 5, 7 and 8 are highly loaded onto Factor 1 and Items 3, 4, and 7 load highly onto Factor 2. The results of the two matrices are somewhat inconsistent but can be explained by the fact that in the Structure Matrix Items 3, 4 and 7 seem to load onto both factors evenly but not in the Pattern Matrix.

There is an argument here that perhaps Item 2 can be eliminated from our survey and to consolidate the factors into one SPSS Anxiety factor. F, represent the non -unique contribution which means the total sum of squares can be greater than the total communality , 3.

F, this is true only for orthogonal rotations, the SPSS Communalities table in rotated factor solutions is based off of the unrotated solution, not the rotated solution. As a special note, did we really achieve simple structure? Although rotation helps us achieve simple structure, if the interrelationships do not hold itself up to simple structure, we can only modify our model.

In this case we chose to remove Item 2 from our model. Promax rotation begins with Varimax orthgonal rotation, and uses Kappa to raise the power of the loadings. Promax really reduces the small loadings. Suppose the Principal Investigator is happy with the final factor analysis which was the two-factor Direct Quartimin solution.

She has a hypothesis that SPSS Anxiety and Attribution Bias predict student scores on an introductory statistics course, so would like to use the factor scores as a predictor in this new regression analysis.

Since a factor is by nature unobserved, we need to first predict or generate plausible factor scores. In order to generate factor scores, run the same factor analysis model but click on Factor Scores Analyze — Dimension Reduction — Factor — Factor Scores. Then check Save as variables, pick the Method and optionally check Display factor score coefficient matrix.

Here we picked the Regression approach after fitting our two-factor Direct Quartimin solution. After generating the factor scores, SPSS will add two extra variables to the end of your variable list, which you can view via Data View.

These are now ready to be entered in another analysis as predictors. For those who want to understand how the scores are generated, we can refer to the Factor Score Coefficient Matrix.

These are essentially the regression weights that SPSS uses to generate the scores. Using the Factor Score Coefficient matrix, we multiply the participant scores by the coefficient matrix for each column. For the first factor:. This table can be interpreted as the covariance matrix of the factor scores, however it would only be equal to the raw covariance if the factors are orthogonal. For example, if we obtained the raw covariance matrix of the factor scores we would get.

You will notice that these values are much lower. Among the three methods, each has its pluses and minuses. The regression method maximizes the correlation and hence validity between the factor scores and the underlying factor but the scores can be somewhat biased. This means even if you have an orthogonal solution, you can still have correlated factor scores.

Unbiased scores means that with repeated sampling of the factor scores, the average of the scores is equal to the average of the true factor score. The Anderson-Rubin method perfectly scales the factor scores so that the factor scores are uncorrelated with other factors and uncorrelated with other factor scores.

Is it safe to say that factor analysis is the the analysis done in seeking the relationship of demographic and the variables dependent, mediator, moderator in the study? Do help me as I still cant figure out what factor analysis is. Kindly assist. Many thanks. Factor Analysis is a measurement model for an unmeasured variable a construct. Thanks big time. Very nice explanation of factor analysis.

Keep up the nice work. As I have searched many of websites for factor analysis. This was the best and easiest explanation i found yet.

Really helpful! Great attempt! Keep on doing social service! Keep up the good work! Explained in one of the best ways possible!!! Helps you understand by just reading it once quite the contrary for the definitions on the other websites.

Hi Maike, I have a survey with 15 q, 3 measure reading ability, 3 writing, 3 understanding, 3 measure monetary values and 3 measure literacy unrelated aspects. Thanks for your help. Very clear explanation and useful examples. I woudl liek to aks you somehting.

I would like to design a questionnaire using Likert scale that I can use for factor analysis. Let us say I need to find out the view of a student if they have a negative attitude towards learning a subject.

Where you talked about the amount of variance a factor captures and eigenvalue that measures that. Thanks Doc This has been the most understandable explanation I have so far had. You mentioned something about your next post? May you please also talk about factor analysis using R.

Good day to you. I have a question on factor analysis. I have a pool of 30 items for my construct, then I conducted the PCs, with nine items. After conducted the CFA, it only has three items. Does this acceptable? I have two kinds of questions: one with a 5-option response and another with a 7-option one. Can I run exploratory FA on both at the same time?

But, mathematically, is it right? You can then use those combination variables — indices or subscales — in other analyses. Kindly guide me about this I will waiting for your answer. I am grateful to have little idea on how to apply factor analysis. But stil sir! How would I enter data on exel spreat sheet and how will I start running the analysis? I am ph. D student and one of my objective of the study has to do with factor analysis. I have identify four factors with twenty three variable in question.

Pls explain step by step for me. Thanks and best regard. Looking forward to hear from you sir. Thank you very much Dr. I have struggled 13 months to understand Factor Analysis, and this has been the simple and very helpful. Thank you again. Dear Dr Thanks very much for you explanation on factor analysis, even those who beginners in statistics like me can follow your elaborations.

As i am using Factor analysis by SPSS in my master research, i got five factors related to my research. What does this matrix endicated for? Can you help please? Thank you very much for posting it! Hence, your first formula, represents the required info. How can I emerge these values to one value and group each respondent into e. Very clear and useful description, also understandable for non-mathematicians, e. Many thanks for posting this! This was the best and and easiest to understand explanation of Factor Analysis I have found.

I will book mark your page as a future reference. Your email address will not be published. What is a factor? What are factor loadings? Variables Factor 1 Factor 2 Income 0. Principal Component Analysis. Besides that their usage is limited.

Eigenvector value squared has the meaning of the contribution of a variable into a pr. Although eigenvectors and loadings are simply two different ways to normalize coordinates of the same points representing columns variables of the data on a biplot , it is not a good idea to mix the two terms.

This answer explained why. See also. There seems to be a great deal of confusion about loadings, coefficients and eigenvectors. The word loadings comes from Factor Analysis and it refers to coefficients of the regression of the data matrix onto the factors.

They are not the coefficients defining the factors. See for example Mardia, Bibby and Kent or other multivariate statistics textbooks. In recent years the word loadings has been used to indicate the PCs coefficients. Here it seems that it used to indicate the coefficients multiplied by the sqrt of the eigenvalues of the matrix.

These are not quantities commonly used in PCA. The principal components are defined as the sum of the variables weighted with unit norm coefficients. In this way the PCs have norm equal to the corresponding eigenvalue, which in turn is equal to the variance explained by the component.

It is in Factor Analysis that the factors are required to have unit norm. Rotating the PCs' coefficient is very rarely done because it destroys the optimality of the components.

In FA the factors are not uniquely defined and can be estimated in different ways. The important quantities are the loadings the true ones and the communalities which are used to study the structure of the covariance matrix.

I am a bit confused by those names, and I searched in the book named "Statistical Methods in the Atmospherical Science", and it gave me a summary of varied Terminology of PCA, here are the screenshots in the book, hope it will help. There appears to be some confusion over this matter, so I will provide some observations and a pointer to where an excellent answer can be found in the literature. In general, principal components are orthogonal by definition whereas factors - the analogous entity in FA - are not.

Simply put, principal components span the factor space in an arbitrary but not necessarily useful way due to their being derived from pure eigenanalysis of the data.

Factors on the other hand represent real-world entities which are only orthogonal i. Say we take s observations from each of l subjects. These can be arranged into a data matrix D having s rows and l columns. S will have s rows, and L will have l columns, the second dimension of each being the number of factors n. The purpose of factor analysis is to decompose D in such a way as to reveal the underlying scores and factors. The loadings in L tell us the proportion of each score which make up the observations in D.

These are conventionally arranged in descending order of the corresponding eigenvalues. The value of n - i. The columns of S in PCA form the n abstract principal components themselves. The value of n is the underlying dimensionality of the data set. ST is the transformed score matrix, and T -1 L is the transformed loading matrix.

The above explanation roughly follows the notation of Edmund R. Malinowski from his excellent Factor Analysis in Chemistry. I highly recommend the opening chapters as an introduction to the subject.

The loadings show the relationship between the raw data and the rotated data. Try it:. The function princomp returns this in the element loadings.