Why principal axis factoring

If we had simply used the default 25 iterations in SPSS, we would not have obtained an optimal solution. The elements of the Factor Matrix table are called loadings and represent the correlation of each item with the corresponding factor. Just as in PCA, squaring each loading and summing down the items rows gives the total variance explained by each factor.

Note that they are no longer called eigenvalues as in PCA. This number matches the first row under the Extraction column of the Total Variance Explained table. We can repeat this for Factor 2 and get matching results for the second row. Additionally, we can get the communality estimates by summing the squared loadings across the factors columns for each item.

For example, for Item Note that these results match the value of the Communalities table for Item 1 under the Extraction column. This means that the sum of squared loadings across factors represents the communality estimates for each item. The figure below shows the path diagram of the orthogonal two-factor EFA solution show above note that only selected loadings are shown. The loadings represent zero-order correlations of a particular factor with each item. Looking at absolute loadings greater than 0.

From speaking with the Principal Investigator, we hypothesize that the second factor corresponds to general anxiety with technology rather than anxiety in particular to SPSS. However, use caution when interpretation unrotated solutions, as these represent loadings where the first factor explains maximum variance notice that most high loadings are concentrated in first factor.

In the sections below, we will see how factor rotations can change the interpretation of these loadings. We will use the term factor to represent components in PCA as well. These elements represent the correlation of the item with each factor. Now, square each element to obtain squared loadings or the proportion of variance explained by each factor for each item.

Summing the squared loadings across factors you get the proportion of variance explained by all factors in the model. This is known as common variance or communality, hence the result is the Communalities table. These now become elements of the Total Variance Explained table. Summing down the rows i. In words, this is the total common variance explained by the two factor solution for all eight items.

Equivalently, since the Communalities table represents the total common variance explained by both factors for each item, summing down the items in the Communalities table also gives you the total common variance explained, in this case. True or False the following assumes a two-factor Principal Axis Factor solution with 8 items. F, the sum of the squared elements across both factors, 3. F, the total Sums of Squared Loadings represents only the total common variance excluding unique variance, 7.

F, eigenvalues are only applicable for PCA. To run a factor analysis using maximum likelihood estimation under Analyze — Dimension Reduction — Factor — Extraction — Method choose Maximum Likelihood. Although the initial communalities are the same between PAF and ML, the final extraction loadings will be different, which means you will have different Communalities, Total Variance Explained, and Factor Matrix tables although Initial columns will overlap.

The other main difference is that you will obtain a Goodness-of-fit Test table, which gives you a absolute test of model fit. Non-significant values suggest a good fitting model. Here the p -value is less than 0. In practice, you would obtain chi-square values for multiple factor analysis runs, which we tabulate below from 1 to 8 factors. The table shows the number of factors extracted or attempted to extract as well as the chi-square, degrees of freedom, p-value and iterations needed to converge.

Note that as you increase the number of factors, the chi-square value and degrees of freedom decreases but the iterations needed and p-value increases. Practically, you want to make sure the number of iterations you specify exceeds the iterations needed. In SPSS, no solution is obtained when you run 5 to 7 factors because the degrees of freedom is negative which cannot happen.

The number of factors will be reduced by one. It looks like here that the p -value becomes non-significant at a 3 factor solution. Note that differs from the eigenvalues greater than 1 criterion which chose 2 factors and using Percent of Variance explained you would choose factors.

We talk to the Principal Investigator and at this point, we still prefer the two-factor solution. We will talk about interpreting the factor loadings when we talk about factor rotation to further guide us in choosing the correct number of factors. F, the two use the same starting communalities but a different estimation process to obtain extraction loadings, 3. F, only Maximum Likelihood gives you chi-square values, 4.

F, greater than 0. T, we are taking away degrees of freedom but extracting more factors. As we mentioned before, the main difference between common factor analysis and principal components is that factor analysis assumes total variance can be partitioned into common and unique variance, whereas principal components assumes common variance takes up all of total variance i.

For both methods, when you assume total variance is 1, the common variance becomes the communality. The communality is unique to each item, so if you have 8 items, you will obtain 8 communalities; and it represents the common variance explained by the factors or components.

However in the case of principal components, the communality is the total variance of each item, and summing all 8 communalities gives you the total variance across all items. In contrast, common factor analysis assumes that the communality is a portion of the total variance, so that summing up the communalities represents the total common variance and not the total variance.

In summary, for PCA, total common variance is equal to total variance explained , which in turn is equal to the total variance, but in common factor analysis, total common variance is equal to total variance explained but does not equal total variance. The following applies to the SAQ-8 when theoretically extracting 8 components or factors for 8 items:. F, the total variance for each item, 3.

F, communality is unique to each item shared across components or factors , 5. After deciding on the number of factors to extract and with analysis model to use, the next step is to interpret the factor loadings.

Factor rotations help us interpret factor loadings. There are two general types of rotations, orthogonal and oblique. The goal of factor rotation is to improve the interpretability of the factor solution by reaching simple structure. Without rotation, the first factor is the most general factor onto which most items load and explains the largest amount of variance. This may not be desired in all cases. Suppose you wanted to know how well a set of items load on each factor; simple structure helps us to achieve this.

For the following factor matrix, explain why it does not conform to simple structure using both the conventional and Pedhazur test. Using the Pedhazur method, Items 1, 2, 5, 6, and 7 have high loadings on two factors fails first criterion and Factor 3 has high loadings on a majority or 5 out of 8 items fails second criterion. We know that the goal of factor rotation is to rotate the factor matrix so that it can approach simple structure in order to improve interpretability.

Orthogonal rotation assumes that the factors are not correlated. The benefit of doing an orthogonal rotation is that loadings are simple correlations of items with factors, and standardized solutions can estimate the unique contribution of each factor. The most common type of orthogonal rotation is Varimax rotation. We will walk through how to do this in SPSS. First, we know that the unrotated factor matrix Factor Matrix table should be the same.

Additionally, since the common variance explained by both factors should be the same, the Communalities table should be the same. The main difference is that we ran a rotation, so we should get the rotated solution Rotated Factor Matrix as well as the transformation used to obtain the rotation Factor Transformation Matrix.

Finally, although the total variance explained by all factors stays the same, the total variance explained by each factor will be different. The Rotated Factor Matrix table tells us what the factor loadings look like after rotation in this case Varimax.

Kaiser normalization is a method to obtain stability of solutions across samples. After rotation, the loadings are rescaled back to the proper size. This means that equal weight is given to all items when performing the rotation.

The only drawback is if the communality is low for a particular item, Kaiser normalization will weight these items equally with items with high communality. As such, Kaiser normalization is preferred when communalities are high across all items. You can turn off Kaiser normalization by specifying. Here is what the Varimax rotated loadings look like without Kaiser normalization. Compared to the rotated factor matrix with Kaiser normalization the patterns look similar if you flip Factors 1 and 2; this may be an artifact of the rescaling.

The biggest difference between the two solutions is for items with low communalities such as Item 2 0. Kaiser normalization weights these items equally with the other high communality items.

In the both the Kaiser normalized and non-Kaiser normalized rotated factor matrices, the loadings that have a magnitude greater than 0. We can see that Items 6 and 7 load highly onto Factor 1 and Items 1, 3, 4, 5, and 8 load highly onto Factor 2.

Item 2 does not seem to load highly on any factor. The figure below shows the path diagram of the Varimax rotation. Comparing this solution to the unrotated solution, we notice that there are high loadings in both Factor 1 and 2. This is because Varimax maximizes the sum of the variances of the squared loadings, which in effect maximizes high loadings and minimizes low loadings.

In SPSS, you will see a matrix with two rows and two columns because we have two factors. How do we interpret this matrix?

How do we obtain this new transformed pair of values? The steps are essentially to start with one column of the Factor Transformation matrix, view it as another ordered pair and multiply matching ordered pairs. We have obtained the new transformed pair with some rounding error. The figure below summarizes the steps we used to perform the transformation. The Factor Transformation Matrix can also tell us angle of rotation if we take the inverse cosine of the diagonal element.

Notice that the original loadings do not move with respect to the original axis, which means you are simply re-defining the axis for the same loadings. This makes sense because if our rotated Factor Matrix is different, the square of the loadings should be different, and hence the Sum of Squared loadings will be different for each factor. However, if you sum the Sums of Squared Loadings across all factors for the Rotation solution,. This is because rotation does not change the total common variance.

Looking at the Rotation Sums of Squared Loadings for Factor 1, it still has the largest total variance, but now that shared variance is split more evenly. Varimax rotation is the most popular orthogonal rotation. The benefit of Varimax rotation is that it maximizes the variances of the loadings within the factors while maximizing differences between high and low loadings on a particular factor. Higher loadings are made higher while lower loadings are made lower.

This makes Varimax rotation good for achieving simple structure but not as good for detecting an overall factor because it splits up variance of major factors among lesser ones. Quartimax may be a better choice for detecting an overall factor. It maximizes the squared loadings so that each item loads most strongly onto a single factor.

Here is the output of the Total Variance Explained table juxtaposed side-by-side for Varimax versus Quartimax rotation. You will see that whereas Varimax distributes the variances evenly across both factors, Quartimax tries to consolidate more variance into the first factor.

Equamax is a hybrid of Varimax and Quartimax, but because of this may behave erratically and according to Pett et al. Like orthogonal rotation, the goal is rotation of the reference axes about the origin to achieve a simpler and more meaningful factor solution compared to the unrotated solution.

In oblique rotation, you will see three unique tables in the SPSS output:. Suppose the Principal Investigator hypothesizes that the two factors are correlated, and wishes to test this assumption. The other parameter we have to put in is delta , which defaults to zero. Larger positive values for delta increases the correlation among factors. In fact, SPSS caps the delta value at 0. Negative delta may lead to orthogonal factor solutions. F, larger delta values, 3.

The factor pattern matrix represent partial standardized regression coefficients of each item with a particular factor. Just as in orthogonal rotation, the square of the loadings represent the contribution of the factor to the variance of the item, but excluding the overlap between correlated factors. The figure below shows the Pattern Matrix depicted as a path diagram.

Remember to interpret each loading as the partial correlation of the item on the factor, controlling for the other factor. The more correlated the factors, the more difference between pattern and structure matrix and the more difficult to interpret the factor loadings. Looking at the Factor Pattern Matrix and using the absolute loading greater than 0. In the Factor Structure Matrix, we can look at the variance explained by each factor not controlling for the other factors.

In general, the loadings across the factors in the Structure Matrix will be higher than the Pattern Matrix because we are not partialling out the variance of the other factors. The figure below shows the Structure Matrix depicted as a path diagram.

Remember to interpret each loading as the zero-order correlation of the item on the factor not controlling for the other factor. Recall that the more correlated the factors, the more difference between Pattern and Structure matrix and the more difficult it is to interpret the factor loadings.

In our case, Factor 1 and Factor 2 are pretty highly correlated, which is why there is such a big difference between the factor pattern and factor structure matrices. Observe this in the Factor Correlation Matrix below. The difference between an orthogonal versus oblique rotation is that the factors in an oblique rotation are correlated. The angle of axis rotation is defined as the angle between the rotated and unrotated axes blue and black axes.

The structure matrix is in fact derived from the pattern matrix. If you multiply the pattern matrix by the factor correlation matrix, you will get back the factor structure matrix.

Performing matrix multiplication for the first column of the Factor Correlation Matrix we get. Similarly, we multiple the ordered factor pair with the second column of the Factor Correlation Matrix to get:.

This neat fact can be depicted with the following figure:. Decrease the delta values so that the correlation between factors approaches zero. T, the correlations will become more orthogonal and hence the pattern and structure matrix will be closer. The column Extraction Sums of Squared Loadings is the same as the unrotated solution, but we have an additional column known as Rotation Sums of Squared Loadings. This is because unlike orthogonal rotation, this is no longer the unique contribution of Factor 1 and Factor 2.

How do we obtain the Rotation Sums of Squared Loadings? This means that the Rotation Sums of Squared Loadings represent the non- unique contribution of each factor to total common variance, and summing these squared loadings for all factors can lead to estimates that are greater than total variance. First we bold the absolute loadings that are higher than 0. We see that the absolute loadings in the Pattern Matrix are in general higher in Factor 1 compared to the Structure Matrix and lower for Factor 2.

This makes sense because the Pattern Matrix partials out the effect of the other factor. Looking at the Pattern Matrix, Items 1, 3, 4, 5, and 8 load highly on Factor 1, and Items 6 and 7 load highly on Factor 2.

Looking at the Structure Matrix, Items 1, 3, 4, 5, 7 and 8 are highly loaded onto Factor 1 and Items 3, 4, and 7 load highly onto Factor 2. The results of the two matrices are somewhat inconsistent but can be explained by the fact that in the Structure Matrix Items 3, 4 and 7 seem to load onto both factors evenly but not in the Pattern Matrix. There is an argument here that perhaps Item 2 can be eliminated from our survey and to consolidate the factors into one SPSS Anxiety factor.

F, represent the non -unique contribution which means the total sum of squares can be greater than the total communality , 3. F, this is true only for orthogonal rotations, the SPSS Communalities table in rotated factor solutions is based off of the unrotated solution, not the rotated solution.

As a special note, did we really achieve simple structure? Although rotation helps us achieve simple structure, if the interrelationships do not hold itself up to simple structure, we can only modify our model.

In this case we chose to remove Item 2 from our model. Promax is an oblique rotation method that begins with Varimax orthgonal rotation, and then uses Kappa to raise the power of the loadings. They are the reproduced variances from the factors that you have extracted. You can find these values on the diagonal of the reproduced correlation matrix. Factor — The initial number of factors is the same as the number of variables used in the factor analysis. However, not all 12 factors will be retained.

In this example, only the first three factors will be retained as we requested. Initial Eigenvalues — Eigenvalues are the variances of the factors. Because we conducted our factor analysis on the correlation matrix, the variables are standardized, which means that the each variable has a variance of 1, and the total variance is equal to the number of variables used in the analysis, in this case, Total — This column contains the eigenvalues.

The first factor will always account for the most variance and hence have the highest eigenvalue , and the next factor will account for as much of the left over variance as it can, and so on. Hence, each successive factor will account for less and less variance. For example, the third row shows a value of This means that the first three factors together account for Extraction Sums of Squared Loadings — The number of rows in this panel of the table correspond to the number of factors retained.

In this example, we requested that three factors be retained, so there are three rows, one for each retained factor. The values in this panel of the table are calculated in the same way as the values in the left panel, except that here the values are based on the common variance. The values in this panel of the table will always be lower than the values in the left panel of the table, because they are based on the common variance, which is always smaller than the total variance.

Rotation Sums of Squared Loadings — The values in this panel of the table represent the distribution of the variance after the varimax rotation. Varimax rotation tries to maximize the variance of each of the factors, so the total amount of variance accounted for is redistributed over the three extracted factors. The scree plot graphs the eigenvalue against the factor number. You can see these values in the first two columns of the table immediately above.

From the third factor on, you can see that the line is almost flat, meaning the each successive factor is accounting for smaller and smaller amounts of the total variance. Factor Matrix — This table contains the unrotated factor loadings, which are the correlations between the variable and the factor.

This makes the output easier to read by removing the clutter of low correlations that are probably not meaningful anyway. Factor — The columns under this heading are the unrotated factors that have been extracted. As you can see by the footnote provided by SPSS a. Reproduced Correlations — This table contains two tables, the reproduced correlations in the top part of the table, and the residuals in the bottom part of the table.

Reproduced Correlation — The reproduced correlation matrix is the correlation matrix based on the extracted factors. You want the values in the reproduced matrix to be as close to the values in the original correlation matrix as possible.

This means that the residual matrix, which contains the differences between the original and the reproduced matrix to be close to zero. If the reproduced matrix is very similar to the original correlation matrix, then you know that the factors that were extracted accounted for a great deal of the variance in the original correlation matrix, and these few factors do a good job of representing the original data.

The numbers on the diagonal of the reproduced correlation matrix are presented in the Communalities table in the column labeled Extracted.

For example, the original correlation between item13 and item14 is. The residual is. Rotated Factor Matrix — This table contains the rotated factor loadings, which represent both how the variables are weighted for each factor but also the correlation between the variables and the factor.

For orthogonal rotations, such as varimax, the factor pattern and factor structure matrices are the same. Factor — The columns under this heading are the rotated factors that have been extracted.

These are the factors that analysts are most interested in and try to name. The third factor has to do with comparisons to other instructors and courses. The table below is from another run of the factor analysis program shown above, except with a promax rotation.

We have included it here to show how different the rotated solutions can be, and to better illustrate what is meant by simple structure. As you can see with an oblique rotation, such as a promax rotation, the factors are permitted to be correlated with one another. With an orthogonal rotation, such as the varimax shown above, the factors are not permitted to be correlated they are orthogonal to one another.

Oblique rotations, such as promax, produce both factor pattern and factor structure matrices. For orthogonal rotations, such as varimax and equimax, the factor structure and the factor pattern matrices are the same. The factor structure matrix represents the correlations between the variables and the factors. The factor pattern matrix contain the coefficients for the linear combination of the variables.

The table below indicates that the rotation done is an oblique rotation. If an orthogonal rotation had been done like the varimax rotation shown above , this table would not appear in the output because the correlations between the factors are set to 0.