Table 23.1, 23.11, ... Principal components/contrast plots of item loadings

Please do not interpret this as a usual factor analysis. These plots show contrasts between opposing factors, identified as "A,B,.." and "a,b,...", not loadings on one factor. For more discussion, see dimensionality and contrasts.

 

Quick summary:

(a) the X-axis is the measurement axis. So we are not concerned about quadrants, we are concerned about vertical differences. The Table 23 plots show contrasts between types of items: those at the top vs. those at the bottom. The Table 24 plots show contrasts between types of persons: those at the top vs. those at the bottom.

 

(b) "How much" is important. See the Variance Table explained in Table 23.0. Important differences have eigenvalues greater than 2.0.

 

(c) If the difference is important, it suggests that we divide the test into pieces, clustering the items in the top half of the plot and the items in the bottom half. Winsteps estimates a measure for each person on each cluster of items. The correlations of the measures are reported. Small disattenuated correlations indicate that the clusters of items measure different sub-dimensions.

 

(d) Perform separate analyses for the target clusters of items. Cross-plot and correlate the person measures. We will then see for whom the differences are important. Usually, for a carefully designed instrument, it is such a small segment that we decide it is not worth thinking of the test as measuring two dimensions. Tables 23.4 also helps us think about this.

1. Put a code into the item label to indicate the item subset to which the item belongs.

2a. Use ISELECT= for each subset code, and produce a person measure (PFILE=). Cross-plot the person measures.

or

2b. Do a Differential Person Functioning (DPF=) analysis based on the subset code. Table 31.1 will give you an inter-item-subset t-test for each person.

 

Table 23.0 Variance components scree plot for items

Table 23.1, 23.11 Principal components plots of item loadings

Table 23.2, 23.12 Item Principal components analysis/contrast of residuals

Table 23.3, 23.13 Item contrast by persons

Table 23.4, 23.14 Item contrast loadings sorted by measure

Table 23.5, 23.15 Item contrast loadings sorted by entry number

Table 23.6, 23.16 Person measures for item clusters in contrast. Cluster Measure Plot for Table 23.6.

Table 23.99 Largest residual correlations for items

Youtube video explaining Table 23

 


 

These plots show the contrasts by plotting the unstandardized "raw" loading on each component against the item calibration (or person measure). The contrast shows items (or persons) with different residual patterns. A random pattern with few high loadings is expected.

 

The horizontal axis is the Rasch dimension. This has been extracted from the data prior to the analysis of residuals.

 

Letters "A,B,C,..." and "a,b,c,..." identify items (persons) with the most opposed loadings on the first contrast in the residuals. On subsequent contrasts, the items retain their first contrast identifying letters. When there are 9 items (persons) or less, the item number is displayed.

 

The items are clustered into 3 clusters on the right-side of the plot. This is because interpreting the PCA Components ("Contrasts") usually requires us to compare the items at the top of the plot against the items at the bottom, often ignoring the middle items. The purpose of the plot associated with Table 23.6 is to help us see whether the 3 clusters of items are truly measuring different things. If they are measuring the same thing statistically, then usually no action is needed. If the clusters of items are measuring different things, then the analyst must decide what to do.

 

In the residuals, each item (person) is modeled to contribute one unit of randomness. Thus, there are as many residual variance units as there are items (or persons). For comparison, the amount of person (item) variance explained by the item (person) measures is approximated as units of that same size.

 

In the Figure below from Example0.txt , the first contrast in the standardized residuals separates the items into 3 clusters. To identify the items, see Tables 23.3, 24.3. In this example, the dimension is noticeable, with strength of around 5 out of 25 items. This is in the residual variance, i.e., in the part of the observations unexplained by the measurement model. But, hopefully, most of the variance in the observations has been explained by the model. The part of that explained variance attributable to the Persons is shown in variance units locally-rescaled to accord with the residual variances. In this example, the variance explained by the person measures is equivalent to 10 items. Consequently, the secondary dimension in the items is noticeable. The disattenuated correlation between person measures on item in Cluster 1 and person measures on items in Cluster 3 is less than 0.3. The secondary dimension underlying the 1st Contrast is biasing the person measures.

 

Letters display when there are more than 9 items on the plot. Numbers display when there are 9 or less items on the plot.

 

For items:

 

          Table of STANDARDIZED RESIDUAL variance in Eigenvalue units = ACT information units

                                           Eigenvalue   Observed   Expected

Total raw variance in observations     =      50.9521 100.0%         100.0%

  Raw variance explained by measures   =      25.9521  50.9%          50.7%

    Raw variance explained by persons  =      10.3167  20.2%          20.2%

    Raw Variance explained by items    =      15.6354  30.7%          30.6%

  Raw unexplained variance (total)     =      25.0000  49.1% 100.0%   49.3%

    Unexplned variance in 1st contrast =       4.6287   9.1%  18.5%

 

       STANDARDIZED RESIDUAL CONTRAST 1 PLOT

 

      -4     -3     -2     -1      0      1      2      3      4      5

      -+------+------+------+------+------+------+------+------+------+- COUNT  CLUSTER

   .8 +                            |                                   +

      |                            |            A B                    | 2      1

   .7 +                            |                                   +

      |                            |                                   |

   .6 +                            |                C                  + 1      1

      |                            |                                   |

C  .5 +                            |                                   +

O     |                            |                                   |

N  .4 +                            |           DE                      + 2      1

T     |                            |   F                               | 1      1

R  .3 +                            |                                   +

A     |                            |                                   |

S  .2 +                            |       G                           + 1      2

T     |                            |                                   |

   .1 +                            |                                   +

1     |                            | H                                 | 1      2

   .0 +----------------------------|-----------------------------------+

L     |                            |    I                              | 1      2

O -.1 +                         K  |   J                               + 2      2

A     |                            |L                                  | 1      2

D -.2 +                            |                                   +

I     |                            |M l                                | 2      3

N -.3 +                     j      |  k                                + 2      3

G     |                         i  |                                   | 1      3

  -.4 +      f    h              g |                                   + 3      3

      |                            |                                   |

  -.5 +                      e     |                                   + 1      3

      |              d        c    |                                   | 2      3

  -.6 +                  b         |                                   + 1      3

      |                   a        |                                   | 1      3

      -+------+------+------+------+------+------+------+------+------+-

      -4     -3     -2     -1      0      1      2      3      4      5

                                 ACT MEASURE

 COUNT:      1    1  1   11 111 21  21221  1   21 1 1

 

KID                     1 2 112 1 5346832333342 23222 1 1 1 1  1     11

                       T        S         M        S         T

%TILE                   0      10 20 40 50 60 70 80 90               99

 

 

 

If your plot looks like this, please change the font in your text editor (NotePad) to "fixed width": Courier New, Consolas, Lucida Console

     -4     -3     -2     -1      0      1      2      3      4      5

     -+------+------+------+------+------+------+------+------+------+- COUNT  CLUSTER

  .8 +                            |                                   +

     |                            |            A B                    | 2      1

  .7 +                            |                                   +

     |                            |                                   |

  .6 +                            |                C                  + 1      1

     |                            |                                   |

C  .5 +                            |                                   +

O     |                            |                                   |

N  .4 +                            |           DE                      + 2      1

T     |                            |   F                               | 1      1

R  .3 +                            |                                   +

A     |                            |                                   |

S  .2 +                            |       G                           + 1      2

T     |                            |                                   |

  .1 +                            |                                   +

1     |                            | H                                 | 1      2

  .0 +----------------------------|-----------------------------------+

L     |                            |    I                              | 1      2

O -.1 +                         K  |   J                               + 2      2

A     |                            |L                                  | 1      2

D -.2 +                            |                                   +

I     |                            |M l                                | 2      3

N -.3 +                     j      |  k                                + 2      3

G     |                         i  |                                   | 1      3

 -.4 +      f    h              g |                                   + 3      3

     |                            |                                   |

 -.5 +                      e     |                                   + 1      3

     |              d        c    |                                   | 2      3

 -.6 +                  b         |                                   + 1      3

     |                   a        |                                   | 1      3

     -+------+------+------+------+------+------+------+------+------+-

     -4     -3     -2     -1      0      1      2      3      4      5

 

 

Example: A teacher survey had hundreds of items contributed by numerous stakeholders (with special agendas), but almost no hypothesized constructs. In other words, it was a mess! PCA of residuals was used to discover subsets of items that cooperated. A useful strategy is to use IWEIGHT=. Weight a core subset of items on a construct of interest "1", and weight all the other items "0". Then do a Winsteps analysis. All the items will be calibrated, but only the weighted items will contribute to the person measures. Then do a PCA of residuals. All the items will participate. The core items will cluster. Unweighted items that cluster with the core can be inspected, and added to the core if suitable. Then the process is repeated. When all the items for one construct have been identified, those items can be deleted from the dataset. The process begins again with the next core subset of items.

 


 

Correlation Table of person Measures on each of the Clusters:

 

Approximate relationships between the KID measures

 PCA      ACT       Pearson       Disattenuated Pearson+Extr  Disattenuated+Extr

Contrast  Clusters  Correlation   Correlation   Correlation   Correlation  Cluster Sizes

 1        1 - 2      0.2951        0.4683        0.3592        0.5497      6       6

 1        1 - 3      0.1411        0.2191        0.1958        0.2941      6       13

 1        2 - 3      0.8065        1.0000        0.8123        1.0000      6       13

 1        1 - B      0.4541                      0.5506                    6       25

 1        2 - B      0.9030                      0.8928                    6       25

 1        3 - B      0.8934                      0.8652                    13      25

    

The items are anchored (fixed) at their difficulties from the main analysis (as reported in Table 14.1). Then the items are segmented into subtests according to their cluster numbers, and each person is measured on each subtest. The person measures for each cluster of items are correlated with their measures from the other clusters of items, and reported here as the "Pearson Correlation". Each person measure for each cluster of items has a standard error. These error variances are removed to produce the "Disattenuated Correlation". If the disattenuated correlation approaches 1.0, then the person measures from the two clusters of items are statistically the same. We cannot reject the hypothesis that the two clusters of items are measuring the same thing. If the disattenuated correlation is out of range or undefined, it is reported as (1.00) or (-1.00) matching the sign of the observed correlation.

 

The correlation and disattenuated correlation in, say, Table 23.1, are computed from the person measures and model standard errors shown in Table 23.6 . This would be equivalent to using the Model Reliability from Table 3.1.  In the computations, the "Real" standard errors would be larger, so the "Real" disattenuated correlations would also be larger. The "Model" disattenuated correlation is a more conservative (lower) estimate, a lower bound. The computation is based on Rasch measures, so Cronbach Alpha, which is based on raw scores, does not apply.

 

For the Pearson Correlation and the Disattenuated (Pearson) Correlation, persons with extreme scores on a cluster are omitted. For the Pearson+Extr(eme) Correlation and the Disattenuated (Pearson)+Extr(eme) Correlation, persons with extreme scores on a cluster are included. Cluster sizes are the counts of items in each cluster.

 

These correlations are approximate. For more accurate correlations, please perform separate analyses of each cluster of items, and then use the Scatterplot function to investigate the relationships between the person measures estimated from the different clusters.

 

The middle cluster (2) usually corresponds roughly with the Rasch dimension. So the important correlations are 1-2 and 2-3. Clusters 1 and 3 are both somewhat off-dimension so the correlation 1-3 is somewhat accidental. It is nice when it is high, but low 1-3 correlation merely tells us that 1 and 3 are off-dimension in different ways.

 

Roughly speaking, we look at the disattenuated correlations (otherwise measurement error clouds everything):

Correlations below 0.57 indicate that person measures on the two item clusters have half as much variance in common as they have independently. (Cut-off for probably different latent variables?)

Correlations above 0.71 indicate that person measures on the two item clusters have more than half their variance in common, so they are more dependent ( = more the same thing) than independent (= different things).

Correlations above 0.82, twice as dependent as independent. (Cut-off for probably the same latent variable?)

Correlations above 0.87, three times as dependent as independent (definitely the same thing).

 

Cluster B is the baseline measures, the estimates from the main analysis. No disattenuated correlation is shown for B because B is dependent on the other clusters.

 

Example 1: From Table 23.3 of the analysis of a large empirical dataset:

Approximate relationships between the person measures

Contrast 1        Pearson        Disattenuated

Item Clusters     Correlation    Correlation

1 - 2             .5151          .5762

 

Actual relationships of person measures estimated from separate analyses of the two clusters of items:

Contrast 1        Pearson        Disattenuated

Item Clusters     Correlation    Correlation

1 - 2             .5083          .6352


Help for Winsteps Rasch Measurement and Rasch Analysis Software: www.winsteps.com. Author: John Michael Linacre

Facets Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation Minifac download
Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation Ministep download

Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang Statistical Analyses for Language Testers (Facets), Rita Green Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind Rasch Measurement: Applications, Khine Winsteps Tutorials - free
Facets Tutorials - free
Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse Rasch Measurement Theory Analysis in R, Wind, Hua Applying the Rasch Model in Social Sciences Using R, Lamprianou Journal of Applied Measurement
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Rasch Models for Measurement, David Andrich Constructing Measures, Mark Wilson Best Test Design - free, Wright & Stone
Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias Diseño de Mejores Pruebas - free, Spanish Best Test Design A Course in Rasch Measurement Theory, Andrich, Marais Rasch Models in Health, Christensen, Kreiner, Mesba Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
As an Amazon Associate I earn from qualifying purchases. This does not change what you pay.

facebook Forum: Rasch Measurement Forum to discuss any Rasch-related topic

To receive News Emails about Winsteps and Facets by subscribing to the Winsteps.com email list,
enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Winsteps.com
The Winsteps.com email list is only used to email information about Winsteps, Facets and associated Rasch Measurement activities. Your email address is not shared with third-parties. Every email sent from the list includes the option to unsubscribe.

Questions, Suggestions? Want to update Winsteps or Facets? Please email Mike Linacre, author of Winsteps mike@winsteps.com


State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied
 
Rasch, Winsteps, Facets online Tutorials


 

 
Coming Rasch-related Events
May 17 - June 21, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 12 - 14, 2024, Wed.-Fri. 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
June 21 - July 19, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 5 - Aug. 6, 2024, Fri.-Fri. 2024 Inaugural Conference of the Society for the Study of Measurement (Berkeley, CA), Call for Proposals
Aug. 9 - Sept. 6, 2024, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

 

Our current URL is www.winsteps.com

Winsteps® is a registered trademark