﻿ Table 23.1, 23.11 Principal components plots of item loadings

# Table 23.1, 23.11, ... Principal components/contrast plots of item loadings

Please do not interpret this as a usual factor analysis. These plots show contrasts between opposing factors, identified as "A,B,.." and "a,b,...", not loadings on one factor. For more discussion, see dimensionality and contrasts.

Quick summary:

(a) the X-axis is the measurement axis. So we are not concerned about quadrants, we are concerned about vertical differences. The Table 23 plots show contrasts between types of items: those at the top vs. those at the bottom. The Table 24 plots show contrasts between types of persons: those at the top vs. those at the bottom.

(b) "How much" is important. See the Variance Table explained in Table 23.0. Important differences have eigenvalues greater than 2.0.

(c) If the difference is important, it suggests that we divide the test into pieces, clustering the items in the top half of the plot and the items in the bottom half. Winsteps estimates a measure for each person on each cluster of items. The correlations of the measures are reported. Small disattenuated correlations indicate that the clusters of items measure different sub-dimensions.

(d) Perform separate analyses for the target clusters of items. Cross-plot and correlate the person measures. We will then see for whom the differences are important. Usually, for a carefully designed instrument, it is such a small segment that we decide it is not worth thinking of the test as measuring two dimensions. Tables 23.4 also helps us think about this.

1. Put a code into the item label to indicate the item subset to which the item belongs.

2a. Use ISELECT= for each subset code, and produce a person measure (PFILE=). Cross-plot the person measures.

or

2b. Do a Differential Person Functioning (DPF=) analysis based on the subset code. Table 31.1 will give you an inter-item-subset t-test for each person.

Table 23.0 Variance components scree plot for items

Table 23.1, 23.11 Principal components plots of item loadings

Table 23.2, 23.12 Item Principal components analysis/contrast of residuals

Table 23.3, 23.13 Item contrast by persons

Table 23.4, 23.14 Item contrast loadings sorted by measure

Table 23.5, 23.15 Item contrast loadings sorted by entry number

Table 23.6, 23.16 Person measures for item clusters in contrast. Cluster Measure Plot for Table 23.6.

Table 23.99 Largest residual correlations for items

These plots show the contrasts by plotting the unstandardized "raw" loading on each component against the item calibration (or person measure). The contrast shows items (or persons) with different residual patterns. A random pattern with few high loadings is expected.

The horizontal axis is the Rasch dimension. This has been extracted from the data prior to the analysis of residuals.

Letters "A,B,C,..." and "a,b,c,..." identify items (persons) with the most opposed loadings on the first contrast in the residuals. On subsequent contrasts, the items retain their first contrast identifying letters. When there are 9 items (persons) or less, the item number is displayed.

The items are clustered into 3 clusters on the right-side of the plot. This is because interpreting the PCA Components ("Contrasts") usually requires us to compare the items at the top of the plot against the items at the bottom, often ignoring the middle items. The purpose of the plot associated with Table 23.6 is to help us see whether the 3 clusters of items are truly measuring different things. If they are measuring the same thing statistically, then usually no action is needed. If the clusters of items are measuring different things, then the analyst must decide what to do.

In the residuals, each item (person) is modeled to contribute one unit of randomness. Thus, there are as many residual variance units as there are items (or persons). For comparison, the amount of person (item) variance explained by the item (person) measures is approximated as units of that same size.

In the Figure below from Example0.txt , the first contrast in the standardized residuals separates the items into 3 clusters. To identify the items, see Tables 23.3, 24.3. In this example, the dimension is noticeable, with strength of around 5 out of 25 items. This is in the residual variance, i.e., in the part of the observations unexplained by the measurement model. But, hopefully, most of the variance in the observations has been explained by the model. The part of that explained variance attributable to the Persons is shown in variance units locally-rescaled to accord with the residual variances. In this example, the variance explained by the person measures is equivalent to 10 items. Consequently, the secondary dimension in the items is noticeable. The disattenuated correlation between person measures on item in Cluster 1 and person measures on items in Cluster 3 is less than 0.3. The secondary dimension underlying the 1st Contrast is biasing the person measures.

For items:

Table of STANDARDIZED RESIDUAL variance in Eigenvalue units = ACT information units

Eigenvalue   Observed   Expected

Total raw variance in observations     =      50.9521 100.0%         100.0%

Raw variance explained by measures   =      25.9521  50.9%          50.7%

Raw variance explained by persons  =      10.3167  20.2%          20.2%

Raw Variance explained by items    =      15.6354  30.7%          30.6%

Raw unexplained variance (total)     =      25.0000  49.1% 100.0%   49.3%

Unexplned variance in 1st contrast =       4.6287   9.1%  18.5%

STANDARDIZED RESIDUAL CONTRAST 1 PLOT

-4     -3     -2     -1      0      1      2      3      4      5

-+------+------+------+------+------+------+------+------+------+- COUNT  CLUSTER

.8 +                            |                                   +

|                            |            A B                    | 2      1

.7 +                            |                                   +

|                            |                                   |

.6 +                            |                C                  + 1      1

|                            |                                   |

C  .5 +                            |                                   +

O     |                            |                                   |

N  .4 +                            |           DE                      + 2      1

T     |                            |   F                               | 1      1

R  .3 +                            |                                   +

A     |                            |                                   |

S  .2 +                            |       G                           + 1      2

T     |                            |                                   |

.1 +                            |                                   +

1     |                            | H                                 | 1      2

.0 +----------------------------|-----------------------------------+

L     |                            |    I                              | 1      2

O -.1 +                         K  |   J                               + 2      2

A     |                            |L                                  | 1      2

D -.2 +                            |                                   +

I     |                            |M l                                | 2      3

N -.3 +                     j      |  k                                + 2      3

G     |                         i  |                                   | 1      3

-.4 +      f    h              g |                                   + 3      3

|                            |                                   |

-.5 +                      e     |                                   + 1      3

|              d        c    |                                   | 2      3

-.6 +                  b         |                                   + 1      3

|                   a        |                                   | 1      3

-+------+------+------+------+------+------+------+------+------+-

-4     -3     -2     -1      0      1      2      3      4      5

ACT MEASURE

COUNT:      1    1  1   11 111 21  21221  1   21 1 1

KID                     1 2 112 1 5346832333342 23222 1 1 1 1  1     11

T        S         M        S         T

%TILE                   0      10 20 40 50 60 70 80 90               99

Approximate relationships between the KID measures

PCA      ACT       Pearson       Disattenuated Pearson+Extr  Disattenuated+Extr

Contrast  Clusters  Correlation   Correlation   Correlation   Correlation

1        1 - 3      0.1404        0.2175        0.1951        0.2923

1        1 - 2      0.2950        0.4675        0.3589        0.5485

1        2 - 3      0.8065        1.0000        0.8123        1.0000

Example: A teacher survey had hundreds of items contributed by numerous stakeholders (with special agendas), but almost no hypothesized constructs. In other words, it was a mess! PCA of residuals was used to discover subsets of items that cooperated. A useful strategy is to use IWEIGHT=. Weight a core subset of items on a construct of interest "1", and weight all the other items "0". Then do a Winsteps analysis. All the items will be calibrated, but only the weighted items will contribute to the person measures. Then do a PCA of residuals. All the items will participate. The core items will cluster. Unweighted items that cluster with the core can be inspected, and added to the core if suitable. Then the process is repeated. When all the items for one construct have been identified, those items can be deleted from the dataset. The process begins again with the next core subset of items.

Approximate relationships between the KID measures

PCA      ACT       Pearson       Disattenuated Pearson+Extr  Disattenuated+Extr

Contrast  Clusters  Correlation   Correlation   Correlation   Correlation

1        1 - 3      0.1404        0.2175        0.1951        0.2923

1        1 - 2      0.2950        0.4675        0.3589        0.5485

1        2 - 3      0.8065        1.0000        0.8123        1.0000

The items are anchored (fixed) at their difficulties from the main analysis (as reported in Table 14.1). Then the items are segmented into subtests according to their cluster numbers, and each person is measured on each subtest. The person measures for each cluster of items are correlated with their measures from the other clusters of items, and reported here as the "Pearson Correlation". Each person measure for each cluster of items has a standard error. These error variances are removed to produce the "Disattenuated Correlation". If the disattenuated correlation approaches 1.0, then the person measures from the two clusters of items are statistically the same. We cannot reject the hypothesis that the two clusters of items are measuring the same thing. If the disattenuated correlation is out of range or undefined, it is reported as (1.00) or (-1.00) matching the sign of the observed correlation.

For the Pearson Correlation and the Disattenuated (Pearson) Correlation, persons with extreme scores on a cluster are omitted. For the Pearson+Extr(eme) Correlation and the Disattenuated (Pearson)+Extr(eme) Correlation, persons with extreme scores on a cluster are included.

These correlations are approximate. For more accurate correlations, please perform separate analyses of each cluster of items, and then use the Scatterplot function to investigate the relationships between the person measures estimated from the different clusters.

Roughly speaking, we look at the disattenuated correlations (otherwise measurement error clouds everything):

Correlations below 0.57 indicate that person measures on the two item clusters have half as much variance in common as they have indpendently. (Cut-off for probably different latent variables?)

Correlations above 0.71 indicate that person measures on the two item clusters have more than half their variance in common, so they are more dependent than independent.

Correlations above 0.82, twice as dependent as independent. (Cut-off for probably the same latent variable?)

Correlations above 0.87, three times as dependent as independent.

Example 1: From Table 23.3 of the analysis of a large empirical dataset:

Approximate relationships between the person measures

Contrast 1        Pearson        Disattenuated

Item Clusters     Correlation    Correlation

1 - 2             .5151          .5762

Actual relationships of person measures estimated from separate analyses of the two clusters of items:

Contrast 1        Pearson        Disattenuated

Item Clusters     Correlation    Correlation

1 - 2             .5083          .6352

Help for Winsteps Rasch Measurement Software: www.winsteps.com. Author: John Michael Linacre

For more information, contact info@winsteps.com or use the Contact Form

 Forum Rasch Measurement Forum to discuss any Rasch-related topic

Click here to ask a question or make a suggestion about Winsteps and Facets software.

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

Coming Rasch-related Events
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
Jan. 22-24, 2018, Mon-Wed. In-person workshop: Rasch Measurement for Everybody en español (A. Tristan, Winsteps), San Luis Potosi, Mexico. www.ieia.com.mx
April 10-12, 2018, Tues.-Thurs. Rasch Conference: IOMW, New York, NY, www.iomw.org
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Sept. 3 - 6, 2018, Mon.-Thurs. IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

Our current URL is www.winsteps.com

Winsteps® is a registered trademark

Concerned about aches, pains, youthfulness? Mike and Jenny suggest Liquid Biocell