Invariance Viewed Thru 2.2, 2.5 & 2.7 - Rasch Measurement Forum

Mike,

I'm hoping you could shed some light on the application of Table 2.7 versus 2.2. We've spoken about this before, but my concern is the stability of measures I am getting. One advantage of the Rasch model is the issue of invariance. Most of the samples I use range from about 1,000 to 7,000 respondents. I am always amazed at the significant difference between the sample free 2.2 and sample dependent 2.7 in all of my data. If I am to set cut points for performance levels, then I would have a hard time basing this on 2.2 and would look more to 2.7, which always seems to be much closer to the empirical data in 2.5. So, it seems if I assume the invariance of my measures, then I should be able to apply them to other samples knowing full well there will always be differences, but none to the extent that the fit to these different samples would reveal dramatic differences. Or put differently, what good are my item calibrations given a stable and large enough sample upon which they are based if I can't rely on their invariance across other populations? I've included a typical example.

If invariance has been achieved, it would seem to me that these would be much closer, especially if the empirical data in 2.5 are similar to 2.7.

I'd greatly appreciate your thoughts on this.

EXPECTED SCORE: MEAN  (Rasch-score-point threshold, ":"
indicates Rasch-half-point threshold) (BY CATEGORY SCORE)
-3       -2        -1         0         1         2         3
+---------+---------+---------+---------+---------+---------�
1                   1     :     2   :   3     :     4       4
�                                                           �
1                  1    :     2   :    3     :     4        4
1                 1     :     2   :   3     :     4         4
1                 1    :     2    :   3     :     4         4
1                1     :     2   :   3     :     4          4
1                1     :     2   :   3     :     4          4
�                                                           �
1              1     :     2   :   3     :     4            4
1              1    :     2   :    3     :     4            4
�                                                           �
1            1     :    2    :   3     :     4              4
�                                                           �
1          1     :     2   :   3     :     4                4
1          1     :    2    :   3     :     4                4
1          1    :     2   :    3     :     4                4
�                                                           �
�                                                           �
1       1     :     2   :   3     :     4                   4
�                                                           �
�                                                           �
1    1     :     2   :   3      :     4                     4
+---------+---------+---------+---------+---------+---------�
-3       -2        -1         0         1         2         3

EXPECTED AVERAGE MEASURES FOR Person (scored) (BY CATEGORY SCOR
-3       -2        -1         0         1         2         3
+---------+---------+---------+---------+---------+---------�
�                              1  2   3          4          �
�                                                           �
�                              1  2   3         4           �
�                              1  2  3          4           �
�                              1  2  3          4           �
�                             1   2  3         4            �
�                             1   2  3         4            �
�                                                           �
�                             1  2  3         4             �
�                            1   2  3         4             �
�                                                           �
�                            1  2   3        4              �
�                                                           �
�                           1   2  3        4               �
�                           1   2  3        4               �
�                           1   2  3        4               �
�                                                           �
�                                                           �
�                          1   2   3       4                �
�                                                           �
�                                                           �
�                        1    2   3       4                 �
+---------+---------+---------+---------+---------+---------�
-3       -2        -1         0         1         2         3

  OBSERVED AVERAGE MEASURES FOR Person (scored) (BY CATEGORY SCORE)
-3       -2        -1         0         1         2         3
+---------+---------+---------+---------+---------+---------�
�                              1  2   3          4          �
�                                                           �
�                             1   2   3         4           �
�                             1   2   3          4          �
�                             1   2  3         4            �
�                             1  2   3         4            �
�                             1   2  3         4            �
�                                                           �
�                             1  2  3         4             �
�                            1   2  3         4             �
�                                                           �
�                            1   2  3        4              �
�                                                           �
�                           1   2  3        4               �
�                           1   2  3        4               �
�                           1   2  3        4               �
�                                                           �
�                                                           �
�                         1    2  3        4                �
�                                                           �
�                                                           �
�                           1   2 3       4                 �
+---------+---------+---------+---------+---------+---------�
-3       -2        -1         0         1         2         3