SPSS & Table 3.1 Comparison June 20th, 2013, 5:36am

 Rasch Measurement Forum  ›  General Boards  ›  Rasch on the run ...  ›  SPSS & Table 3.1 Comparison
Users Browsing Forum
Googlebot and 4 Guests

Pages: 1
Author    SPSS & Table 3.1 Comparison  (currently 928 views)
uve
 Posted: July 23rd, 2012, 11:09pm
Big Member

Posts: 377
 Mike,I am continuing my comparison of the advantages of the Rasch model as compared to CTT using SPSS with the same survey data in my previous post. This time I ran 4 separate reliablity analyses for two comparisons:1) Comparison of the control group versus the treatment group on all 17 items2) Comparison of items 1-11 versus 12-17 on all personsSPSS Cronbach's Alpha: 1) .81  .822) .83  .79Winsteps Cronbach:1)  .97   .812) 1.00  1.00Winsteps Model Person Reliability, All Persons1)  .85   .662)  .81   .81Again, I am wondering why there is such a great difference between the two. I realize that model reliability is a different calculation and so may not be directly comparable, but others are confusing me. Thanks for taking the time as always to look this over.
 Logged Offline
Mike.Linacre
 Posted: July 24th, 2012, 12:05pm
Maximum Member

Posts: 812
 Thank you for these computations, Uve.Based on their theories, we expect Cronbach Alpha to be greater than Rasch reliability, see http://www.rasch.org/rmt/rmt113l.htm- a situation many analysts have noticed in practice.As an independent check on Cronbach Alpha (=KR-20 for dichotomies), I analyzed Guilford's Table 17.2. Here it is:Title = "Guilford Table 17.2. His KR-20 = 0.81"ni=8item1=1name1=1&ENDEND LABELS00000000100000001010000011001000010100101110101011111100111111001111010111111111He reports KR-20 = 0.81. Winsteps reports Cronbach Alpha = 0.82. The difference is probably computational precision and rounding error.Uve, what does SPSS report?
 Logged Offline
 Reply: 1 - 9
uve
 Posted: July 24th, 2012, 5:57pm
Big Member

Posts: 377
 Mike,SPSS reports .815To be honest, for my purposes I would not be that concerned over a .01 or .02 difference. What concerned me more was the that Winsteps reported a perfect correlation of 1.00 for items 1-11 as well as items 12-17 while SPSS reported these as .83 and .79 respectively, which is a significant difference. Even the overall Cronbach in SPSS of .88, which I didn't mention before,  is significantly lower than the Winsteps Cronbach of .93. I can't help but think I've done something serioulsy wrong here or am interpreting a technique incorrectly.
 Logged Offline
 Reply: 2 - 9
Mike.Linacre
 Posted: July 24th, 2012, 10:32pm
Maximum Member

Posts: 812
 Uve, good. It looks like SPSS and Winsteps agree about the basic computation.Here are some areas where there can be differences:1. Missing data. 2. Polytomous data. 3. Weighted data.4. Rescored/recounted data.Do any of these apply to your situation?
 Logged Offline
 Reply: 3 - 9
uve
 Posted: July 24th, 2012, 11:45pm
Big Member

Posts: 377
 Yes: 1, 2 and 4. The data was attached in my previous SPSS and Table 23 comparison.Several students did not responde to all data.There are 4 options for each item.None of the options are weighted differentlyI received the initial file in SPSS with items 12-17 scored 1-4 for some reason while items 1-11 were scored 0-3. I simply changed all student responses to these last 6 items as opposed to rescoring them, which would probably have been easier.
 Logged Offline
 Reply: 4 - 9
Mike.Linacre
 Posted: July 25th, 2012, 4:36am
Maximum Member

Posts: 812
 Ouch, Uve! Both polytomous and missing data! So we need to isolate the difference.1) Please choose a subset of polytomous dataline with no missing data. How do Winsteps and SPSS compare? They should agree.2) Please choose a dichotomous dataset, such as exam1.txt. Make some observations missing. How do Winsteps and SPSS compare? They may differ. Winsteps skips missing observations in its computation, but SPSS may use case-wise deletion.
 Logged Offline
 Reply: 5 - 9
uve
 Posted: July 25th, 2012, 4:30pm
Big Member

Posts: 377
 Thanks Mike. That must be it. I chose a sub-sample of 21 students who responded to all 17 items and compared both Winsteps and SPSS Cronbach. They were identical at .78. I then took a closer look at the original SPSS output and found this message: "Listwise deletion based on all variables in the procedure."There were a total of 121 respondents, but 3 chose no options to any of the items. I deleted these in the Winsteps file but kept them in the SPSS file. SPSS deleted them as well along with 18 others for a total of 21. So I guess it was working off of 100 valid cases. There were also a few students who responded to only two or three items and I deliberated for quite some time as to whether I should remove them from the Winsteps file but decided against it. So with a listwise deletion process, are both SPSS and Winsteps treating missing data in the same manner?Thanks again as always!
 Logged Offline
 Reply: 6 - 9
Mike.Linacre
 Posted: July 25th, 2012, 10:26pm
Maximum Member

Posts: 812
 Uve, listwise deletion drops the data record if even one observation is missing. Winsteps does not do this. Winsteps computes the Cronbach variance terms using all the non-missing observations. For example, on a CAT test, listwise deletion would omit every person. Winsteps keeps every person.So, if SPSS does automatic listwise deletion, and we deliberately delete records with missing data from a Winsteps analysis (using IDELETE= etc.), then the two computations should be the same.
 Logged Offline
 Reply: 7 - 9
uve
 Posted: August 2nd, 2012, 11:57pm
Big Member

Posts: 377
 Mike,Continuing the discussion of missing data but for a different purpose, my data had about 4% missing responses from various respondents to various items. With 118 respondents and 17 items with 4 Likert options each, can one use a percentage as a rough guide for when it might be best to switch from Mantel-Heanszel probabilities to Welch t-tests for DIF? Also, I understand that DIF is primarily intended to pick up on individual item idiosyncrasies and one can have DIF without multi-dimensionality. But is the reverse true?
 Logged Offline
 Reply: 8 - 9
Mike.Linacre
 Posted: August 3rd, 2012, 3:20am
Maximum Member

Posts: 812
 Uve, MH is based on cross-tabs with one cross-tab for each ability stratum. MH is implemented in Winsteps using "thin" slicing (one stratum for each raw-score level). Look at Winsteps Table 20.2, this will show you how many respondents there are at each score-level (FREQUENCY). Generally we would need at least 10 respondents in each cell of the cross-tab, so that would be at least 40 respondents in each stratum. If many stratum have fewer than 40, then increase MHSLICE= to encompass two or more ability strata.Multidimensional items (e.g. a geography item in an arithmetic test) without DIF would indicate that the ability distribution on geography matches the ability distribution on arithmetic for both the focal and reference groups.
 Logged Offline
 Reply: 9 - 9
Pages: 1

 Locked Board Board Index  ›  Rasch on the run ...  [ previous | next ] Switch to: Rasch on the run ... (only location)

Forum Rules
 You may not post new threadsYou may not post repliesYou may not post pollsYou may not post attachments HTML is offBlah Code is onSmilies are on