Hello,
I am currently working on my dissertation data and have run into some confusion.  I have two groups of 66 people who took an online training course.  One group took it face-to-face and another online.  At the completion of module 1, they took a 12 item survey.  They did the same survey after four subsequent modules. The items are on a rating scale of four thresholds.  I am measuring one construct and that is whether they were engagement in the course.

After reviewing the first module data my person separation is quite low at 1.54 and the reliability is .7.  I looked at infit mean square with a range of .6 to 1.4 and ended up removing 24 people and the separation only went up to 1.66 with a reliability of .73.

I am wondering if I could be doing something wrong in that I cannot get the separation higher.

I was able to remove three items and increase my item separation to 2.5.  Any help would be appreciated.

Christian:

Person reliability and separation increase when there are more items, and also when there is a wider person ability range.

Item reliability and separation increase when there are more persons, and also when there is a wider item difficulty range.

Christian, why do you need a higher item separation? An item Separation of 2.5 indicates that, statistically, we can distinguish high difficulty items from middle difficulty items from low difficulty items. This is usually enough for most measurement purposes and also for confirming the construct validity (= item difficulty hierarchy) of the items.

Similarly, a person Separation of 2.5 indicates that, statistically, we can distinguish high ability persons from middle ability persons from low ability persons. This is usually enough for most measurement purposes and also for confirming the predictive validity (= person ability hierarchy) of the persons.

Mike,

I believe that my item separation is where it should be but am concerned about my person separation.  The study has already undergone and I am receiving my data as secondary.  Thus, I cannot add more items to my survey to increase person separation.  I may not need to be concerned about this but I wanted to confirm.  This is my first use of Rasch.

Christian, Rasch is probably the only psychometric methodology that reports Item Separation (-> Item Reliability). All other methodologies report Test Reliability = Person Reliability -> Person Separation.

The general rule is: if your item separation is low, then you need a bigger sample of persons. This is helpful for deciding whether we have collected enough data to produce statistically valid findings.

Mike, This is helpful information. Obviously I cannot change the people I test but to know my Item Separation is high enough is helpful.  Thank you Mike.