Scoring table construction

Top Up Down  A A

Constructing a score-to-measure table for a mnay-faceted situation can be challenging. Here is a procedure:

 

1. Do an analysis of your data. Let's assume it has 3 facets: 1. Examinees. 2. Raters. 3. Items. We want a score-to-measure table for the items with a standard rater.

 

2. If "disconnected" subsets are reported, then resolve that through group-anchoring or direct anchoring. We need to have the set of definitive measures.

 

3. Write out an anchorfile= with everything anchored.

 

4. Delete the current data= reference. We will construct a new data file

 

5. Replace the raters with a "standard" rater in the rater facet, with measure 0 (or whatever leniency you want your standard rater to have)

 

2, Rater, A ; assuming facet 2 is the rater facet

10, standard rater, 0 ; this is to be the standard rater for the score table

*

 

6. Identify the items that are to be part of the standard test. They must have anchor values. For convenience, let's assume there are 5 items. Renumber them with element numbers 1 to 5. The previous element numbers won't be needed here. Let's say they are on a rating scale 0-4. So the possible scores range from 0 to 5*4 = 20.

 

3, Item, A ; assumer facet 3 is items

1, (whatever its name is), (whatever its anchor measure value is)

....

5, (whatever its name is), (whatever its anchor measure value is)

*

 

7. delete all current examinee element labels. Replace them with artificial examinee labels. One for every possible score on the test, e.g.,

 

1, Examinee ; these aren't anchored

1000 = Examinee who scored 0

1001-1020 ; I am assuming that 20 is the maximum score on a 5 item test.

*

 

8. set up a new data file, giving the corresponding scores. There are 5 items in my example, numbered 1-5, and the possible scores are 0 to 20 assigned to examinees 1000 to 1020. Rater 10 is the standard rater. So this is the data file:

 

data=

1000, 10, 1-5, 0, 0, 0, 0, 0 ; raw score of 0

1001, 10, 1-5, 1 ,0, 0, 0, 0 ; it does not matter which item is item 1. It is only the raw score that matters.

1002, 10, 1-5, 2 ,0, 0, 0, 0

1003, 10, 1-5, 3 ,0, 0, 0, 0

1004, 10, 1-5, 4 ,0, 0, 0, 0

1005, 10, 1-5, 4, 1, 0, 0, 0

1006, 10, 1-5, 4, 2, 0, 0, 0

1007, 10, 1-5, 4, 3, 0, 0, 0

1008, 10, 1-5, 4, 4, 0, 0, 0

1009, 10, 1-5, 4, 4, 1, 0, 0

1010, 10, 1-5, 4, 4, 2, 0, 0

1011, 10, 1-5, 4, 4, 3, 0, 0

1012, 10, 1-5, 4, 4, 4, 0, 0

1013, 10, 1-5, 4, 4, 4, 1, 0

1014, 10, 1-5, 4, 4, 4, 2, 0

1015, 10, 1-5, 4, 4, 4, 3, 0

1016, 10, 1-5, 4, 4, 4, 4, 0

1017, 10, 1-5, 4, 4, 4, 4, 1

1018, 10, 1-5, 4, 4, 4, 4, 2

1019, 10, 1-5, 4, 4, 4, 4, 3

1020, 10, 1-5, 4, 4, 4, 4, 4 ; raw score of 20

 

9. Perform this analysis. Items, raters and the rating scale are anchored. Examinees are not. Examinees 1000 to 1020 will have scores 0 to 20. And so their measures correspond to 0 to 20. Ignore all other statistics.

 

10. The examinee scores and measures provide your score-to-measure table.


Help for Facets Rasch Measurement Software: www.winsteps.com.