|
Scoring table construction |
Top Up Down
A A |
Constructing a score-to-measure table for a mnay-faceted situation can be challenging. Here is a procedure:
1. Do an analysis of your data. Let's assume it has 3 facets: 1. Examinees. 2. Raters. 3. Items. We want a score-to-measure table for the items with a standard rater.
2. If "disconnected" subsets are reported, then resolve that through group-anchoring or direct anchoring. We need to have the set of definitive measures.
3. Write out an anchorfile= with everything anchored.
4. Delete the current data= reference. We will construct a new data file
5. Replace the raters with a "standard" rater in the rater facet, with measure 0 (or whatever leniency you want your standard rater to have)
2, Rater, A ; assuming facet 2 is the rater facet
10, standard rater, 0 ; this is to be the standard rater for the score table
*
6. Identify the items that are to be part of the standard test. They must have anchor values. For convenience, let's assume there are 5 items. Renumber them with element numbers 1 to 5. The previous element numbers won't be needed here. Let's say they are on a rating scale 0-4. So the possible scores range from 0 to 5*4 = 20.
3, Item, A ; assumer facet 3 is items
1, (whatever its name is), (whatever its anchor measure value is)
....
5, (whatever its name is), (whatever its anchor measure value is)
*
7. delete all current examinee element labels. Replace them with artificial examinee labels. One for every possible score on the test, e.g.,
1, Examinee ; these aren't anchored
1000 = Examinee who scored 0
1001-1020 ; I am assuming that 20 is the maximum score on a 5 item test.
*
8. set up a new data file, giving the corresponding scores. There are 5 items in my example, numbered 1-5, and the possible scores are 0 to 20 assigned to examinees 1000 to 1020. Rater 10 is the standard rater. So this is the data file:
data=
1000, 10, 1-5, 0, 0, 0, 0, 0 ; raw score of 0
1001, 10, 1-5, 1 ,0, 0, 0, 0 ; it does not matter which item is item 1. It is only the raw score that matters.
1002, 10, 1-5, 2 ,0, 0, 0, 0
1003, 10, 1-5, 3 ,0, 0, 0, 0
1004, 10, 1-5, 4 ,0, 0, 0, 0
1005, 10, 1-5, 4, 1, 0, 0, 0
1006, 10, 1-5, 4, 2, 0, 0, 0
1007, 10, 1-5, 4, 3, 0, 0, 0
1008, 10, 1-5, 4, 4, 0, 0, 0
1009, 10, 1-5, 4, 4, 1, 0, 0
1010, 10, 1-5, 4, 4, 2, 0, 0
1011, 10, 1-5, 4, 4, 3, 0, 0
1012, 10, 1-5, 4, 4, 4, 0, 0
1013, 10, 1-5, 4, 4, 4, 1, 0
1014, 10, 1-5, 4, 4, 4, 2, 0
1015, 10, 1-5, 4, 4, 4, 3, 0
1016, 10, 1-5, 4, 4, 4, 4, 0
1017, 10, 1-5, 4, 4, 4, 4, 1
1018, 10, 1-5, 4, 4, 4, 4, 2
1019, 10, 1-5, 4, 4, 4, 4, 3
1020, 10, 1-5, 4, 4, 4, 4, 4 ; raw score of 20
9. Perform this analysis. Items, raters and the rating scale are anchored. Examinees are not. Examinees 1000 to 1020 will have scores 0 to 20. And so their measures correspond to 0 to 20. Ignore all other statistics.
10. The examinee scores and measures provide your score-to-measure table.
Help for Facets Rasch Measurement Software: www.winsteps.com.