﻿ Scoring table construction

# Scoring table construction

Constructing a score-to-measure table for a many-faceted situation can be challenging. Here is a procedure:

1. Do an analysis of your data. Let's assume it has 3 facets: 1. Examinees. 2. Raters. 3. Items. We want a score-to-measure table for the items with a standard rater.

2. If "disconnected" subsets are reported, then resolve that through group-anchoring or direct anchoring. We need to have the set of definitive measures.

3. Write out an anchorfile= with everything anchored.

4. Delete the current data= reference. We will construct a new data file

5. Replace the raters with a "standard" rater in the rater facet, with measure 0 (or whatever leniency you want your standard rater to have)

2, Rater, A ; assuming facet 2 is the rater facet

10, standard rater, 0 ; this is to be the standard rater for the score table

*

6. Identify the items that are to be part of the standard test. They must have anchor values. For convenience, let's assume there are 5 items. Renumber them with element numbers 1 to 5. The previous element numbers won't be needed here. Let's say they are on a rating scale 0-4. So the possible scores range from 0 to 5*4 = 20.

3, Item, A ; assume facet 3 is items

1, (whatever its name is), (whatever its anchor measure value is)

....

5, (whatever its name is), (whatever its anchor measure value is)

*

7. delete all current examinee element labels. Replace them with artificial examinee labels. One for every possible score on the test, e.g.,

1, Examinee ; these aren't anchored

1000 = Examinee who scored 0

1001-1020 ; I am assuming that 20 is the maximum score on a 5 item test.

*

8. set up a new data file, giving the corresponding scores. There are 5 items in my example, numbered 1-5, and the possible scores are 0 to 20 assigned to examinees 1000 to 1020. Rater 10 is the standard rater. So this is the data file:

data=

1000, 10, 1-5, 0, 0, 0, 0, 0 ; raw score of 0

1001, 10, 1-5, 1 ,0, 0, 0, 0 ; it does not matter which item is item 1. It is only the raw score that matters.

1002, 10, 1-5, 2 ,0, 0, 0, 0

1003, 10, 1-5, 3 ,0, 0, 0, 0

1004, 10, 1-5, 4 ,0, 0, 0, 0

1005, 10, 1-5, 4, 1, 0, 0, 0

1006, 10, 1-5, 4, 2, 0, 0, 0

1007, 10, 1-5, 4, 3, 0, 0, 0

1008, 10, 1-5, 4, 4, 0, 0, 0

1009, 10, 1-5, 4, 4, 1, 0, 0

1010, 10, 1-5, 4, 4, 2, 0, 0

1011, 10, 1-5, 4, 4, 3, 0, 0

1012, 10, 1-5, 4, 4, 4, 0, 0

1013, 10, 1-5, 4, 4, 4, 1, 0

1014, 10, 1-5, 4, 4, 4, 2, 0

1015, 10, 1-5, 4, 4, 4, 3, 0

1016, 10, 1-5, 4, 4, 4, 4, 0

1017, 10, 1-5, 4, 4, 4, 4, 1

1018, 10, 1-5, 4, 4, 4, 4, 2

1019, 10, 1-5, 4, 4, 4, 4, 3

1020, 10, 1-5, 4, 4, 4, 4, 4 ; raw score of 20

9. Perform this analysis. Items, raters and the rating scale are anchored. Examinees are not. Examinees 1000 to 1020 will have scores 0 to 20. And so their measures correspond to 0 to 20. Ignore all other statistics.

10. The examinee scores and measures provide your score-to-measure table.

Help for Facets Rasch Measurement Software: www.winsteps.com Author: John Michael Linacre.

The Languages of Love: draw a map of yours!

 Forum Rasch Measurement Forum to discuss any Rasch-related topic

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

Coming Winsteps & Facets Events
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

Our current URL is www.winsteps.com