Investigating differences between rater groups
Videotapes of 2 different flight crews, consisting of 2 crew members each, the pilot-in-command (PIC) and the second-in-command (SIC), are rated on 10 items that measure technical and teamwork skills. 2 overall items assess crew technical and teamwork performance. There is one item for overall evaluation of the PIC, and one item for overall evaluation of the SIC. There are 3 groups of judges, each group consisting of 7 to 10 judges. Each judge rates both crews on all items (4-point Likert scale). I want to investigate the differences between the groups of judges.
Here is the skeleton of a suggested Facets control file:
Title="Air Crew Videotapes"
; facet 1 is judges
; facet 2 is judge group (a dummy facet)
; facet 3 is crew number
; facet 4 is crew component: crew, PIC or SIC
; facet 5 is items
Investigating differences between rater groups
?,#,?,?,?,Likert ; model each group to have its own rating scale (or partial credit)
?,?B,?,?,?B,R ; look for group by item interactions
?,?B,?B,?,?,R ; look for group by crew number interactions
rating scale = Likert,R4,Specific ;we model each group to have its own scale
1=SD ; compare the rating scale (or partial credit) structures
2=D ; reported for the 3 groups
1=Fred,,1 ;Fred is in group 1: Note the ,, to skip over the pre-set logit value field.
2=Mary,,3 ;Mary in group 3
3=Alfonso,,2 ;Alfonson in group 2
.... ;there will be subtotals by group
2,Group,A ;dummy facet - used for classification
1=Group A,0 ;anchored at 0 to remove for measurement,
2=Group B,0 ;group facet used for classification and summary fit statistics
3,Crew number ; the videotapes
1=Crew ;used for items 1-12
2=PIC ;used for item 13
3=SIC ;used for item 14
data= ;here are some example data lines:
2,3,1,1,1-12, 2,1,3,2,4,1,2,1,3,2,1,2 ;Mary in group 3 rates crew 1's crew on items 1-12
2,3,1,2,13, 4 ;Mary in group 3 rates crew 1's PIC on item 13
2,3,1,2,14, 3 ;Mary in group 3 rates crew 1's SIC on item 14
Here is the skeleton of a likely Facform control file:
Facform can be downloaded from www.winsteps.com/facets.htm
Original data format (say) in file rawdata.txt:
judge id cols 1-3
judge group cols 4
Crew number col 5
Item ratings cols 6-19 (14 items)
$Title = "Format the Air Crew judge data"
$Input = "rawdata.txt"
$Facets=5 ;define number of facets
$Flabel=1,"Judges" ;define facet names
$Label=1,$S1W3 ;identification for facet 1 judge
$Label=2,$S4W1 ;identification for facet 2 group
$Label=3,$S5W1 ;identification for facet 3 crew number
$label=4,1,"Crew" ; identification for facet 4 - the crew
item = 1
$Label=5,item ; identification for facet 5 - item number
$Rating=$S(item+5)W1 ; response to item 1 etc.
item = item +1 ; next item
$label=4,2,"PIC" ; identification for facet 4 - PIC
$Label=5,13,"PIC overall" ; identification for facet 5 - item 13
$Rating=$S18W1 ; response to item 13
$label=4,3,"SIC" ; identification for facet 4 - SIC
$Label=5,14,"SIC overall" ; identification for facet 5 - item 14
$Rating=$S19W1 ; response to item 14
Help for Facets Rasch Measurement Software: www.winsteps.com Author: John Michael Linacre.
For more information, contact firstname.lastname@example.org or use the Contact Form
|Facets Rasch measurement software.
Buy for $149. & site licenses.
Freeware student/evaluation download
Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download
|State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied|
Rasch, Winsteps, Facets online Tutorials
|Forum||Rasch Measurement Forum to discuss any Rasch-related topic|
Click here to add your email address to the Winsteps and Facets email list for notifications.
Click here to ask a question or make a suggestion about Winsteps and Facets software.
|Rasch Measurement Transactions (free, online)||Rasch Measurement research papers (free, online)||Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch||Applying the Rasch Model 3rd. Ed., Bond & Fox||Best Test Design, Wright & Stone|
|Rating Scale Analysis, Wright & Masters||Introduction to Rasch Measurement, E. Smith & R. Smith||Introduction to Many-Facet Rasch Measurement, Thomas Eckes||Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind||Statistical Analyses for Language Testers, Rita Green|
|Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar||Journal of Applied Measurement||Rasch models for measurement, David Andrich||Constructing Measures, Mark Wilson||Rasch Analysis in the Human Sciences, Boone, Stave, Yale|
|in Spanish:||Análisis de Rasch para todos, Agustín Tristán||Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez|
|Winsteps Tutorials||Facets Tutorials||Rasch Discussion Groups|
|Coming Rasch-related Events|
|April 10-12, 2018, Tues.-Thurs.||Rasch Conference: IOMW, New York, NY, www.iomw.org|
|April 13-17, 2018, Fri.-Tues.||AERA, New York, NY, www.aera.net|
|May 22 - 24, 2018, Tues.-Thur.||EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de|
|May 25 - June 22, 2018, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com|
|June 27 - 29, 2018, Wed.-Fri.||Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org|
|June 29 - July 27, 2018, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com|
|July 25 - July 27, 2018, Wed.-Fri.||Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org|
|Aug. 10 - Sept. 7, 2018, Fri.-Fri.||On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com|
|Sept. 3 - 6, 2018, Mon.-Thurs.||IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org|
|Oct. 12 - Nov. 9, 2018, Fri.-Fri.||On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com|
Our current URL is www.winsteps.com
Winsteps® is a registered trademark
John "Mike" Linacre, author of Winsteps, and Jenny use and recommend eco-friendly, safe and effective wellness and beauty products such as skincare