Combining Items into Super-Items and Testlets using Excel

Testlets are groups of locally dependent items, such as the questions about the same topic. Testlets are obvious in theory, but obscure in practice. If the local dependency of all the items is about the same across testlets, then the testlet effect disappears. So, "testlets" are not the problem, the problem is unevenness of local dependency across the testlets. Table 23.99 - inter-item dependency - tends to be too local. It rarely gives us a good picture of dependency within testlets or across testlets.

 

Suggestion: in each item label, put a testlet code. Then from the individual-item analysis, output Winsteps Table 28 - item subtotals by testlet. Testlets with average item mean-squares much below 1.0 have noticeably greater dependency (of some type) than the other testlets. These testlets would be the obvious candidates to become PCM items. However, there are usually many reasons for local dependency across items within a dataset, so we would want to see the Table 28 findings confirmed by Table 23.1, etc. Each more dependent testlet should also be the dominant feature in a Contrast.

 


 

How are you conceptualizing the testlets?

 

A. If the testlets are conceptualized as groups of separate items, then please put a testlet-code in column 1 of the item label. You can then obtain item-level summary statistics in Winsteps using ISUBTOTAL=1 and Table 27. You can also obtain person-by-testlet measures using DPF=1 and Table 31, and person-group-by-testlet using DPF=1, DIF=person group code column and Table 33.

 

B. If two or more items are highly dependent (so that their residuals are highly correlated in Table 23.99) , then it may be better combine them into one "testlet" item. Testlet Response Theory is  technically complex and requires special software, but we can approximate it by combining the dependent items into partial-credit items.

 

From a Winsteps analysis of the separate items, Output files menu, RFILE= to Excel (with headings). In Excel, sum the separate items into new combined-item columns. Save the Excel file In Winsteps, Excel/RSSST menu, Excel option. Click on the saved Excel file. Select the combined items from the list of Excel columns. Format a Winsteps control and data file.  (You can also select the original uncombined items, but will need to weight them zero with IWEIGHT= in the new analysis.) Analyze the new Winsteps control and data file with ISGROUPS=0  (partial-credit model)

 

Cross-plot the person measures from the original analysis and the partial credit analysis. Are there any differences? Which analysis produces more meaningful results?

 

How to combine items into super-items or testlets

1.

If your data are already in columns in an Excel worksheet, jump to 8.

 

2.

Place your data in a Winsteps control and data file.

Our example is the Knox Cube Test data, exam1.txt

3.

Launch Winsteps with your control and data file.

Analyze your data

Verify that all is correct

4.

For the scored responses, Output Files menu

RFILE=

or for the original responses,  Output Files menu IPMATRIX=

5.

Output File dialog box:

Excel

Permanent file

OK

6.

File name dialog box:

Enter name for testlet Excel file

 

Our example: Exam1-testlet.xls

 

Save

7.

Excel displays with the data.

 

Check that the item labels are in Row 1.

8.

Insert columns for the testlets, super-items or combined items

 

In our example, we will have a super-item item for each length of tapping pattern: Two, Three, ..., Seven

9.

Sum the responses to the relevant items into the super-item columns

10.

Save or “Save As” the Excel worksheet

11.

Winsteps menu bar

Excel/RSSST

Selection box: Excel

12.

Select the saved Excel worksheet

13.

The Excel column labels display with the items and super-items

14.

Copy-and-paste the super-item column labels under “Item Response Variables”

Other items can be copied, especially if you plan to weight them zero with IWEIGHT=

Also copy the person column labels under “Person Label Variables”

15.

Click on “Construct Winsteps File”

16.

Save the Winsteps control and data file with a suitable name.

17.

The new Winsteps control and data file displays. It contains the superitems.

CODES= contains the scores in the superitems.

 

18.

Remove the ; before GROUPS = 0 because we expect each super-item to have its own structure (partial credit model).

19.

Save the Winsteps control and data file

20.

Click on “Launch Winsteps”

21.

The analysis of the super-items is performed

22.

Appendix: Impact of the super-items on measurement.

 

Output PFILE=pf.txt from the super-item analysis, then in the standard analysis, Scatterplot the person measures against pf.txt

 

In this example, the logit spread of the person measures is less for the super-items than for the standard items. This indicates that there is local dependency among the standard items that comprise each super-item.

23.

In the plot in 22. all the measures have shifted. The super-items are automatically correctly weighted. However, the mean difficulty of the items changes when separate items are collapsed into super-items. In this situation, either (i) set the mean ability of the persons to be the same in both analyses,

UPMEAN=0 ; anchor the mean person ability at 0 logits

or (ii) anchor an unchanged item at the same logit difficulty in both analyses,

IAFILE=*

1 0  ;  anchor item 1 at 0 logits

*


Help for Winsteps Rasch Measurement Software: www.winsteps.com. Author: John Michael Linacre

For more information, contact info@winsteps.com or use the Contact Form
 

Facets Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download
Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download

State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied
 
Rasch, Winsteps, Facets online Tutorials

 

Forum Rasch Measurement Forum to discuss any Rasch-related topic

Click here to add your email address to the Winsteps and Facets email list for notifications.

Click here to ask a question or make a suggestion about Winsteps and Facets software.

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

 


 

 
Coming Rasch-related Events
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
Jan. 22-24, 2018, Mon-Wed. In-person workshop: Rasch Measurement for Everybody en español (A. Tristan, Winsteps), San Luis Potosi, Mexico. www.ieia.com.mx
April 10-12, 2018, Tues.-Thurs. Rasch Conference: IOMW, New York, NY, www.iomw.org
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Sept. 3 - 6, 2018, Mon.-Thurs. IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

 

Our current URL is www.winsteps.com

Winsteps® is a registered trademark
 


 
Concerned about aches, pains, youthfulness? Mike and Jenny suggest Liquid Biocell