Weighting items and persons

There are circumstances in which certain items are to be given more influence in constructing the measures than others. For instance, certain items may be considered critical to the demonstration of competence. Though Winsteps supports several methods, IWEIGHT= is simplest for items, and PWEIGHT= for persons. Another approach is to replicate the data for particular items. This can be done with FORMAT= without changing the data file. Items can also be rescored from say, 0-1 to 0-2, but this makes variable maps difficult to interpret.

 

Unweighted and Weighted analysis: unweighted data is preferable for calibrating the Rasch items. This is because each observation is modeled to contribute one unit of independent statistical information. The effect of weighting is to distort the distribution of independent statistical information in the data. A practical approach is:

 

Step 1. Analyze the data without weighting. Investigate misfit, construct validity,  etc.

 

Step 2. Weight the items. Compare the item calibrations with weighted and unweighted data to identify where there are discrepancies.

 

The true reliability of the measures is from the unweighted analysis. Weighting introduces an arbitrariness into the analysis. One solution is to adjust the weights to maintain the unweighted reliability = Ru. The reliability of the weighted analysis, using an initial set of weights, = Rw. We can then scale  the weights using the Spearman-Brown Prophecy Formula:  S = Ru * (1-Rw) / ((1-Ru)*Rw)). Multiply the initial set of weights by S. Then the weighted and unweighted reliabilities should be the same.

 

Standard errors and fit statistics: The weights applied to items or persons are used in computing the measure estimates, standard errors and fit statistics. When using significance tests with weighting, normalize the weights so that the total amount of independent statistical information in the data is not over- or under-inflated, i.e., when using PWEIGHT= with an observed sample size of N, multiply all PWEIGHT= values by N / (sum of all weights).

 

The standard is weights = 1.

 

When an item or person is weighted as 2, then the estimation acts as though that item or person appears twice in the data file.

 

When an item or person is weighted as 0, then that person does not influence the estimates, standard errors or fit statistics of other persons and items, but does have measure, standard error and fit statistics computed on all observations for itself. This is useful for evaluating pilot or off-dimensional items, or measuring idiosyncratic persons.

 

Estimation with weighting

Observation = Xni

Expected value (computed using the Rasch model) = Eni

 

Accumulated raw score = Accumulated raw score = Xni * IWEIGHT * PWEIGHT

Accumulated expected score = Accumulated expected score = Eni * IWEIGHT * PWEIGHT

Accumulated marginal count for item = Accumulated marginal count for item + IWEIGHT * PWEIGHT

Accumulated marginal count for person = Accumulated marginal count for person + IWEIGHT * PWEIGHT

 

Special rules apply when IWEIGHT=0 or PWEIGHT=0.

IWEIGHT=0 the item totals are incremented by PWEIGHT. The person totals are not incremented.

PWEIGHT=0 the person totals are incremented by IWEIGHT. The item totals are not incremented.

 

JMLE Estimation Accumulated expected score (for each person and each item) = Accumulated raw score (for each person and each item).

 


 

Weight Selection for Tables 23 and 24: On the output tables menu, these are the options for persons and/or items. When IWEIGHT= or PWEIGHT= are used in estimation, reports can be adjusted to reflect those weights or not. Weights of zero are useful for pilot items, variant items or persons with unusual characteristics. These can be reported exclusively or excluded from reports.

 

(1) all items or persons are reported, with their weights (the standard). Tables 23 and 24 are computed as though all weights are 1.

 

(2) items or persons with a weight of 0 are excluded from the reporting. Tables 23 and 24 are computed as though all weights are 1, but zero weights are omitted.

 

(3) only items or persons with a weight of 0 are reported. Tables 23 and 24 are computed only from items or persons with a weight of 0.

 

(4) all items or persons are reported as though they have a weight of 1.

 


 

Example 1: MCQ items are scored 0-1. CR items are scored 0-0.5-1. How can we combine them in one Winsteps analysis.

 

Approach A (recommended). Double the scores of the CR items to 0-1-2, and then IWEIGHT= them by 0.5.

 

Approach B (not recommended). Double the scores of the MCQ items to 0-(1)-2, and double the scores of the CR items to 0-1-2. IWEIGHT= them all by 0.5. This gives the MCQ items a rating scale in which the middle category is not observed, making their ICCs steeper.

 

Example 2: What is the cut-score in a weighted analysis corresponding to a cut-score in an unweighted analysis?

 

Here is an approach:

1.In the unweighted analysis, identify the logit value of the cut-score. Save the person measures to Excel.

2.In the weighted analysis, save the person measures to Excel.

3.Cross-plot the weighted person measures (y-axis) against the unweighted person measures (x-axis)

4.Use the Excel "trend line" function to obtain a reasonable curve through the person-measure points.

5.Identify the value on the y-axis (weighted cut-score measure) corresponding to the value on the x-axis of the unweighted cut-score measure.


Help for Winsteps Rasch Measurement Software: www.winsteps.com. Author: John Michael Linacre

For more information, contact info@winsteps.com or use the Contact Form
 

Facets Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download
Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download

State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied
 
Rasch, Winsteps, Facets online Tutorials

 

Forum Rasch Measurement Forum to discuss any Rasch-related topic

Click here to add your email address to the Winsteps and Facets email list for notifications.

Click here to ask a question or make a suggestion about Winsteps and Facets software.

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

 


 

 
Coming Rasch-related Events
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
Jan. 22-24, 2018, Mon-Wed. In-person workshop: Rasch Measurement for Everybody en español (A. Tristan, Winsteps), San Luis Potosi, Mexico. www.ieia.com.mx
April 10-12, 2018, Tues.-Thurs. Rasch Conference: IOMW, New York, NY, www.iomw.org
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Sept. 3 - 6, 2018, Mon.-Thurs. IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

 

Our current URL is www.winsteps.com

Winsteps® is a registered trademark
 


 
Concerned about aches, pains, youthfulness? Mike and Jenny suggest Liquid Biocell