﻿ Estimation considerations: JMLE estimation bias

# Estimation considerations: JMLE estimation bias

Sample size

Facets produces estimates for any data set in which there is overlapping randomness in the observations across elements. It cannot estimate measures when the data have a Guttman pattern or there is only one observation element. But even in these cases, anchoring and other analytical technique may make the element measures estimable. See also "My analysis does not converge."

Estimates which are likely to have some degree of stability across samples require at least 30 observations per element, and at least 10 observations per rating-scale category.

There is no maximum sample size apart from those imposed by computer constraints. Large sample sizes tend to make convergence slower and fit statistics overly sensitive.

Estimation techniques: JMLE and PMLE

Facets generally uses JMLE (= Joint Maximum-Likelihood Estimation), but for Models=?,-? Facets uses a customized implementation of PMLE (= Pairwise Maximum-Likelihood Estimation). RSA and MFRM derive Newton-Raphson iteration equations for the estimation of Rasch measures using the JMLE and PMLE approach. In Facets, this approach is not robust enough against spiky data. Consequently, since the basic functional shape of all the estimation equations is the logistic ogive, Facets estimates measures by means of iterative curve-fitting to that shape. The resulting measures also accord with the JMLE and PMLE approach.

1. All the parameters are given reasonable starting values.
2. The estimate of each parameter (element measure or Andrich threshold) is updated as though the values of all the other parameters are known.
3. Then all the parameters are reestimated using the updated estimates of all the other parameters.
4. (3.) is repeated until no parameter value changes by more than the convergence limit.
5. At convergence, for every element or rating-scale category, the expected score = the observed score.

For a two-facet analysis, this process is the same as shown in the Excel spreadsheets linked from www.rasch.org/moulton.htm

This is explained in Linacre "Many-Facet Rasch Measurement"

Choice of estimation algorithm
JMLE: Every estimation method has strengths and weaknesses. The advantages of JMLE far outweigh its disadvantages. JMLE is estimable under almost all conditions including idiosyncratic data designs, arbitrary and accidental patterns of missing data, arbitrary anchoring (fixing) of parameter estimates, unobserved intermediate categories in rating scales, flexible equating designs and multiple different Rasch models in the same analysis. All elements are treated as equal. Each element of every facet has a parameter estimate, standard error and fit statistics obtained in exactly the same way.

The primary weakness of JMLE is that estimates can have estimation bias. In MFRM analyses, estimation bias is usually of minor concern because either the dataset is large or the structure of the data negates the importance of estimation bias.  The size of the estimation bias can be discovered using simulated datasets.

In a typical two-facet analysis, JMLE estimation bias is most obvious in a test of two dichotomous items (Andersen, 1973) with a large sample of persons, a two-facet analysis. In such an analysis, JMLE estimates the difference between the item difficulties of the two items to be twice its true value. In Winsteps, the estimation bias for two-response situations is corrected by specifying PAIRED=Yes. The estimation bias for longer tests is corrected by STBIAS=Yes, but there are drawbacks to bias correction as discussed in Winsteps Help. In many practical situations, the statistical bias is less than the standard errors of the estimates, and becomes negligible as test length and sample size increase. See Wright (1988).

PMLE: When Models=?,-? (or similar) are specified, Facets uses a customized version of PMLE. PMLE is also implemented in RUMM2030. PMLE produces unbiased estimates under suitable conditions.

MFRM analysis can be performed with other estimation methods including CMLE (Conditional Maximum Likelihood Estimation), MMLE (Marginal Maximum Likelihood Estimation) which can all produce better estimates than JMLE under ideal conditions. In practice, they all have the same crucial drawback. They model MFRM data as two-facet data. One of the MFRM facets is modeled to be the "item" facet. All the other MFRM facets are compressed into the second, "person", facet. After the "person" estimates are obtained from a two-facet analysis, those estimates are linearly decomposed into the estimates for all the elements of all the other facets. This approach limits admissible data designs, restricts element-anchoring and introduces distortions into the estimates of most elements. Changing the facet modeled to be the "item" facet results in different estimates of all elements of all facets.

Andersen, E. B. Conditional inference for multiple-choice questionnaires. British Journal of Mathematical and Statistical Psychology, 1973, 26, 31-44.

Wright, B.D. The Efficacy of Unconditional Maximum Likelihood (JMLE) Bias Correction. 1988. www.rasch.org/memo45.htm

JMLE estimation bias

Facets makes no correction for estimation bias in JMLE. The size of the bias adjustment (if any) depends on exact details of the data design. Estimation bias is almost always less than the standard error of measurement. Estimation bias is approximately a linear transformation of the unbiased estimates. This bias slightly expands the logit range of the elements. If you encounter a situation in which this slight expansion is crucial for decision-making, please do tell Winsteps.com.

To estimate the size of the estimation bias with Facets:

1.analyze your data with Facets. Output the element measures to Excel.

2.simulate many datasets (at least 10, but 100 would be better) using the Facets "Simulate" function

3.analyze all the simulated datasets. Use Facets in batch mode to do this.

4.average each element measure across all the simulated datasets.

5.crossplot the averaged measures (y-axis) against the original measures from 1 (x-axis).

6.the slope of the trend line in the plot indicates the size of the estimation bias. We expect the trend line to have a slope slightly above 1.0.

7.divide the original element measures by the slope of the trend line to obtain unbiased estimates.

8.the bias in the original standard errors approximates the square-root of the slope of the trend-line.

Help for Facets Rasch Measurement Software: www.winsteps.com Author: John Michael Linacre.

For more information, contact info@winsteps.com or use the Contact Form

 Forum Rasch Measurement Forum to discuss any Rasch-related topic

Click here to ask a question or make a suggestion about Winsteps and Facets software.

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

Coming Rasch-related Events
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
Jan. 22-24, 2018, Mon-Wed. In-person workshop: Rasch Measurement for Everybody en español (A. Tristan, Winsteps), San Luis Potosi, Mexico. www.ieia.com.mx
April 10-12, 2018, Tues.-Thurs. Rasch Conference: IOMW, New York, NY, www.iomw.org
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Sept. 3 - 6, 2018, Mon.-Thurs. IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

Our current URL is www.winsteps.com

Winsteps® is a registered trademark

Concerned about aches, pains, youthfulness? Mike and Jenny suggest Liquid Biocell