Diagnosing Misfit |
This is for 32-bit Facets 3.87. Here is Help for 64-bit Facets 4
What do Infit Mean-square, Outfit Mean-square, Infit Zstd (z-standardized), Outfit Zstd (z-standardized) mean?
General rules:
Mean-squares show the size of the randomness, i.e., the amount of distortion of the measurement system. 1.0 are their expected values. Values less than 1.0 indicate observations are too predictable (redundancy, model overfit). Values greater than 1.0 indicate unpredictability (unmodeled noise, model underfit). Mean-squares usually average to 1.0, so if there are high values, there must also be low ones. Examine the high ones first, and temporarily remove them from the analysis if necessary, before investigating the low ones.
If the mean-squares average much below 1.0, then the data may have an almost Guttman-pattern. Please use much tighter convergence criteria.
Zstd are t-tests of the hypotheses "do the data fit the model (perfectly)?". These are reported as z-scores, i.e., unit normal deviates. They show the improbability (significance). 0.0 are their expected values. Less than 0.0 indicate too predictable. More than 0.0 indicates lack of predictability. If mean-squares are acceptable, then Zstd can be ignored. They are truncated towards 0, so that 1.00 to 1.99 is reported as 1. So a value of 2 means 2.00 to 2.99, i.e., at least 2. See Score files for more exact values.
The Wilson-Hilferty cube root transformation converts the mean-square statistics to the normally-distributed z-standardized ones. For more information, please see Patel's "Handbook of the Normal Distribution".
Guidelines:
(a) Look for negative bi-serial correlations and large response residuals. Explain or eliminate these first.
(b) If Zstd is acceptable, usually <|2| or <|3|, then there may not be much need to look further.
(c) If mean-squares indicate only small departures from model-conditions, then the data are probably useful for measurement.
(d) If there are only small proportion of misfitting elements, including or omitting them will make no substantive difference. If in doubt, do analyses with and without them and compare results.
(e) If measurement improves without misfitting elements, then
(i) omit misfitting elements
(ii) do an analysis without them and produce an anchorfile=
(iii) edit the anchorfile= to reinstate misfitting elements.
(iv) do an analysis with the anchorfile.
The misfitting elements will now be placed in the measurement framework, but without degrading the measures of the other elements.
Anchored runs:
Anchor values may not exactly accord with the current data. To the extent that they don't, they fit statistics may be misleading. Anchor values that are too central for the current data tend to make the data appear to fit too well. Anchor values that are too extreme for the current data tend to make the data appear noisy.
Mean-square interpretation:
>2.0 Distorts or degrades the measurement system.
1.5 - 2.0 Unproductive for construction of measurement, but not degrading.
0.5 - 1.5 Productive for measurement.
<0.5 Less productive for measurement, but not degrading. May produce misleadingly good reliabilities and separations.
In general, mean-squares near 1.0 indicate little distortion of the measurement system, regardless of the Zstd value.
Evaluate high mean-squares before low ones, because the average mean-square is usually forced to be near 1.0.
Outfit mean-squares: influenced by outliers. Usually easy to diagnose and remedy. Less threat to measurement.
Infit mean-squares: influenced by response patterns. Usually hard to diagnose and remedy. Greater threat to measurement.
Diagnosing Misfit |
||||
Classification |
INFIT |
OUTFIT |
Explanation |
Investigation |
|
Noisy |
Noisy |
Lack of convergence Loss of precision Anchoring |
Final values in Table 0 large? Many categories? Large logit range? Displacements reported? |
Hard Item |
Noisy |
Noisy |
Bad item |
Ambiguous or negative wording? Debatable or misleading options? |
Muted |
Muted |
Only answered by top people |
At end of test? |
|
Item |
Noisy |
Noisy |
Qualitatively different item Incompatible anchor value |
Different process or content? Anchor value incorrectly applied? |
? |
Biased (DIF) item |
Stratify residuals by person group? |
||
Muted |
Curriculum interaction |
Are there alternative curricula? |
||
Muted |
? |
Redundant item |
Similar items? One item answers another? Item correlated with other variable? |
|
Rating scale |
Noisy |
Noisy |
Extreme category overuse |
Poor category wording? Combine or omit categories? Wrong model for scale? |
Muted |
Muted |
Middle category overuse |
||
Person |
Noisy |
? |
Processing error Clerical error Idiosyncratic person |
Scanner failure? Form markings misaligned? Qualitatively different person? |
High Person |
? |
Noisy |
Careless Sleeping Rushing |
Unexpected wrong answers? Unexpected errors at start? Unexpected errors at end? |
Low Person |
? |
Noisy |
Response set "Special" knowledge |
Unexpected right answers? Systematic response pattern? Content of unexpected answers? |
Muted |
? |
Plodding Caution |
Did not reach end of test? Only answered easy items? |
|
Person/Judge Rating |
Noisy |
Noisy |
Extreme category overuse |
Extremism? Defiance? |
Muted |
Muted |
Middle category overuse |
Conservatism? Resistance? |
|
Judge Rating |
Apparent unanimity |
Collusion? |
||
INFIT: OUTFIT: Muted: Noisy: |
information-weighted mean-square, sensitive to irregular inlying patterns usual unweighted mean-square, sensitive to unexpected rare extremes unmodeled dependence, redundancy, error trends unexpected unrelated irregularities |
Help for Facets Rasch Measurement and Rasch Analysis Software: www.winsteps.com Author: John Michael Linacre.
Facets Rasch measurement software.
Buy for $149. & site licenses.
Freeware student/evaluation Minifac download Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation Ministep download |
---|
Forum: | Rasch Measurement Forum to discuss any Rasch-related topic |
---|
Questions, Suggestions? Want to update Winsteps or Facets? Please email Mike Linacre, author of Winsteps mike@winsteps.com |
---|
State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied Rasch, Winsteps, Facets online Tutorials |
---|
Coming Rasch-related Events | |
---|---|
May 17 - June 21, 2024, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
June 12 - 14, 2024, Wed.-Fri. | 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024 |
June 21 - July 19, 2024, Fri.-Fri. | On-line workshop: Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com |
Aug. 5 - Aug. 7, 2024, Mon.-Wed. | 2024 Inaugural Conference of the Society for the Study of Measurement (Berkeley, CA), Call for Proposals |
Aug. 9 - Sept. 6, 2024, Fri.-Fri. | On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com |
Oct. 4 - Nov. 8, 2024, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
Jan. 17 - Feb. 21, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
May 16 - June 20, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
June 20 - July 18, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com |
Oct. 3 - Nov. 7, 2025, Fri.-Fri. | On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com |
Our current URL is www.winsteps.com
Winsteps® is a registered trademark