Diagnosing Misfit

What do Infit Mean-square, Outfit Mean-square, Infit Zstd (z-standardized), Outfit Zstd (z-standardized) mean?

 

General rules:

Mean-squares show the size of the randomness, i.e., the amount of distortion of the measurement system. 1.0 are their expected values. Values less than 1.0 indicate observations are too predictable (redundancy, model overfit). Values greater than 1.0 indicate unpredictability (unmodeled noise, model underfit). Mean-squares usually average to 1.0, so if there are high values, there must also be low ones. Examine the high ones first, and temporarily remove them from the analysis if necessary, before investigating the low ones.

 

If the mean-squares average much below 1.0, then the data may have an almost Guttman-pattern. Please use much tighter convergence criteria.

 

Zstd are t-tests of the hypotheses "do the data fit the model (perfectly)?". These are reported as z-scores, i.e., unit normal deviates. They show the improbability (significance). 0.0 are their expected values. Less than 0.0 indicate too predictable. More than 0.0 indicates lack of predictability. If mean-squares are acceptable, then Zstd can be ignored. They are truncated towards 0, so that 1.00 to 1.99 is reported as 1. So a value of 2 means 2.00 to 2.99, i.e., at least 2. See Score files for more exact values.

 

The Wilson-Hilferty cube root transformation converts the mean-square statistics to the normally-distributed z-standardized ones. For more information, please see Patel's "Handbook of the Normal Distribution".

 

Guidelines:

(a) Look for negative bi-serial correlations and large response residuals. Explain or eliminate these first.

(b) If Zstd is acceptable, usually <|2| or <|3|, then there may not be much need to look further.

(c) If mean-squares indicate only small departures from model-conditions, then the data are probably useful for measurement.

(d) If there are only small proportion of misfitting elements, including or omitting them will make no substantive difference. If in doubt, do analyses with and without them and compare results.

(e) If measurement improves without misfitting elements, then

(i) omit misfitting elements

(ii) do an analysis without them and produce an anchorfile=

(iii) edit the anchorfile= to reinstate misfitting elements.

(iv) do an analysis with the anchorfile.

The misfitting elements will now be placed in the measurement framework, but without degrading the measures of the other elements.

 

Anchored runs:

Anchor values may not exactly accord with the current data. To the extent that they don't, they fit statistics may be misleading. Anchor values that are too central for the current data tend to make the data appear to fit too well. Anchor values that are too extreme for the current data tend to make the data appear noisy.

 

Mean-square interpretation:

>2.0 Distorts or degrades the measurement system.

1.5 - 2.0 Unproductive for construction of measurement, but not degrading.

0.5 - 1.5 Productive for measurement.

<0.5 Less productive for measurement, but not degrading. May produce misleadingly good reliabilities and separations.

 

In general, mean-squares near 1.0 indicate little distortion of the measurement system, regardless of the Zstd value.

 

Evaluate high mean-squares before low ones, because the average mean-square is usually forced to be near 1.0.

 

Outfit mean-squares:  influenced by outliers. Usually easy to diagnose and remedy. Less threat to measurement.

Infit mean-squares: influenced by response patterns. Usually hard to diagnose and remedy. Greater threat to measurement.