Mike,

I've attached a Word document detailing the first two items of a 50 item high school earth science exam. I'm particularly puzzled by the fact that Table 30.1 does not seem to reflect any gender DIF for item 1, while Table 30.4 clearly reports this exists. Perhaps 30.4 is over sensitive to the possible non-uniform functioning of item 1 for lower performers, while 30.1 does not seem to see this as significant. What are your thoughts? I've included item 2 only for contrast as an item for which both tables seem to agree.

Uve, Table 30.1 is well authenticated.

In Table 30.4, we are applying large-sample statistics to only two data-points. The standard errors of the reported values are probably large. This is a problem with nearly all reported statistical results. The precision of the computation (often reported to 4 or 6 decimal places) is much higher than the precision of the statistical estimates (their standard errors).

If in doubt, always take the more conservative finding, i.e., that the null hypothesis is not rejected. In this case, Table 30.1 is more conservative.

Looking at Table 14.1, we can diagnose a likely problem. The item difficulty of Item 1 is anchored with a displacement of 0.8 logits. Our fit statistics, DIF computations, etc. are formulated based on free (unanchored) estimates. So please do fit and DIF investigations before anchoring item difficulties or person abilities.

Unanchoring did the trick. Both tables now report no significant DIF. Thanks!

Thanks for telling us, Uve. It is good to know that theory and practice agree in this case