Cut-off points

Setting pass-fail cut-off points requires a precise conceptualization of the cut-off point. There are many possible conceptualizations. Here is one that is usually easy to explain:

 

In the "Essays" example, we want a cut-off point for the examinees between "5" and "6" on the rating scale.

 

So let's conceptualize this as:

"1,000 examinees whose performance is at the cut-off point between 5 and 6 would have an average rating of 5.5 on a "standard" (average) essay when rated by a standard (average) reader."

 

Table 7.2.1  Essay Measurement Report  (arranged by MN).

-------------------------------------------------------------------------------------------------

|  Obsvd  Obsvd  Obsvd  Fair-M|        Model | Infit      Outfit   |Estim.|                     |

|  Score  Count Average Avrage|Measure  S.E. | MnSq ZStd  MnSq ZStd|Discrm| N Essay             |

-------------------------------------------------------------------------------------------------

|  1843.0   384.0   4.8   4.73|    .00   .04 | 1.00  -.2   .99  -.3|      | Mean (Count: 3)     |

 

Table 7.3.1  Reader Measurement Report  (arranged by MN).

----------------------------------------------------------------------------------------------------------------

|  Obsvd  Obsvd  Obsvd  Fair-M|        Model | Infit      Outfit   |Estim.| Exact Agree. |                     |

|  Score  Count Average Avrage|Measure  S.E. | MnSq ZStd  MnSq ZStd|Discrm| Obs %  Exp % | Nu Reader           |

----------------------------------------------------------------------------------------------------------------

|   460.8    96.0   4.8   4.73|    .00   .08 | 1.00  -.1   .99  -.2|      |              | Mean (Count: 12)    |

 

According to Table 7:

The average difficulty of the essays is 0.00 - this would be the "standard" essay difficulty

The average reader has a severity/leniency of 0.00

 

According to Table 8:

 

Table 8.1  Category Statistics.

--------------------------------------------------------------------------------------------------

|      DATA            |  QUALITY CONTROL  |RASCH-ANDRICH|  EXPECTATION  |  MOST  |.5 Cumul.| Cat|

| Category Counts  Cum.| Avge  Exp.  OUTFIT| THRESHOLDS  |  Measure at   |PROBABLE|Probabil.|PEAK|

|Score   Used   %    % | Meas  Meas   MnSq |Measure  S.E.|Category  -0.5 |  from  |    at   |Prob|

--------------------------------------------------------------------------------------------------

|  5      232  20%  66%|  -.01    .00  1.0 |   .04    .07|    .05    -.24|    .04 |   -.17  | 27%|

|  6      207  18%  84%|   .24    .22  1.0 |   .23    .07|    .61     .33|    .23 |    .32  | 29%|

 

A rating of 5.5 on the rating scale has a Rasch-Andrich threshold (step calibration) of 0.33 [see "EXPECTATION at -0.5" for category 6 in Table 8]

So the cut-off level for 7 to 8 is (0.0 + 0.0 + 0.33) = 0.33 logits.

 

You can check this by looking at the "Fair Average" column in Table 7 for the examinees. Look for a candidate with a fair average close to 5.5. The candidate measure should be close to 0.33.

 

Table 7.1.1  examinee Measurement Report  (arranged by MN).

-------------------------------------------------------------------------------------------------

|  Obsvd  Obsvd  Obsvd  Fair-M|        Model | Infit      Outfit   |Estim.|                     |

|  Score  Count Average Avrage|Measure  S.E. | MnSq ZStd  MnSq ZStd|Discrm| Nu examinee         |

-------------------------------------------------------------------------------------------------

|    198     36     5.5   5.50|    .33   .12 | 1.17   .8  1.17   .8|  .70 |  8 8                |