Meaning of the Fair Average:

The "fair average" transforms the Rasch measure back into an expected average raw response value. This value is in a standardized environment in which all other elements interacting with this element have a zero measure or the mean measure of all elements in their facet. This is "fair" to all elements in the facet, e.g., this adjusts raw ratings for severe and lenient raters. This enables a "fair" comparison to be made in the raw score metric, in the same way that the measure does on the linear latent variable. Fair-M uses the facet element means as the baseline. Fair-Z uses the facet local origins (zero points) as the baseline. These are set by Fair average=.

 

The original purpose of Facets (in 1986) was to construct software that would automatically adjust for differences in rater severity/leniency. So this has been done for your data. No adjustment or trimming of usefully-fitting raters is necessary, regardless of their severity/leniency.

 

But Facets does assume that the average leniency of the raters is at the required standard of severity/leniency (usually by centering the rater facet at zero logits). If you see, from evidence external to the data, that the average leniency of the of the raters is too high or too low, then please

1. include an additional "adjustment" facet in your analysis with a ? in Models=

2. this facet has one element.

3. Anchor this element at the adjustment value

4. Include the element in all the observations in the dataset by using dvalues=

5. perform an analysis of all the data including the adjustment element The rater leniencies are now correct, but the fair averages are not correct for the adjusted leniencies.

6. output an anchorfile

7. replace the ? in the model specification with an X

8. analyze the anchorfile with the dataset

9. there will be displacements of the size of the adjustment value. The fair averages are correct for the adjusted leniences.

 

This procedure will require several attempts before it produces the correct results.

 

Standard Error of the Fair Average:

We can approximate it by using two elements from the Facet as reported in Table 7.

facet with similar characteristics but a different measure.

S.E. of Fair score for element 1 =  S.E. of Measure for element 1 * (Fair score for element 1 - Fair score for element 2) / (Measure for element 1 - Measure for element 2)

Standard errors are always written as positive numbers.

 

Example:

Anne: Fair average = 4.64  measure = -.07  S.E. of measure = .17

Chris: Fair average = 4.23  measure = -.25

S.E. of Anne's Fair average = .17 * (4.64 - 4.23) / (-.07 - -0.25) = 0.39 score points.

 

Calculation of the Fair Average Score

The observed average score is the average rating received by the element. The logit measure is the linear measure implied by the observations. This is adjusted for the measures of whatever other elements of other facets participated in producing the observed data. It is often useful to transform these measures back into the original raw score metric to communicate their substantive meaning. Fair Average does this. It is the observed average adjusted for the measures of the other elements encountered. It is the observed average that would have been received if all the measures of the other elements had been located at the average measure of the elements in each of their facets.

 

A basic many-facet Rasch model for observation Xnmij is:

 

log ( Pnmijk / Pnmij(k-1)) = Bn - Am - Di - Cj - Fk

 

where

Bn is the ability of person n, e.g., examinee: Mary,

Am is the challenge of task m, e.g., an essay: "My day at the zoo".

Di is the difficulty of item i, e.g., punctuation,

Cj is the severity of judge j, e.g., the grader: Dr. Smith,

Fk is the barrier to being observed in category k relative to category k-1,

where k=0 to t, and F0 0.

 

To compute the fair average for person n (or task m, item i, judge j), set all element parameters except Bn (or Am, Di, Cj) to their mean (or zero) values. Thus, the model underlying a fair rating, when Fair=Mean, is:

 

log ( Pnmijk / Pnmij(k-1)) = Bn - Amean - Dmean - Cmean - Fk

 

or, when Fair=Zero, it becomes:

 

log ( Pnmijk / Pnmij(k-1)) = Bn - Fk

 

and the Fair average is sum (k Pnmijk) across categories 0 to t.

 

Example: Students are rated on 6 items. I want a fair-average for each student on each item.

1. Perform the analysis of all the data.

2. Output an Anchorfile=

3. Unanchor the students (remove the ,A for students in Labels=)

4. Analyze the anchorfile one item at a time by commenting out all items except 1 in Labels=

5. The reported measures and fair-averages for the students will be the fair averages for each item.

6. To assemble these fair-averages, output a Scorefile= from each one-item analysis to, say, Excel, selecting student identification and fair average fields.

 

---

 

Verifying the Fair Average

 

Analyze your data in Facets. Choose a Facet and an element whose Fair Average in Table 7 you want to verify.

 

In Facets,

"Output Files Menu"

"Residuals File"

"Select fields to output"

"Uncheck all"

Check - observation, expected value, element numbers

OK

Output to Excel.

 

In Excel, sort on the element numbers for your Facet.

Delete all rows except those for your element of your Facet.

 

Count and Sum the observations for your element of your Facet.

The count and sum should agree with Facets Table 7.

Average the observations. This should agree with the Table 7 Observed Average.

 

Now for the Fair Average:  This is the expected value for your element when it encounters elements of mean difficulty (usually 0) in all the other facets. Extreme-score elements are excluded from computing the mean.

 

From Facets Table 7, choose elements near the mean of the other facets.

 

In the Excel table, discover the expected values for your element combined with the mean elements of the other facets. The expected values should agree with the Table 7 Fair Average.

 

---

 

Problems with the Fair Average

 

If the "Fair averages" do not monotonically increase with the element measures, then

the misalignment of fair score with Rasch measures can occur when some items/task/raters etc. have different rating scales to other items/task/raters and all candidates are not rated on all rating scales.

 

For instance,

you do task 1 which has a rating scale from 0-10

but I do task 2 which has a rating scale from 0-5

 

We both have a measure of 2.00 logits.

 

Then your "fair score" will be 8.2 on the 0-10 item

But my "fair score" will be 4.1 on the 0-5 item

 

To get around this,

1. Write out an Anchorfile= from the Facets analysis.

2. Construct dummy data in which every candidate has a rating on every task (it doesn't matter what the value of the rating is).

3. Analyze the dummy data

4. The "fair score" for each candidate will be averaged across all the tasks.