Reliability and separation of measures 
Winsteps report reliability and separation statistics treating the sample of measures as the population. If it is not the entire population, then the reliability and separation are slightly higher than the reported values.
Usually person and item reliability and separation have different applications and implications.
Person separation is used to classify people. Low person separation (< 2, person reliability < 0.8 )with a relevant person sample implies that the instrument may not be not sensitive enough to distinguish between high and low performers. More items may be needed.
Item separation is used to verify the item hierarchy. Low item separation (< 3 = high, medium, low item difficulties, item reliability < 0.9) implies that the person sample is not large enough to confirm the item difficulty hierarchy (= construct validity) of the instrument.
Reliability (separation index) means "reproducibility of relative measure location". It does not report on the quality of the data. So "high reliability" (of persons or items) means that there is a high probability that persons (or items) estimated with high measures actually do have higher measures than persons (or items) estimated with low measures. If you want high reliability, you need a wide sample and/or low measurement error. So, if you want high person (test) reliability, you need a person sample with a large ability (or whatever) range and/or an instrument with many items (or long rating scales). If you want high item reliability, you need a test with a large item difficulty range and/or a large sample of persons. Usually low item reliability is because the person sample size is too small to establish a reproducible item difficulty hierarchy.
Missing data: if some persons have missing observations, these can considerably reduce precision, and so lower reliability estimates. Suggestion: omit personrecords with missing data when estimating reliabilities.
Person (sample, test) reliability depends chiefly on
1) Sample ability variance. Wider ability range = higher person reliability.
2) Length of test (and rating scale length). Longer test = higher person reliability
3) Number of categories per item. More categories = higher person reliability
4) Sampleitem targeting. Better targeting = higher person reliability
It is independent of sample size. It is largely uninfluenced by model fit.
In general, Test Reliability reported by Classical Test Theory (Cronbach Alpha, KR20) is higher than Rasch Reliability. Rasch Reliability is higher than 3PL IRT Reliability.
Item reliability depends chiefly on
1) Item difficulty variance. Wide difficulty range = high item reliability
2) Person sample size. Large sample = high item reliability
It is independent of test length. It is largely uninfluenced by model fit.
Tentative guidelines:
Person reliability: Does your test discriminate the sample into enough levels for your purpose? 0.9 = 3 or 4 levels. 0.8 = 2 or 3 levels. 0.5 = 1 or 2 levels.
Item reliability: Low reliability means that your sample is not big enough to precisely locate the items on the latent variable.
Rater reliability: Low "separation" reliability is better, because we want raters to be reliably the same, not reliably different.
The Winsteps "person reliability" is equivalent to the traditional "test" reliability. Low values indicate a narrow range of person measures, or a small number of items. To increase person reliability, test persons with more extreme abilities (high and low), lengthen the test. Improving the test targeting may help slightly.
The Winsteps "item reliability" has no traditional equivalent. Low values indicate a narrow range of item measures, or a small sample. To increase "item reliability", test more people. In general, low item reliability means that your sample size is too small for stable item estimates based on the current data. If you have anchored values, then it is the item reliability of the source from which the anchor values emanate which is crucial, not the current sample.
The "model" person reliability (including measures for extreme scores) is an upper bound to this value, when persons are ordered by measures.
The "real" person reliability (including measures for extreme scores) is a lower bound to this value, when persons are ordered by measures
The traditional "test reliability", as defined by Charles Spearman in 1904, etc., is the "true person variance / observed person variance" for this sample on these test items. So it is really a "person sample reliability" rather than a "test reliability", where reliability = reproducibility of person ordering. The "true person variance" cannot be known, but it can be approximated. KR20 approximates it by summarizing item pointbiserials. Cronbach Alpha approximates it with an analysis of variance. Winsteps approximates it using the measure standard errors.
The separation coefficient and reliability computations are computed with and without any elements with extreme measures. Since the measures for extreme scores are imprecise, reliability statistics which include extreme scores are often lower than their nonextreme equivalents. Conventional computation of a reliability coefficient (KR20, Cronbach Alpha) includes persons with extreme scores. The classical reliability computation includes extreme scores (if any) is the conventional reliability, and usually produces an estimate between the MODEL and REAL values, closer to the MODEL or even above it.
KR20 value is an estimate of the value when persons are ordered by raw scores. CRONBACH ALPHA (KR20) KID RAW SCORE RELIABILITY is the conventional "test" reliability index. It reports an approximate test reliability based on the raw scores of this sample. It is only reported for complete data. An apparent paradox is that extreme scores have perfect precision, but extreme measures have perfect imprecision.
Winsteps computes upper and lower boundary values for the True Reliability. The lower boundary is the Real Reliability. The upper boundary is the Model Reliability. The unknowable True Reliability lies somewhere between these two. As contradictory sources of noise are remove from the data, the True Reliability approaches the Model Reliability
Cronbach Alpha and KR20 Reliability
Here is a check on the computations. Guilford reports 0.81. Winsteps reports 0.82. The difference is probably computational precision and rounding error.
Title = "Guilford Table 17.2. His Cronbach Alpha = KR20 = 0.81"
ni=8
item1=1
name1=1
&END
END LABELS
00000000
10000000
10100000
11001000
01010010
11101010
11111100
11111100
11110101
11111111
Conventionally, only a Person ("Test") Reliability is reported. The relationship between rawscorebased reliability (i.e., KR20, Cronbach Alpha) and measurebased reliability is complex, see www.rasch.org/rmt/rmt113l.htm  in general, Cronbach Alpha overestimates reliability, Rasch underestimates it. So, when it is likely that the Rasch reliability will be compared with conventional KR20 or Cronbach Alpha reliabilities (which are always computed assuming the data match their assumptions), then include extreme persons and report the higher Rasch reliability, the "Model" reliability, computed on the assumption that all unexpectedness in the data is in accord with Rasch model predictions. The big differences between Score and Measure reliabilities occur when
(a) there are extreme scores. These increase score reliability, but decrease measure reliability.
(b) missing data. Missing data always decreases measure reliability. If the missing data are imputed at their expected values (in order to make conventional reliability formulas computable), they increase score reliability. Winsteps attempts to adjust the rawscore reliability for this inflation in the rawscore reliability, but can only do the adjustment in an approximate way.
Winsteps also reports an item reliability, "true item variance / observed item variance". When this value is low, it indicates that the sample size may be too small for stable comparisons between items.
Anchored values are treated as though they are the "true values" of the MLE estimates. Their local standard errors are estimated using the current data in the same way as unanchored MLE standard error estimates. It is the measures (anchored or unanchored) and local standard errors that are used in the reliability computations. If you wish to compute reliabilities using different standard error estimates (e.g., the ones when the anchor values were generated), then please perform a separate reliability computation (using Excel).
You can easily check the Winsteps reliability estimate computation yourself.
Read the Winsteps PFILE= into an Excel spreadsheet.
Compute the STDEVP standard deviation of the person measures. Square it. This is the "Observed variance".
"Model" Reliability: Take the standard ERROR column. Square each entry. Sum the squared entries. Divide that sum by the count of entries. This is the "Model Error variance" estimate. Then,
Model Reliability = True Variance / Observed Variance = (Observed Variance  Model Error Variance) / Observed Variance.
"Real" Reliability: Take the standard ERROR column. Square each entry, SE². In another column, put SE²*Maximum [1.0, INFIT meansquare). Divide that sum by the count of entries. This is the "Real Error variance" estimate. Then,
Real Reliability = True Variance / Observed Variance = (Observed Variance  Real Error Variance) / Observed Variance.
Separation, Strata and Reliability
The crucial elements in the computation of reliability are the "True" variance and the Error variance. These are squared distances and so difficulty to conceptualize directly. It is easier to think of their squareroots, the "True" standard deviation (TSD) and the rootmeansquare standard error (RMSE).
SEPARATION coefficient is the ratio of the PERSON (or ITEM) TRUE S.D., the "true" standard deviation, to RMSE, the error standard deviation. It provides a ratio measure of separation in RMSE units, which is easier to interpret than the reliability correlation. This is analogous to the Fisher Discriminant Ratio. SEPARATION coefficient ² is the signaltonoise ratio, the ratio of "true" variance to error variance.
RELIABILITY (separation index) is a separation reliability. The PERSON (or ITEM) reliability is equivalent to KR20, Cronbach Alpha, and the Generalizability Coefficient. The relationship between SEPARATION coefficient and RELIABILITY (separation index) is
RELIABILITY = SEPARATION coefficient ²/(1+SEPARATION coefficient ²)
or SEPARATION coefficient = squareroot(RELIABILITY/(1RELIABILITY)).
Separation (if the outlying measures are accidental) or Strata (if the outlying measures represent true performances). These numbers are statistical abstractions, but there empirical meaning is indicated by locating the Separation or Strata levels in the observed distribution at (3 * "Observed S.D." / Separation) units apart, centered on the sample mean.
Error RMSE 
True SD 
True Variance 
Observed Variance 
Signal toNoise Ratio 
Separation = True SD / RMSE 
Strata = (4*Sep.+1) / 3 
Reliability = True Variance / Observed Variance 
1 
0 
0 
1 
0 
0 
0 
0 
1 
1 
1 
2 
1 
1 
1.67 
0.5 
1 
2 
4 
5 
2 
2 
3 
0.8 
1 
3 
9 
10 
3 
3 
4.33 
0.9 
1 
4 
16 
17 
4 
4 
5.67 
0.94 
There is more at www.rasch.org/rmt/rmt94n.htm and www.rasch.org/rmt/rmt163f.htm
SpearmanBrown Prophecy Formula for person "Test" reliability with different numbers of items (test lengths)
How many items (or persons) are required to produce the reliability I want with the sample of persons and the same type of items (or this test and the same type of persons)?
T = target number of items, RT = target person reliability
C = current number of items, RC = current person reliability
1. Predict number of items = T = C * RT * (1RC) / ( (1RT) * RC)
Example: the current test length is C = 10 items, and the current person reliability is RC = 0.3. We want a person reliability of RT = 0.8.
Target number of items is T = 10 * 0.8 * (10.3) / ( (10.8)* 0.3) = 94 items.
2. Predicted person reliability = RT = T * RC / ( C * (1RC) + T * RC)
Example: we have a test of C = 11 items of person reliability RC = 0.5, what is the predicted reliability of a test of T = 17 items?
Predicted person reliability RT = 17 * 0.5 / ( 11 * (10.5) + 17 * 0.5) = 0.61
Prophecy Formula for person "Test" reliability with different observed standard deviations of the person measures/scores
assuming the average error variance is unchanged.
T = target observed standard deviation, RT = target person reliability
C = current observed standard deviation, RC = current person reliability
1. Predict person standard deviation = T = C * sqrt ( (1RC) / (1RT) )
2, Predict person reliability = RT = 1  ( (1RC)*C² / T² )
TestRetest Reliability
is the correlation between the person measures obtained from two administrations of the same test to the same persons. The expected value of the testretest reliability is the person reliability of the first administration.
The "Smallest Detectable Difference" = "Smallest statistically significant difference in a person's measures" when a test is administered twice to a person under normal conditions = 1.96*(person standard deviation)*(testretest reliability).
Population and Sample Standard Deviation, Reliability, Separation and Strata
Winsteps assumes that the N statistics being summarized are the entire population. It reports the population observed Standard Deviation = P.SD, "True" Standard Deviation = T.P.SD, Reliability = P.Rel, Separation = P.Sep and Strata = P.Strata, average error S.D. = RMSE, where
P.Sep = √ ( P.Rel / (P.Rel + 1) )
RMSE = P.SD * √ ( 1  P.Rel )
If the N statistics are a sample of the entire population, then the summary statistics are the sample observed Standard Deviation = S.SD, "True" Standard Deviation = T.S.SD, Reliability = S.Rel, Separation = S.Sep and Strata = S.Strata, where
S.SD = P.SD * √ ( N / (N1))  the sample S.D. is bigger than the population S.D.
T.S.SD = √ ( S.SD**2  RMSE**2)
S.Sep = T.S.SD / RMSE
S.Strata = (4 * S.Sep + 1) / 3
S.Rel = ( 1 + (N1)*P.Rel ) / N
Help for Winsteps Rasch Measurement Software: www.winsteps.com. Author: John Michael Linacre
The Languages of Love: draw a map of yours!
For more information, contact info@winsteps.com or use the Contact Form
Facets Rasch measurement software.
Buy for $149. & site licenses.
Freeware student/evaluation download Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download 

Stateoftheart : singleuser and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied Rasch, Winsteps, Facets online Tutorials 

Forum  Rasch Measurement Forum to discuss any Raschrelated topic 
Click here to add your email address to the Winsteps and Facets email list for notifications.
Click here to ask a question or make a suggestion about Winsteps and Facets software.
Coming Winsteps & Facets Events  

May 22  24, 2018, Tues.Thur.  EALTA 2018 preconference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank WeissMotz), https://ealta2018.testdaf.de 
May 25  June 22, 2018, Fri.Fri.  Online workshop: Practical Rasch Measurement  Core Topics (E. Smith, Winsteps), www.statistics.com 
June 27  29, 2018, Wed.Fri.  Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org 
June 29  July 27, 2018, Fri.Fri.  Online workshop: Practical Rasch Measurement  Further Topics (E. Smith, Winsteps), www.statistics.com 
July 25  July 27, 2018, Wed.Fri.  PacificRim Objective Measurement Symposium (PROMS), (Preconference workshops July 2324, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org 
Aug. 10  Sept. 7, 2018, Fri.Fri.  Online workshop: ManyFacet Rasch Measurement (E. Smith, Facets), www.statistics.com 
Oct. 12  Nov. 9, 2018, Fri.Fri.  Online workshop: Practical Rasch Measurement  Core Topics (E. Smith, Winsteps), www.statistics.com 
Our current URL is www.winsteps.com
Winsteps^{®} is a registered trademark
John "Mike" L.'s Wellness Report:
I'm 72, take no medications and, March 2018, my doctor is annoyed with me  I'm too healthy! According to Wikipedia, the human body requires about 30 minerals, maybe more. There are 60 naturallyoccurring minerals in the liquid Mineral Supplement which I take daily. 