Correlations: point-biserial, point-measure, residual

In Rasch analysis, we use item correlations as an immediate check that the response-level scoring makes sense. If the observed correlation is negative, something may have gone wrong (MCQ miskey, reversed survey item, etc.)

In general, correlations are much too difficult to interpret, so we switch over to using mean-squares. The "expected correlation" indicates when conventional rules such as eliminate items with point-biserials less than 0.2 are misleading.

Item correlations are difficult to interpret because they are influenced by:

1. predictability of the data

2. targeting of the item on the person sample

3. distribution of the person sample

In Rasch analysis, we are chiefly concerned about the predictability of the data when assessing item quality, so we examine the predictability directly using the mean-square statistics, rather than indirectly through the correlations.

 

All correlations are computed as Pearson product-moment correlation coefficients. If you wish to compute other correlations, the required data are in XFILE= IPMATRIX=, IFILE= or PFILE=. The Biserial correlation can be computed from the point-biserial.

 

Point-Biserial and Point-Measure Correlations

Control Instruction

Observed value

Explanation

Expected value (EXP.)

PTBISERIAL= Yes

PTBISERIAL= Exclude

PBSX

PTBISERL-EX

 

Point-biserial (or point-polyserial) correlation excluding the current observation from the raw score. Computes the point-biserial or point-polyserial correlation coefficients, rpbis, for persons and items. This is the Pearson product-moment correlation between the scored responses (dichotomies and polytomies) and the "rest scores", the corresponding total (marginal) scores excluding the scored responses to be correlated.  This is a point-biserial correlation for dichotomies, or a point-polyserial correlation for polytomies. Extreme (perfect, maximum possible and zero, minimum possible) scores are included in the computation, but missing observations are omitted pairwise. The Biserial correlation can be computed from the Point-biserial. This correlation loses its meaning when there are missing data or with CUTLO= or CUTHI=. Specify PTBISERIAL=X instead.

PBSX-E

PBX-E

PTBISERIAL= All

PTBISERIAL= Include

PBSA

PTBISERL-AL

 

 

Point-biserial correlation for all observations including the current observation in the raw score. Computes the Pearson correlation between the total (marginal) scores including all responses and the responses to the targeted item and person. This is a point-biserial correlation for dichotomies, or a point-polyserial correlation for polytomies.  This correlation loses its meaning when there are missing data or with CUTLO= or CUTHI=. Specify PTBISERIAL=N instead.

PBSA-E

PBA-E

PTBISERIAL= No

PTBISERIAL= RPM

PTBISERIAL= Measure

PTMA

PTMEASUR-AL

 

Point-measure correlation for all observations. Computes the Pearson point-measure correlation coefficients, rpm between the observations and the measures, estimated from the raw scores including the current observation or the anchored values. Measures corresponding to extreme scores are included in the computation.

PTMA-E

PMA-E

PTBISERIAL= X

PTMX

PTMEASUR-EX

Point-measure correlation excluding the current observation from the estimation of the measure. Computes the Pearson point-measure correlation coefficients, rpm between the observations and the measures or anchor values adjusted to exclude the current observation. Measures corresponding to extreme scores are included in the computation.

PTMX-E

PMX-E

 

Point-correlations are always reported for items. Point-correlations are reported for persons when (i) all the items are dichotomies, (ii) all the items have three categories, or (iii) all the items are in the same item group (Andrich Rating Scale Model). Otherwise, the observed and expected correlations are reported as zero in PFILE=.

 

Example: Here are the point-biserial-type coefficient options for Example0.txt, Item 1 (Watch Birds).

Control Instruction

PTBISERIAL= ALL

PTBISERIAL= YES

PTBISERIAL= NO

PTBISERIAL= X

Calculation:

Correlate observation with

raw score

raw score - current observation

Rasch measure

Rasch measure estimated from
raw score - current observation

Table 14, etc., Heading

PTBISERL-AL

PTBISERL-EX

PTMEASURE-A

PTMEASUR-EX

IFILE= / PFILE= Heading

PBSA

PBSX

PTMA

PTMX

Observed correlation

.69

.66

.64

.61

Expected correlation (-E)

.53

.47

.49

.48

 

In Table 14.1 and other measure Tables: Point-Biserial Correlations for dichotomies, and Point-Polyserial for polytomies

 

when PTBISERIAL=Yes

PTBSE is the point-biserial correlation between the responses to this item by each person and the total marginal score by each person (omitting the response to this item). This is the "point-biserial corrected for spuriousness". Henrysson, S. (1963). Correction for item-total correlations in item analysis. Psychometrika, 28, 211-218.

 

when PTBISERIAL=All

PTBSA is the point-biserial correlation between the responses to this item by each person and the total marginal score by each person (including the response to this item). This is the conventional point-biserial.

 

In Table 14.3 and other or distractor Tables:

when PTBISERIAL=Yes or PTBISERIAL=All

PTBSD is the distractor point-biserial correlation between the indicated response to this item (scored 1 and other responses scored 0) by each person and the total marginal score by each person.

 

There is a closer match between Table 14.1 and Table 14.3 when PTBISERIAL=All

 

PTBIS=Y or E (indicated by PTBSE): The point-biserial correlation rpbis for item i (when i=1,L for persons n=1,N) is the correlation between the observation for each person on item i and the total score for each person on all the items excluding item i (and similarly for the point-biserial for each person):

 

 

PTBIS=All (indicated by PTBSA): All the observations are included in the total score:

 

 

where X1,..,XN are the responses, and Y1,..,YN are the total scores. The range of the correlation is -1 to +1.

 

Under classical (raw-score) test theory conventions, point-biserial correlations should be 0.3, 0.4 or better. Under Rasch conditions, point-biserial (or point-measure) correlations should be positive, so that the item-level scoring accords with the latent variable, but the size of a positive correlation is of less importance than the fit of the responses to the Rasch model, indicated by the mean-square fit statistics.

 

Point-Measure Correlations

 

PTBIS=No (indicated by PTMEA): The correlation between the observations and the Rasch measures:

 

 

where X1,..,XN are the responses by the persons (or on the items), and Y1,..,YN are the person measures (or the item easinesses = - item difficulties). The range of the correlation is -1 to +1.

 

Jaspen, N. (1946) "Serial Correlation", Psychometrika, 11, 23-30.

Olsson, U., Drasgow, F. and Dorans, N.J. (1982) "The Polyserial Correlation Coefficient", Psychometrika, 47, 337-347.

 


 

The Expected Value of a Correlation

 

Interpreting an observed value is made easier if we can compare it with its expected value. Is is much higher than expected or much lower than expected?

 

The general formula for a Pearson correlation coefficient is shown above. Suppose that Xn is Xni the observation of person n on item i. Yn is Bn the ability of person n, then the point-measure correlation is:

 

 

According to the Rasch model, the expected value of Xni is Eni and its model variance around the expectation is Wni. For dichotomies, Eni is the Rasch probability of success and Wni = Eni(1-Eni). For polytomies, Eni and Wni are given by RSA, p. 100 also at www.rasch.org/rmt/rmt34e.htm.  For JMLE estimates, Sum(Eni) = Sum(Xni) for n=1,N. Thus an estimate of the expected value of the point-measure correlation is given by the Rasch model proposition that: Xni = Eni ±Wni. Other variance terms are much smaller.

 

 

±Wni is a random residual with mean 0 and variance Wni. Its cross-product with any other variable is modeled to be zero. Thus, simplifying,

 

and similarly for the point-biserial correlations. Here is an example point-measure correlation and its expected value computed with an Excel spreadsheet:

 

Xni

Xni-Xi

(Xni-Xi)2

Bn

Bn-B

(Bn-B)2

(Xni-Xi)(Bn-B)

Eni

Eni-Xi

(Eni-Xi)(Bn-B)

Eni(1-Eni) = Wni

(Eni-Xi)2 + Wni

0

-0.50

0.25

-1.33

-1.66

2.74

0.83

0.16

-0.34

0.56

0.13

0.25

1

0.50

0.25

-0.03

-0.36

0.13

-0.18

0.41

-0.09

0.03

0.24

0.25

0

-0.50

0.25

1.33

1.01

1.01

-0.50

0.73

0.23

0.23

0.20

0.25

1

0.50

0.25

1.33

1.01

1.01

0.50

0.73

0.23

0.23

0.20

0.25

average

Xi = 0.50

 

(i) ∑ = 1.00

average

B = 0.33

 

(ii) ∑ = 4.89

(iii) ∑ = 0.65

 

 

(iv) ∑ = 1.05

 

(v) ∑ = 1.00

Observed point-measure Correlation = (iii) / √ ((i)(iii)) =

0.29






Expected point-measure Correlation = (iv) / √ ((v)(ii)) =

0.47



Difficulty of item i = 0.35 logits

Table courtesy of Peter Karaffa

 


 

Point-Correlations and the Partial Credit Model or Grouped Rating Scale Model: ISGROUPS=

The usual computations for correlation coefficients assume that all scores or measures within each of the two variables is in the same metric. For instance, when correlating age with height for children, the ages are all in months and the heights are all in centimeters. If we mix ages in months with ages in years, or height in centimeters with height in inches, then the age-height correlations can become meaningless.

The same situation can arise when we use the Partial Credit Model (ISGROUPS=0) or Grouped Rating Scale Model (ISGROUPS=AABB..). In particular, the Expected value of a Point-Correlation can be reported as negative. This is a warning is that the reported correlations are meaningless.

 


 

Disattenuated correlation coefficients

"Attenuated" means "reduced". "Disattenuation" means "remove the attenuation".

The observed correlation between two variables is attenuated (reduced toward zero) because the variables are measured with error. So, when we remove the measurement error (by a statistical operation), the resulting correlation is disattenuated. Disattenuated correlations are always further from zero.

Algebraically:

{A} and {B} are the "true" values of two variables. Their true (disattenuated) correlation is

Disattenuated ("true") correlation = r(A,B)

But the observed values of the variables are measured with error {A±a}, {B±b}, so the observed correlation is

observed correlation = r(A,B) * (var(A)*var(B))/((var( + a +a ²)*(var( + b + b ²)).

"Disattenuation" reverses this process.

 

If the reliability of {A} is RA, and the reliability of {B} is RB, then the disattenuated correlation between {A} and {B} is:

disattenuated r(A,B) = r(A,B) / (RA*RB).

 


 

Point-Correlations in Table 14.3, etc. and DISFILE=

 

For the item point-biserial correlations, PTBISERIAL=A and PTBISERIAL=E:,  scored observations are correlated with their corresponding person raw scores.For the point-measure correlations, PTBISERIAL=M, scored observations are correlated with their corresponding Rasch measures.

 

Each response code (scored or "missing") is correlated only with scored response codes. With PTBISERIAL=Yes or All, the correlation is between the occurrence and the person raw score, indicated by PTBSD CORR. When this correlation is high positive for a correct MCQ option, then the item exhibits convergent validity. When this correlation is low or negative for incorrect MCQ options, then the item exhibits discriminant validity. Krus, D. J. & Ney, R. G. (1978) Convergent and discriminant validity in item analysis. Educational and Psychological Measurement, 38, 135-137.
Each scored response code is correlated only with the other scored codes: "1" for target code "0" for other scored codes. . Missing responses are correlated with all responses.
PTBSA CORR is PTBIS = All (include the current item in the person score)
PTBSE CORR is PTBIS = Yes (exclude the current item in the person score)
PTMEA CORR is PTBIS = No (correlate with the person measure)
The "Missing" point-measure correlation is the correlation between the raw scores for the persons and the responses to the item with "missing" scored 1 and the other responses scored 0.
The category (or option) correlation is always computed the same way:
1. We focus on a particular response to a particular item.
2. Everyone who chose that response to the item is scored "1", everyone else is scored "0". Missing data are ignored.
3. The scores of "1" or "0" are correlated with the person measures.
4. We do this for every response to every item.

 

Example: Point-Measure Correlations of Person Measures with Item 6 Options (a,b,c,d):

Person measure

Person's response

to item 6

Category a

b

c

d

0.5

d

0

0

0

1

0.8

c

0

0

1

0

0.9

b

0

1

0

0

0.4

d

0

0

0

1

1.0

a

1

0

0

0

1.2

b

0

1

0

0

 

Correlation of measure

with category:

0.32

0.64

0.00

-0.89

 

For the options reported in Table 14.3 and DISFILE=

 For each item and each option, a correlation is computed for each code in CODES=

   If the response in the data file is the code, then a value of 1 is assigned

   If the response in the data file is not the code and in CODES=, then a value of 0  is assigned

   If the response in the data file is not in CODES= and scored with MISSING-SCORED=, then a value of 0 is assigned.

   If the response in the data file is not in CODES= and not scored with MISSING-SCORED=, then the response is ignored.

   For each value of 0 or 1, an average response score is computed = (raw score / count scored observations)

   The correlation is between the (weighted) values and the average response scores or Rasch measures

   

 For each item, one correlation is computed for all codes in the data not in CODES=

   If the response in the data file is not in CODES=, then a value of 1 is assigned

   If the response in the data file is in CODES=, then a value of 0  is assigned

   For each value of 0 or 1, an average response score is computed = (raw score / count scored observations)

   The correlation is between the weighted values and the average response scores or Rasch measures

 

If your data file contains unscored data or missing data, then the item point-correlations will differ from the option point-correlations. If you want the correlations to be the same, then force missing data to be scored 0:

1. Do your usual analysis

2. Output IFILE=if.txt, PFILE=pf.txt, SFILE=sf.txt

3. Add to your control file:

   IAFILE=if.txt  ; anchor everything at their estimated values

   PAFILE=pf.txt

   SAFILE=sf.txt

   CODES=your codes + all other codes in the data file. For example, if your codes are A,B,C,D and missing data is blank, then CODES=" ABCD "

4. Do the analysis again. Output Table 14.3, DISFILE=. The values should now agree.

5. Do NOT use the fit statistics estimated from this analysis.

 

Suggestion: since the point-biserial is not defined when there is unscored missing data, please use the point-measure correlation, PTBISERIAL=M.


Help for Winsteps Rasch Measurement Software: www.winsteps.com. Author: John Michael Linacre

For more information, contact info@winsteps.com or use the Contact Form
 

Facets Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download
Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation download

State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied
 
Rasch, Winsteps, Facets online Tutorials

 

Forum Rasch Measurement Forum to discuss any Rasch-related topic

Click here to add your email address to the Winsteps and Facets email list for notifications.

Click here to ask a question or make a suggestion about Winsteps and Facets software.

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

 


 

 
Coming Rasch-related Events
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
Jan. 22-24, 2018, Mon-Wed. In-person workshop: Rasch Measurement for Everybody en español (A. Tristan, Winsteps), San Luis Potosi, Mexico. www.ieia.com.mx
April 10-12, 2018, Tues.-Thurs. Rasch Conference: IOMW, New York, NY, www.iomw.org
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Sept. 3 - 6, 2018, Mon.-Thurs. IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

 

Our current URL is www.winsteps.com

Winsteps® is a registered trademark
 


 
Concerned about aches, pains, youthfulness? Mike and Jenny suggest Liquid Biocell