﻿ Compare statistics

# Scatterplot: Compare statistics plot

From the Plots menu, this enables the simple graphical or tabular comparison of equivalent statistics from two runs using a scatterplot (xy plot) produced by Excel. For most versions of Excel, the maximum number of items or persons that can be plotted is 32,000.

To automatically produce this Excel scatterplot of two sets of measures or fits statistics:

Select Compare Statistics on the Plots pull-down menu. If this dialog box is too big for your screen see Display too big.

Measures, standard errors, fit statistics indicate which statistics are to be compared.

Display with columns generates a line-printer graphical-columns plot. It is displayed as Table 34.

The first column is the Outfit Mean-Square of this analysis.

The third column is the Outfit Mean-Square of the Right File (exam12lopf.txt in this case)

The second column is the difference.

The fourth column is the identification, according to the current analysis.

Persons or items are matched and listed by Entry number.

Table 34.1

+-----------------------------------------------------------------------------+

|      PERSON       | Outfit MnSq Difference  |  exam12lopf.txt   | File Compa|

|0     1           3|-2          0           2|0     1           3| NUM  LABEL|

|-------------------+-------------------------+-------------------+-----------|

|      .   *        |      *     .            |   *  .            |  1  21101 |

|      .       *    |         *  .            |      .     *      |  2  21170 |

|  *   .            |            .*           |   *  .            |  3  21174 |

....

|     *.            |            *            |     *.            | 35  22693 |

+-----------------------------------------------------------------------------+

# If two files are cross-plotted, please enter the Label field number in one of the files (either of the green arrows). Click on Display for one one of the files, and count across the fields to the Label field.

There are several decisions to make:

1. Do you want to plot person (row) or item (column) statistics?

2. Which statistic for the x-axis?

3. Which statistic for the y-axis?

4. Do you want to use the statistic from this analysis or from the PFILE= or IFILE= of another analysis?

5. Do you want to display in the statistics as Columns in a Table or as an Excel scatterplot or both?

If you are using the statistic from a PFILE= or IFILE= and Winsteps selects the wrong column, then identify the correct column using the "Statistic field number" area.

When two measures are compared, then their standard errors are used to construct confidence bands when "Plot confidence bands" is checked:

Here the item calibrations in the current analysis are being compared with the item calibrations in file IFILE=SFIF.txt from another analysis. This is the columnar output:

TABLE 34.1 An MCQ Test: administration was Comput ZOU630WS.TXT Apr 21  2:21 2006

INPUT: 30 STUDENTS  69 TOPICS  REPORTED: 30 STUDENTS  69 TOPICS  2 CATS     3.60.2

--------------------------------------------------------------------------------

+----------------------------------------------------------------------------------------------------+

|     Measures      |       Differences        |     Measures      | Comparison                      |

|                   |                          |     SFIF.txt      |                                 |

|-4                1|-2                       5|-3                2| NUM TOPIC                       |

|-------------------+--------------------------+-------------------+---------------------------------|

|   *          .    |       .         *        |          *.       |  1  nl01  Month                 |

|              .  * |  *    .                  |         * .       |  2  nl02  Sign                  |

|    *         .    |       .          *       |           .*      |  3  nl03  Phone number          |

|  *           .    |       .                * |           .    *  |  4  nl04  Ticket                |

|    *         .    |       .                 *|           .      *|  5  nl05  building              |

|*             .    |       .              *   |           .*      |  6  nm01  student ticket        |

|        *     .    |       .        *         |           .  *    |  7  nm02  menu                  |

|       *      .    |       .           *      |           .   *   |  8  nm03  sweater               |

|        *     .    |       .        *         |           . *     |  9  nm04  Forbidden City        |

|   *          .    |       .      *           |      *    .       | 10  nm05  public place          |

|       *      .    |       .   *              |        *  .       | 11  nm06  post office           |

|   *          .    |       .    *             |     *     .       | 12  nm07  sign on wall          |

|          *   .    |       *                  |       *   .       | 13  nh01  supermarket           |

|         *    .    |       .      *           |           .*      | 14  nh02  advertisement         |

|              .*   |    *  .                  |         * .       | 15  nh03  vending machine       |

|        *     .    |       .       *          |           .*      | 16  nh04  outside store         |

|              *    |       *                  |           *       | 17  nh05  stairway              |

|        *     .    |    *  .                  | *         .       | 18  nh06  gas station           |

|          *   .    |    *  .                  |   *       .       | 19  nh07  Taipei                |

|         *    .    |       .         *        |           .    *  | 20  nh08  window at post office |

|              *    |     * .                  |        *  .       | 21  nh09  weather forecast      |

|          *   .    |       .   *              |           *       | 22  nh10  section of newspaper  |

|         *    .    |       .           *      |           .     * | 23  nh11  exchange rate         |

|             *.    |       *                  |          *.       | 24  il01  open                  |

|            * .    |       .   *              |           .*      | 25  il02  vending machine       |

+----------------------------------------------------------------------------------------------------+

and the plotted output:

We are selecting only the first 4 characters of the item label, e.g., "nl01" and plotting only the Label:

1. Plots with confidence bands:

The points are plotted by their labels by Excel. The curved lines are the approximate 95% two-sided confidence bands (smoothed across all the points). They are not straight because the standard errors of the points differ. In this plot called "Plot-Empirical line" (red arrow), the dotted line is the empirical equivalence (best-fit) line. Right-click on a line to reformat or remove it.

A line parallel to the identity line is shown on the "Plot-Identity line" (blue arrow) by selecting the tab on the bottom of the Excel screen. This line is parallel to the standard identity line (of slope 1), but goes through the origin of the two axes. This parallel-identity line goes through the mean of the two sets of measures (vertical and horizontal).

The plotted points are in the Excel Worksheet (green arrow). You can edit the data points and make any other changes you want to the plots.

 Cell and Column Descriptions for Scatterplots of Measures with Confidence Bands Cell Description A1 Scatterplot B1 TITLE= D1 Date and time F1 CI= (Confidence Interval is) G1 1.96 (normal deviate for 95% 2-sided confidence bands) H1 68%=1.00, 90%=1.65, 95%=1.96, 99%=2.58 (Typical normal deviates for 2-sided confidence bands) B22 (or similar), B23 Mean of Measure 1 in Column B and its population S.D. D22 (or similar), D23 Mean of Measure 2 in Column D and its population S.D. Column Meaning Formula for Row B A Entry Entry number of Person or Item B Measure 1 Measure on y-axis C S.E. 1 Standard Error of Measure in column B D Measure 2 Measure on y-axis E S.E. 2 Standard Error of Measure in column D F PERSON or ITEM Person or Item Label G C.I. - Identity (for Identity-line Confidence Band) =SQRT(C3^2+E3^2)*G1*0.5 H Upper x - Identity (for upper Confidence Band on Identity-line plot) =((B22+B3+D3-D22)/2-G3) I Upper y - Identity =((D22+B3+D3-B22)/2+G3) J Lower x - Identity (for lower Confidence Band on Identity-line plot) =((B22+B3+D3-D22)/2+G3) K Lower y - Identity =((D22+B3+D3-B22)/2-G3) L C.I. - Empirical (for Empirical-line Confidence Band) =SQRT( (C3/B23)^2+ (E3/D23)^2)*G1*0.5 M Upper x - Empirical (for upper Confidence Band on Empirical-line plot) =(B22+((((B3-B22)/(2*B23))+((D3-D22)/(2*D23))-L3)*B23)) N Upper y - Empirical =(D22+((((B3-B22)/(2*B23))+((D3-D22)/(2*D23))+L3)*D23)) O Lower x - Empirical =(B22+((((B3-B22)/(2*B23))+((D3-D22)/(2*D23))+L3)*B23)) P Lower y - Empirical =(D22+((((B3-B22)/(2*B23))+((D3-D22)/(2*D23))-L3)*D23)) Q t-statistic (of difference between Measures relative to their means) =((B3-B22+D22-D3)/SQRT(C3^2+E3^2))

The relationship between the variables is summarized in the lower cells.

Empirical slope = S.D.(y-values) / S.D.(x-values)

Intercept = intersection of the line with  empirical slope through the point: mean(x-values), mean(y-values)

Predicted y-value = intercept with y-axis + x-value * empirical slope

Predicted x-value = intercept with x-axis + y-value / empirical slope

Disattenuated correlation approximates the "true" correlation without measurement error.

Disattenuated correlation = Correlation / sqrt (Reliability(x-values) * Reliability(y-values))

In Row 1, the worksheet allows for user-adjustable confidence bands.

2. Plots without confidence bands

The plot can be edited with the full functionality of Excel.

The Worksheet shows the correlation of the points on the two axes.

Help for Winsteps Rasch Measurement Software: www.winsteps.com. Author: John Michael Linacre

 Forum Rasch Measurement Forum to discuss any Rasch-related topic

Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments, George Engelhard, Jr. & Stefanie Wind Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
Winsteps Tutorials Facets Tutorials Rasch Discussion Groups

Coming Rasch-related Events
Jan. 5 - Feb. 2, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 10-16, 2018, Wed.-Tues. In-person workshop: Advanced Course in Rasch Measurement Theory and the application of RUMM2030, Perth, Australia (D. Andrich), Announcement
Jan. 17-19, 2018, Wed.-Fri. Rasch Conference: Seventh International Conference on Probabilistic Models for Measurement, Matilda Bay Club, Perth, Australia, Website
Jan. 22-24, 2018, Mon-Wed. In-person workshop: Rasch Measurement for Everybody en español (A. Tristan, Winsteps), San Luis Potosi, Mexico. www.ieia.com.mx
April 10-12, 2018, Tues.-Thurs. Rasch Conference: IOMW, New York, NY, www.iomw.org
April 13-17, 2018, Fri.-Tues. AERA, New York, NY, www.aera.net
May 22 - 24, 2018, Tues.-Thur. EALTA 2018 pre-conference workshop (Introduction to Rasch measurement using WINSTEPS and FACETS, Thomas Eckes & Frank Weiss-Motz), https://ealta2018.testdaf.de
May 25 - June 22, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 27 - 29, 2018, Wed.-Fri. Measurement at the Crossroads: History, philosophy and sociology of measurement, Paris, France., https://measurement2018.sciencesconf.org
June 29 - July 27, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
July 25 - July 27, 2018, Wed.-Fri. Pacific-Rim Objective Measurement Symposium (PROMS), (Preconference workshops July 23-24, 2018) Fudan University, Shanghai, China "Applying Rasch Measurement in Language Assessment and across the Human Sciences" www.promsociety.org
Aug. 10 - Sept. 7, 2018, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Sept. 3 - 6, 2018, Mon.-Thurs. IMEKO World Congress, Belfast, Northern Ireland www.imeko2018.org
Oct. 12 - Nov. 9, 2018, Fri.-Fri. On-line workshop: Practical Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

Our current URL is www.winsteps.com