P339: Comparing Correlation Accuracy on Skewed vs Centered Data: Practical Guidelines for Evaluating Clinical Outcome Assessments
Poster Presenter
Nicolai D. Ayasse
Clinical Outcome Assessment Scientist
Critical Path Institute United States
Objectives
Correlations are commonly used to evaluate relationships with clinical outcome assessments (COAs), and multiple correlation types are available for use with ordinal and continuous data. We simulated data to evaluate correlation type accuracy for different conditions for ordinal and continuous data.
Method
Two continuous variables were drawn from normal or skew-normal distributions. Ordinal items were produced by polychotomizing a variable using centered or skewed response thresholds. The correlation strength, number of response options in the ordinal item(s), and sample size were varied.
Results
Polychoric (applied to pairs of two ordinal items) or polyserial (applied to pairs of one ordinal and one continuous item) correlations demonstrated good accuracy and consistently outperformed Spearman correlations when their assumptions were met. The assumptions of the polychoric correlation were met when both ordinal items had normal latent variables underlying them, regardless of whether the response thresholds were centered or skewed, and the assumptions of the polyserial correlation were met when the continuous item was normally distributed and the ordinal item had a normal latent variable underlying it, regardless of whether the response thresholds were centered or skewed. Although the accuracy of polychoric and polyserial correlations decreased when their assumptions were violated by using skew-normal variables, Spearman correlations did not always demonstrate better accuracy under these conditions. It is notable that, under conditions where the Spearman correlation was less accurate, it tended to under-estimate the true correlation strength, while under conditions where the polychoric or polyserial correlations were less accurate, they tended to over-estimate the true correlation strength.
Since latent variables are by definition unobserved, one would not be able to differentiate from observed data between an ordinal item with a skewed underlying latent variable versus with a normal underlying latent variable but skewed response thresholds. Therefore, a final analysis was conducted based on the observable distribution of the items and found that broadly the polychoric or polyserial correlations demonstrated better or similar accuracy than Spearman correlations.
Conclusion
When correlating two ordinal items or an ordinal and continuous item, applying a polychoric or polyserial correlation will be a more accurate choice to a Spearman correlation if one can be confident of assumptions being met. However, even when there is uncertainty regarding assumptions, it will not always be a better choice to apply Spearman correlations. Our results demonstrate that under most conditions where items are observed to be either centered or skewed, using the polychoric (ordinal-ordinal pairs) or polyserial (ordinal-continuous pairs) correlation type is a defensible choice. Use of a correlation type that consistently produces under-estimates could result in an item being removed unnecessarily, and use of a correlation type that consistently produces over-estimates could result in overconfidence in the adequacy of the instrument’s measurement properties. It is important for COA developers to be aware of the performance of these different correlations under various conditions to ensure that they are making unbiased evaluations of their COA items and scores.
