Measuring Polarization Updated

I have just updated my R-package to measure agreement, polarization, dispersion — whatever you want to call it — in ordered rating scales to R-Forge. Version 1.40 includes more extensive documentation and a long due update of the package vignette. I’ll push it to CRAN in a moment. Every time I work on this package, it strikes me how many times the ‘problem’ has been solved, how different the approaches are, and sadly how often standard deviations are still used.

Measuring Consensus

I have mentioned Cees van der Eijk’s measure of agreement before, and Leik’s measure of ordinal consensus. Unsurprisingly, others have come across this issue, discontent with the widespread use of standard deviations (inappropriate as this can be). Tastle & Wierman (2007) take a quite different approach, taking the Shannon entropy as the starting point. I have added this to my R package agrmt on R-Forge, and will push it through to CRAN once the documentation is up to scratch. It’s interesting how many different approaches are developed to address the same problem; clearly the different solutions have not spread wide enough to prevent doubling the effort.

Tastle, W., and M. Wierman. 2007. Consensus and dissention: A measure of ordinal dispersion. International Journal of Approximate Reasoning 45 (3): 531-545.

Leik’s Measure of Ordinal Consensus

In 1966 Robert K. Leik introduced a measure of ordinal consensus based on cumulative frequency distributions. It can be used to express agreement or polarization, just like Cees van der Eijk‘s measure of agreement “A”, and its derived measure of polarization. A difference exists in that in Leik’s measure, an equal distribution of frequencies – all categories equally common – does not always give the same value. Leik defends this, arguing that an equal distribution should only be considered the mid-point between agreement and polarization if the number of categories is very large. With a small number of categories, polarization may simply be a result of chance.

Here’s a graphical summary of how Leik’s measure of ordinal dispersal behaves with increasing numbers of categories (consensus is defined as 1 minus dispersal), as outlined in table 3 of the article.
leik1

Leik’s measure of ordinal dispersion is available in the latest version of the package agrmt (version 0.27, not yet on CRAN)

Leik, R. 1966. ‘A measure of ordinal consensus’. Pacific Sociological Review 9 (2): 85–90.