Which Way to Compare? Part 1 – Why Percentage Distributions are Better than Averages

Our work for our clients, especially our employee satisfaction and engagement studies, often includes comparisons to national or industry norms or across groups within their organizations. These comparisons enable clients to where they stand in and help them set reasonable goals for organizational improvement. In recent weeks we’ve had several conversations regarding the pros and cons of different ways of expressing these comparative figures – as percentage distributions, as averages and as indexes. We strongly feel that percentage distributions offer the best approach in most cases. Today we’ll show why we prefer percentage distributions over averages and in the next blog we’ll show why we also prefer percentages over indexes.

Averages offer the benefit of simplicity for the end users of data. If a survey question has a 5-point scale that is converted to the numbers 1 through 5, taking the numerical average of the responses produces a score between 1 and 5. It’s then a simple matter to compare across groups. If we put the “5” at the positive end of the scale, then those groups – workgroups, locations, divisions – with higher scores are doing better than those whose scores are lower. It’s easy to glance at a set of these average scores and identify priorities for improvement.

The problem with using averages, however, lies in the nature of the average (technically known as the arithmetic mean), as a statistic. An average is a measure of central tendency and has an underlying assumption that the answers are more-or-less normally distributed. This assumption is often incorrect. It is not uncommon to find survey responses that are skewed toward one end of the scale or even polarized. Using a central tendency measure when there is no central tendency can reduce the utility of the information or even be misleading. A simple example can show why this is true. Imagine three work groups all answering the question “How much do you like your job?” using s 5-point scale. Each group has 10 employees:

In group one, all 10 employees choose the middle of the scale

In group two, 5 employees choose one end of the scale, and 5 choose the other end

In group three, 2 employees choose each of the 5 points on the scale

The average score for all three groups is a “3.” None of these groups has a central tendency and taking an average obscures an important feature of the data – the way the opinions are distributed. If these three work groups all reported to you, which information would be most actionable – knowing that they all have the same average score or knowing something about how the scores are distributed? We think the answer is pretty obvious.

Whether in market research or national politics the difference between winning and losing is often in the percentage distribution, not the average. In the 2012 election, more votes were cast for Democratic candidates for the House of Representatives than for Republican candidates, but we have a Republican majority in the House because of the way those votes were distributed across congressional districts. Nate Silver made his reputation as a predictor of elections by understanding the details of percentage distributions of voter behavior. We feel our clients need and deserve the same level of information about the issues that are important to them. So even though average scores are easy to calculate and present, we think that looking at percentage distributions is worth it.