Simpson's Paradox

Working throughout my life with numbers and statistics I was aware that at times how they are analyzed can be misleading but until a few years ago I was not aware of Simpson's Paradox.

Simpson’s Paradox is not the same as Homer Simpson’s Paradox* (which may, however, may explain other issues in America). Simpson’s Paradox occurs when a correlation present in different groups is reversed when the groups are combined. That happens when the ratios between the individual groups are different in the comparisons being made.

While aware of it in the pure numerical sense, I first encountered it in a political sense in 2011.  At the time Wisconsin Governor Scott Walker was being denounced as a Nazi for making changes in the collective bargaining power of unions, specifically for teachers, that would have reduced the costs of medical insurance**.  At the time, Wisconsin's existing law required school districts to obtain health insurance through the teachers union captive insurer.

One of the common talking points in opposition was that Walker's initiative was an assault on the quality of education in Wisconsin, and frequent comparisons were made to academic performance in Texas, where K-12 achievement scores were not as high.  The rhetorical question was always, "Do you want to make Wisconsin schools like Texas?".

The assertion about test scores was correct; students in Wisconsin scored higher than Texas.

However, if you looked at the data from the National Assessment of Educational Progress (NAEP), which administers the annual standardized test given to 4th and 8th graders to measure math, science and breaks down results by black, white and Hispanic the picture looks different. (NAEP does not test after 8th grade). In every case (with the exception of Hispanics for 4th grade science), black, Hispanic and white students in Texas scored higher than Wisconsin students of the same demographic. That’s in 17 out of 18 comparisons. In other words, black students in Texas outperformed black students in Wisconsin in math, science, and reading in both 4th and 8th grades. And, in all 18 categories, Texas students scored above the national average. Moreover, the gap between White and Hispanic/Black scores was less in Texas than in Wisconsin. Wisconsin looked better overall because it had a much higher percentage of white students.  Put another way, Texas student results were actually better and more equitable than Wisconsin.

While researching this issue I came across another claim by those disputing claims of Texan prosperity - that its hourly wages were actually lower than the U.S. average. This was true at the time – the US average median hourly wage was $12.50, while Texas was $11.20.

However, what was ignored and was also true, is that if you separately compared average median hourly wages of whites, blacks and Hispanics in the United States to those of each of those groups Texas, they were higher for each group in Texas. 

In both cases the difference between the measurement of the groups individually and collective was because of the diversity of the Texas workforce and school population, which has lower percentages of whites and higher of blacks and Hispanics than Wisconsin and the U.S. as a whole, so that when combined the data led to a misleading conclusion.

Once aware of the paradox I began looking at such comparisons more deeply.  For instance, we have frequent claims that Scandinavian countries are better in material and health outcomes than the U.S.

Yet repeated studies comparing both life expectancy and wealth between the populations of those countries and of their descendants in the U.S., show those living in the U.S. do better. Here’s an example, focusing on economics:

Danish-Americans have a measured living standard about 55 percent higher than the Danes in Denmark. Swedish-Americans have a living standard 53 percent higher than the Swedes, and Finnish-Americans have a living standard 59 percent higher than those back in Finland. Only for Norway is the gap a small one, because of the extreme oil wealth of Norway, but even there the living standard of American Norwegians measures as 3 percent higher than in Norway.

The same holds true for every other comparison I’ve found between countries of origin, including in Latin America and Africa, and descendants in the U.S. The U.S. often shows up lower because of the very diversity of the American population, so that when combined you get a different result than if you look at the components.***

Once you start looking for Simpson’s Paradox, it shows up everywhere.

There are, of course, other statistical tricks used in political arguments.  Covid-19 has seen an eruption of these, from those switching back and forth between mortality and case incidence data depending on the point they want to make or failing to look at results for others in similar situations.  In the latter case, what springs to mind is the claim that Uttar Pradesh state in India successfully controlled Covid by distributing ivermection.  While it was true that cases declined 99% after the state took this action, cases declined by the same amount in the adjoining states which did not follow Uttar Pradesh's direction.

---------------------------------------------------------------------

* From the episode “Lard of the Grease:”

Homer: Okay, boy. This is where all the hard work, sacrifice and painful scaldings pay off.

Clerk: Four pounds of grease. That comes to . . . sixty-three cents.

Homer: Woo-hoo!

Bart: Dad, all that bacon cost twenty-seven dollars.

Homer: Yeah, but your mom paid for that.

Bart: But, doesn’t she get her money from you?

Homer: And I get my money from grease. What’s the problem?

**  Which, after the legislation was enacted, it did.  Some school districts took the savings and hired additional teachers. 

*** There is another aspect to be aware of, beyond the comparisons.  American rates for serious lifestyle health issues like diabetes, obesity, high blood pressure etc, are increasing across the board.  Regardless of international comparisons this is a serious problem.