In 1934, two researchers – Morris Cohen and Ernst Nagel – tabulated death rates due to pulmonary tuberculosis in two cities – New York and Richmond. They found higher death rates in Richmond City than in New York City; 226 per 100,000 population versus 187 per 100,000 population.
However, they disaggregated the rates by ethnic groups; Caucasians versus African Americans. Interestingly, their previous finding was reversed. It was the complete opposite: for each ethnic group, the death rates became higher in New York than in Richmond (Look at the table below).
Death rates due to pulmonary tuberculosis
This conundrum is known as “Simpson’s paradox”.
How did this happen?
When we disaggregate data into two different sub-groups, the real situation appears; in extreme instances, it becomes reverse.
Those situations are identified as “Simpson’s paradox” because Edward Simpson explained the phenomenon using hypothetical data as far back as 1951. However, prior to him, Yule demonstrated the bias again using hypothetical data set much earlier: in 1903.
According to statisticians, Simpson’s paradox, by definition, is not a true paradox; rather, it is a statistical illusion and could also be called aggregate bias. It is also a manifestation of confounding effects.
Its practical implications could well be devastating, particularly when we make decisions based on aggregate data.
In the above example, if the decision-makers were not aware of this, they would have allocated resources erroneously to Richmond instead of New York City to reduce the death rate due to pulmonary tuberculosis.
This bias seems to have been occurring much commoner than earlier thought.
Here are few more examples;
Hospital admissions of men with psychiatric illnesses over the years; gone up or down?
I created the following table using data that appeared in a short paper in the British Medical Journal.
According to the first table, the admission rates of men with psychiatric illnesses out of all admissions with such illnesses have declined slightly from 1970 to 1975.
|Admission rate||46.4% (343/739)||46.2% (238/515)|
Now, look at the following disaggregated data by age. The pattern reversed; the male admission rates have gone up.
|Those aged <=65||59.4% (255/429)||60.5% (156/258)|
|Those >65||28.4% (88/310)||31.9% (82/257)|
|Overall||46.4% (343/739)||46.2% (238/515)|
Another example from a hospital setting
The data that appears below is from a paper published based on a study about the use of prophylactic antibiotics in eight hospitals in the Netherlands. According to the first table, it seems better prophylactic use of antibiotics because the urinary tract infection rate is lower when using it rather than not using it.
|||Prophylactic antibiotics||No prophylactic antibiotics|
|Urinary tract infection rate (UTI)||3.3% (42/1279)||4.6% (104/2240)|
Since the researchers were skeptical about the finding, they dis-aggregated data by grouping hospitals based on UTI infection rates; low-incident and high-incident hospitals using 2.5% as the artificial cut-off rate. Now, the first observation was reversed; the rates were higher when prophylactic antibiotics were used.
|UTI rates||Prophylactic antibiotics||No prophylactic antibiotics|
|Low incident (<=2.5%) hospitals||1.8% (20/1113)||0.7% (5/720)|
|High-incidence (>2.5%) hospitals||13.2% (22/166)||6.5% (99/1520)|
|Overall UTI rate||3.3% (42/1279)||4.6% (104/2240)|
The above study appeared on the Royal Statistical Society website when it discusses Simpson’s paradox.