In this edition of Fun With Statistics we'll take a quick look at the impact of subpopulations of data. We often look at the direction of large populations, without realizing that the perceived direction may disguise what is really going on. As an example, consider divorce. The popular belief is that divorce rates are increasing, with more than half of all marriages eventually dissolving.
But let's take a look at an article in yesterday's New York Times by Justin Wolfers, and the author's related post on Marginal Revolution today.
The great myth about divorce is that marital breakup is an increasing threat to American families, with each generation finding their marriages less stable than those of their parents. This newspaper warned readers, “Don’t stock up on silver anniversary cards” because “women and men who married in the late 1970s had a less than even chance of still being married 25 years later.”
Now of course since most Evolving Excellence readers have higher than average intelligence, they know to question anything printed in The New York Times. Justin's op-ed calls them on the errors with their latest story. And the notorious error-meister himself, Paul Krugman, isn't even at fault this time.
The story of ever-increasing divorce is a powerful narrative. It is also wrong. In fact, the divorce rate has been falling continuously over the past quarter-century, and is now at its lowest level since 1970. While marriage rates are also declining, those marriages that do occur are increasingly more stable. For instance, marriages that began in the 1990s were more likely to celebrate a 10th anniversary than those that started in the 1980s, which, in turn, were also more likely to last than marriages that began back in the 1970s.
What is going on? The specifics of subpopulations, in this case the marriages and divorces by decade. The overall marriage rate has been decreasing, so the population of married couples by decade has also decreased, which overweights those from the 1970s. The higher divorce rate during the 70s then skews the overall statistic. Similarly, the simple fact that marriages during the 70s have been around longer, and not to mention a rather glaring error in the data survey itself.
The census data come from a survey conducted in mid-2004, and at that time, it had not yet been 25 years since the wedding day of around 1 in 10 of those whose marriages they surveyed. And if your wedding was in late 1979, it was simply impossible to have celebrated a 25th anniversary when asked about your marriage in mid-2004.
Why has the great divorce myth persisted so powerfully?
Reporting on our families is a lot like reporting on the economy: statistical tales of woe provide the foundation for reform proposals. The only difference is that conservatives use these data to make the case for greater government intervention in the marriage market, while liberals use them to promote deregulation of marriage.
This kind of misinterpretation of data, which then creates popular myths, and then get transcended into poor policy decisions is unfortunately quite common. Another that I can think of right off the bat is the "poor are getting poorer" statistic, which claims that the bottom 10% of the population are becoming progressively poorer. The reality is that much of that subpopulation moved into the next higher decile, to be replaced by millions of migrant workers, legal and otherwise, earning less. I won't go near the politics of that misperception on this blog.
Bottom line, be cognizant of the size of the statistical population being referenced, and the possibility of movement within subpopulations distorting true trends. Not to mention the reporter's motivation, political or otherwise.