Lotteries and Life Satisfaction – A Comment on the Cardinal Treatment of Ordinal Variables

A long standing belief, held by many, is that winning the lottery actually makes people miserable. This belief is backed up by existing research in psychology finding that lottery winners were no more satisfied with their life than people who did not win the lottery. New research suggests this belief might be wrong.

Here is Justin Wolfers in the New York Times discussing a new working paper written by Erik Lindqvist, Robert Ostling, and David Cesarini:

New research suggests that more money really does lead to a more satisfying life. Surveys of thousands of Swedish lottery winners have provided persuasive evidence of this truth.

Lottery winners said they were substantially more satisfied with their lives than lottery losers. And those who won prizes worth hundreds of thousands of dollars reported being more satisfied than winners of mere tens of thousands.

These effects are remarkably durable. They were still evident up to two decades after a big win.

This new study (for those interested, here is a link to the working paper) contributes to the literature on the relationship between wealth and well-being in two key ways. First, it disentangles correlation from causation. Previous research, despite some debate, has found a rather robust positive correlation between income and life satisfaction (or other measures, such as subjective well-being or happiness). This new study benefits from the fact that winning the lottery is a random occurrence and is therefore unrelated to confounding factors that limit the previous research.

Second, the new study has the benefit of a much larger sample size. Previous research, in particular the best research in psychology, is plagued by small sample sizes. This causes problems relating to statistical power. Since most of these previous studies fail to reject the null hypothesis that money increases well-being, it is difficult to say if these studies simply do not have enough power to detect the true relationship or if the true relationship is in fact insignificant. This new study follows almost 5,000 individuals from Sweden for several decades after they won the lottery.

These contributions are important in the literature. I agree with Justin Wolfers that these results provide the strongest evidence yet in support of the standard economic view that money increases well-being. With that said, there are limitations to this paper. To be fair, every empirical paper has limitations, but I feel these limitations can be addressed.

Here are the primary outcome variables, as presented by the authors in Section 2.4 of the working paper.

  • Happiness: Measured with an 11 point scale with 0 being “extremely unhappy” and 10 being “extremely happy”.
  • Overall Life Satisfaction: Measured with an 11 point scale with 0 being “extremely dissatisfied” and 10 being “extremely satisfied”.
  • Mental Health: Measured with a 12-item variation on the General Health Questionnaire (GHQ-12). Each item uses a 4 point scale indicating how often the respondent experienced a positive or negative emotion. These responses are converted into integers ranging from 1 through 4 by the authors.
  • Financial Life Satisfaction: Measured with a 6 point scale with 1 being “very dissatisfied” and 6 being “very satisfied”.

The authors then use ordinary least squares regressions with each of these primary outcome variables on the left-hand side.

Do you see the problem?

If you ever took an econometrics class, you likely learned pretty early on that ordinary least squares regression finds the weighted average of lines passing through any two points in a given dataset. This being the case, what is the average of an ordinal scale? The answer depends on the numbers assigned to the points on the scale and any set of numerical values is valid as long as the order of the scale is preserved.

This insight is made clear and defended in two recent papers. The first by Schroeder and Yitzhaki (2017) in the European Economic Review and the second by Bond and Lang (forthcoming) in the Journal of Political Economy. Although these papers make distinct contributions, the core point is clear: The valid cardinal treatment of ordinal variables must be robust to monotonic increasing transformations of the ordinal scale. Said differently, empirical results must be robust to any set of numerical values that preserve the order of the points on the ordinal scale. To be fair, Lindqvist, Ostling, and Cesarini show that their results are robust to the use of a conditional logit estimator. This is a nice robustness test of results across estimation specifications, but does little to test robustness to monotonic increasing transformations of the ordinal scale.

These insights can seem quite discouraging for research empirically examining any ordinal variable. Indeed, the article by Bond and Lang (forthcoming) is entitled, “The Sad Truth about Happiness Scales”. Although I take the insights of Schroeder and Yitzhaki (2017) and Bond and Lang (forthcoming) seriously, I feel that the need to empirically analyze ordinal variables is not going to go away anytime soon. This motivates my working paper on this topic.

In this working paper, I make three empirical points:

  1. Failing to pass existing theoretical tests for the cardinal treatment of ordinal variables may not be a serious problem in practice because it may be the case that only extreme monotonic increasing transformations substantially change empirical results.
  2. Not failing existing theoretical tests does not necessarily imply that empirical results are robust because the size of estimated coefficients could change dramatically for reasonable monotonic increasing transformations.
  3. Not failing existing theoretical tests does not imply that the statistical significance of results are robust to reasonable monotonic increasing transformations of the ordinal scale, even if the size of coefficient estimates are robust.

So, the valid cardinal treatment of ordinal variables requires careful justification. My hope is that the methods I propose can help researchers validate their methodological choices when faced with a situation with an ordinal dependent variable. Just as this new study is an example of science “correcting” itself, the careful use of ordinal variables in empirical analysis can be a way that social science further “corrects” itself in the future.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s