Mediation Analysis and the ‘Sequential Unconfoundedness’ Assumption

Students with the Centre for the Study of African Economies (CSAE) at the University of Oxford are creating a wonderful public good. The Coders’ Corner is a collection of tips and tricks for implementing useful statistical techniques in common statistical software (e.g., mostly Stata). This product represents a tremendous service to the broader research community. Almost anyone reading this blog should check out previous posts.

Continue reading

How to Use the Front-Door Criterion — New Working Paper

If you follow Marc Bellemare’s blog or specifically his ‘Metrics Monday series, you will already be aware of our new working paper. The paper is titled: “The Paper of How: Estimating Treatment Effects Using the Front-Door Criterion.” The number of people who are reading this post and who do not already read Marc’s blog is probably very small. So, with that in mind, I will offer a few additional thoughts based on the preliminary work writing this paper.

Continue reading

How to Assess “Economic Significance”

Back in 2015, I read a book by Morten Jerven, in which the author makes the point that over 145 variables have been found to be statistically significant explanatory variables for long-run economic growth. Morten’s point is more nuanced than this, but this suggests that when interpreting regression results we need to not only consider statistical significance, but also economic significance.

Continue reading

Lotteries and Life Satisfaction – A Comment on the Cardinal Treatment of Ordinal Variables

A long standing belief, held by many, is that winning the lottery actually makes people miserable. This belief is backed up by existing research in psychology finding that lottery winners were no more satisfied with their life than people who did not win the lottery. New research suggests this belief might be wrong.

Continue reading

How Much Does the Cardinal Treatment of Ordinal Variables Matter? – New Working Paper

Concepts such as subjective well-being, satisfaction, happiness, trust, measures of quality, and even standardized test scores are all measured using an ordinal variable. This means that we know the rank of the response categories (e.g. a respondent reporting being “very satisfied” indicates they are more satisfied than if they had reported being “satisfied”), but we do not know the interval between response categories (e.g. we don’t know how much more satisfied “very satisfied” is compared to “satisfied”). This is contrasted with cardinal variables, such as earnings, where we know $10 is more than $5 and represents twice as much money.

Continue reading

The Cardinal Treatment of Ordinal Variables

Last weekend, I came across a paper published in the European Economic Review by Carsten Schroder and Shlomo Yitzhaki entitled “Revising the Evidence for Cardinal Treatment of Ordinal Variables” (2017). I found the paper to be well-written, intuitive, and important. Although there are no direct policy implications of this paper, papers like these need to be publicized, for the sake of all research that does aim to provide meaningful implications for the world.

Continue reading

The Economics of “It’s a Wonderful Life”

That’s an okay explanation, but I think there is an even more profound lesson about economics woven into this classic movie. It’s a topic I interact with regularly and one that is often misunderstood.

In the movie, “It’s a Wonderful Life” George Bailey gets the outrageous opportunity to see what the world would be like if he had never been born. He’s able to see the impact he’s made on the people around him. What a gift!

Notice how I use the word “impact” in the last paragraph. It is used correctly under the definition that impact is the outcomes of something minus the outcomes if that something did not happen.

Economists try to measure the impact of all sorts of things: from government policies and social programs to business innovations and market forces. When undertaking this endeavor we are presented with the same problem Clarence, George’s guardian angel, faces. We need to show impact. In Clarence’s case he needed to prove to George that he had a meaningful and positive impact on the people around him and that the world was a better place because he was born. Economists generally need to measure the effectiveness (or non-effectiveness) of a specific policy, program, or innovation.

The problem economists have is we aren’t angels and can’t go back in time and re-do the world in the absence of the thing we want to evaluate. Economists encounter what is called the “problem of the counterfactual”. We don’t have an alternate universe in which to compare hypothetical outcomes.

We have to come up with work-arounds – creating proxies of the counterfactual – to solve this problem.

One way is to randomly assign receipt from the program, policy, or innovation. With a large enough sample we would expect that the group of people assigned to receive the program, policy, or innovation would be exactly equal to the group of people who did not receive the program, policy, or innovation. Impact would then be measured by subtracting the outcomes from the group that didn’t receive the program, policy, or innovation from the group that had. This is the randomized experimental method and if it is done correctly it allows economists to show the exact same thing that Clarence showed George. Pause and reflect on that for a moment.

0010_2A_10_Futo_Peter_Evaluation_and_Impact_Assessment

 

Sometimes (most of the time) a randomized experiment is not feasible or ethical, in which case the experimental ideal (described above) needs to be approximated. There are many (in fact, seemingly infinite) ways to do this, each has it’s own strength and is best suited for a particular situation. Going into the specifics of these is beyond the scope of this blog. (Much more can be learned by reading the excellent Mastering Metrics: The Path From Cause to Effect)

Suffice to say, “It’s a Wonderful Life” has a lot to teach us about economics, particularly the problem with the counterfactual and the power of the randomized experiment.