Evaluating ‘Transformational Development’ – Forthcoming in F&E

I’m happy to report that a paper stemming from my fieldwork in Kenya is (finally) being published in the Faith & Economics journal. Although F&E is a relatively niche journal, it is my first peer-reviewed research publication in an academic journal and reports on work I performed before I even began my MS program. The paper is titled: “Learning Toward Transformation: Evaluating Material, Social, and Spiritual Change in Western Kenya, here is the abstract:

Continue reading

Data Collection in Developing Countries… Stories from the Field

Consider the following unrelated facts:

  • About 1 in every 19 Kenyan children dies before his or her fifth birthday.
  • Only 9 percent of health facilities in Bangladesh offer diagnostic services for tuberculosis.
  • The percentage of households in Uganda owning at least one insecticide-treated anti-malarial net increased from 16 percent in 2006 to 90 percent in 2014-2015.

Our first reaction to numbers like these is to imagine what they imply for the lives of people in those countries. But there’s another dimension to the statistics that’s easy to overlook: someone had to go to the field and collect the underlying data. Behind each of these neatly summarized findings (from the U.S. Agency for International Development’s Demographic and Health Surveys (DHS) program) lie stories of personal hardship, risks, and logistical challenges faced by data gatherers—the unsung heroes of international health and poverty research.

This is from a recent article that made the rounds in ‘development Twitter’, which concludes…

Most charities and aid agencies don’t publicize these behind-the-scenes stories, mainly because they see the challenges of working in the field to be an everyday part of the job. But these aren’t everyday jobs. Next time you see health or poverty statistics, or data used to evaluate the effectiveness of charities, take a moment to reflect on the hard work and dedication that must have gone into collecting that information.

I’ll add two short stories. I hope others add more.

The first is from Kenya. I was working on an impact evaluation for an NGO. A local pastor and I were driving around Kitale Kenya and had to drive about an hour out of town to catch up with an individual in our sample. Just as we arrived at the farm the individual owned, a heavy rain began. After performing the survey the individual, refusing to let us leave in the middle of the rain, fead us tea and mandazi. After the rain stopped we returned to our car, which we found had sunken into the mud. We spent roughly the next hour digging our car out of the mud. The road (actually pictured in the header of this blog) leading back to town was long, narrow, and now sufficiently muddy. As we attempted to drive, the car inevitably slid off the side of the road and into the surrounding bushes and ditch. Eventually a fairly large group of kids came and helped guide our car on the center of the road and away from the deep and muddy puddles on each side until we reached a semi-paved road. All told, we spent an entire afternoon just recording one individual’s survey.

The second is from Myanmar. I was working with the Myanmar Development Resource Institute implementing a rural household survey in Mon State, a coastal region close to the boarder with Thailand. The survey was designed to provide statistics that were representative of Mon State. Representative statistics are so essential to modern policy-making that the effort required to collect such data is often glossed over. To collect this data our wonderful research team and enumerators traveled throughout rural Mon State – in cars, but also by boat across water and on foot along rice paddy bunds. Once at the survey location, enumerators tirelessly walked through a 2-3 hour survey covering topics such as assets, agricultural activities, and household consumption. Below are a few pictures from the Mon State Rural Livelihoods Household Survey.

IMG_3669

IMG_3335

IMG_3425 IMG_2300

 

The Economics of Climate Change and Water in Kenya

Every once in a while, someone writes something on the internet without thinking really hard. (Ok, ok, it happens all the time.) This past week, this article was written which called into question the link between climate change and water access in Kenya. I thought I’d respond to it as someone who studies economic development and environmental economics, is working on a masters degree in MSU’s Department of Agriculture, Food, and Resource Economics, and recently spent a year in Kenya.

[Full disclosure, the article calls out the Office of Social Justice of the Christian Reformed Church (of which I am am a lifelong member) and at least one current member of their team (of who is made up of several fellow alumni of Calvin College and past classmates of mine). I think I’m far enough along in my training as an objective economist that these facts will not bias my analysis.]

A preliminary note about how environmental economists differ from environmentalists, in general. Pollution is a product of many things we like (i.e. roads, transportation, houses, etc.). Therefore zero pollution is not an optimal amount of pollution. Additionally, allowing firms to pollute as much as they would while maximizing profit and disregarding externalities hurts the production of other things we like (i.e. clean air, fish, parks, ecosystems, etc.). The environmental economist is concerned that we produce the things we like at an additional cost of harming other things we like.

The article (written by Dr. Calvin Beisner, the founder and national spokesperson for The Cornwall Alliance for the Stewardship of Creation) disputes the claim that “To help the poor in Kenya (and other developing nations), we must fight global warming”.

It seems fair to be skeptical of such a broad statement. But lets walk through each of the article’s points and try to (objectively) come to a conclusion without letting our emotions bias the result. Dr. Beisner’s first point:

Kenya has not experienced a significant upward trend in average temperature, either monthly or annually.

He’s not wrong about this. In fact, according to NOAA Kenya’s land temperature has only increased less than 1% since the base period of 1981-2010.

Screen shot 2015-03-21 at 12.02.06 AM

 

But this kind-of completely misses the point. Just because the average temperature of the air that happens to be within Kenya’s boarders isn’t rising, doesn’t mean Kenya or the people of Kenya are not effected by rising global temperatures. This highlights the major obstacle with global climate change policy. The climate is a strict public good. One person’s use of it doesn’t restrict another person’s use of it, and we cannot prohibit anyone from using it. (Economists call these characteristics “non-rivalry” and “non-excludability”.) So, just because the air temperature isn’t per say rising the quickest in Kenya doesn’t mean Kenya isn’t vulnerable to climate change. Consider some quick facts.

  • Agriculture accounts for over 30% of added value of Kenya’s annual GDP growth.
  • Agriculture accounts for 18% of wage employment and 50% of revenue from exports in Kenya.
  • 75% of Kenya’s population lives in rural areas.
  • 75% of Kenyans make their living by farming.
  • About 50% of Kenya’s total agricultural output is non-marketed subsistence production.
  • 45.9% of people in Kenya live below the national poverty line.

(All stats from the World Bank)

Therefore when one begins to map the impacts of Climate Change, Kenya (and other developing countries which rely heavily on agricultural production) are extremely vulnerable. The map below shows vulnerability to a 5.5 degrees centigrade increase in global temperature, which is rather dramatic, but not outside of the trend. It is important to note as well, that this map is produced by running agricultural production models through different scenarios and forecasting output, even if you don’t believe climate change is man-made, you can’t deny our earth’s temperatures are rising, and if trends continue dramatic losses will occur, often in our world’s poorest regions. You may be inclined to bet that global temperatures are cyclical, but why bet on the fate of the most vulnerable in the world, when we can try to make the world better?

Screen shot 2015-03-21 at 12.24.34 AM

The article then quibbles at the idea that rainfall patterns have changed in Kenya over the years:

Yet the videos did cite native Kenyans saying January rainfall had diminished over their lifetimes, and February rainfall had increased, making it more difficult for them to schedule planting.

But childhood memories are notoriously poor data sources, both for the past and for comparison with the present. Hard data are indispensable.

And the hard data in the table below show that, while rainfall amounts have risen and fallen in Kenya since 1900, there is no significant trend.

In 1990–2009, Kenya’s average annual rainfall was 7.2% higher than in 1900–1930, 8.5% higher than in 1930–1960, 1.5% higher than in 1960–1990, and 5.7% higher than in 1900–1990. Thus, the United Nations Development Program concluded in its country profile for Kenya, “Observations of rainfall over Kenya since 1960 do not show statistically significant trends,” and because annual amounts vary significantly more than those periodic averages, the same can be said for the entire 110-year period.

Contrary to the perceived memories reported in the videos, there was no reduction in rainfall in January or increase in February. The very opposite was true. Average January rainfall in 1990–2009 was 18.7% higher than in 1900–1930, 25.1% higher than in 1930–1960, 11.2% higher than in 1960–1990, and 18.1% higher than in 1900–1990. And average February rainfall in 1990–2009 was 4% lower than in 1900–1930, 4.4% higher than in 1930–1960, 4% lower than in 1960–1990, and 1.4% lower than in 1900–1990.

Here, I’d just like to point out the danger of using averages to draw inference, because if Dr. Beisner had any statistics education worth its fee of admission he’d be concerned with the validity of an average drawn from a heterogeneous population of observations. Just as a patient asks a doctor if a medical treatment shown to cure an aliment on average will work for her, specific regions of Kenya may be concerned that they experience different rainfall patterns than the entire country of Kenya on average.

Now, perhaps he is correct and rainfall patterns haven’t changed very much over the years in the various regions of Kenya. Remember that the only reason rainfall is brought up as a topic is because rain is considered an input for agricultural productivity. Obviously other factors contribute to agricultural productivity, not just rain. And the models which create maps as the one above takes as many of these factors into account as possible. Maize (corn) is Kenya’s largest agricultural product by far, and maize has an ideal temperature in which it likes to grow, too hot and it will die. While the average temperature of Kenya isn’t shown to have risen very much (again) on average. Even one extra day of extreme temperatures will seriously harm maize productivity. This is only one other factor. Other factors include, soil quality, fertilizer quality, human capital effectiveness, etc.

The article continues:

Are poor Kenyans suffering from water shortages? Yes. Is that because of global warming—manmade or natural? No. Is fighting global warming the solution? No.

Despite its moderate annual rainfall totals (about 26 inches per year, similar to that of Kansas and Minnesota), Kenya is potentially a water-rich nation. It borders on Lake Victoria—the second-largest freshwater lake in the world by area and ninth-largest continental lake by volume.

Most of Kenya, including its driest part, the Great Rift Valley, is within 200 miles of Lake Victoria, a distance readily served by aqueducts.

For comparison, the Roman aqueducts, built two millennia ago, carried water 260 miles, and the system of aqueducts constituting the California State Water Project (SWP) provides drinking water for over 23 million people (roughly half the entire population of Kenya) by transporting water hundreds of miles from the Colorado River, the Sierra Nevada, and central and northern California. The shortest, the Colorado River Aqueduct, is over 240 miles long.

What to do with a statement like this. I’ll just list some the problems:

  1. Kansas and Minnesota don’t have rainy seasons like Kenya. So sure, they might receive similar inches in rainfall per year, but the application is entirely different. If Kenya’s rains are disturbed by just a little bit (by being delayed by a week or by raining less) Kenyan farmers are much more vulnerable than farmers in Kansas and Minnesota.
  2. I’d like to see a feasibility study for aqueducts providing Kenya with water from Lake Victoria. It would have to be mighty creative.
  3. It is borderline hilarious that water access in California and the Colorado River Aqueduct are bought up as a potential model for Kenya to mimic, considering that water access in California and the Colorado River Aqueduct is one of the most unsustainable (strictly speaking economically, not to mention environmentally) in the world.
  4. Lake Victoria is bordered by not only Kenya but also Uganda and Tanzania and has rivers that run into Burundi, Rwanda, South Sudan and beyond. If Kenya began to pump tons of gallons of water out of Lake Victoria (like California does to the Colorado River) the rest of these countries would be outraged, and likely begin to pump water out of Lake Victoria themselves. Here we would have a “common pool resource problem” similar to that of the climate and the world.

The article ends with this dramatic statement:

Sad to say, however, if climate change activists succeed in enacting policies to fight global warming, Kenya’s economic growth will be curtailed.

Why? Because abundant, reliable, affordable energy is an essential condition of economic growth, and activists seek to fight global warming by shunning the use of the most reliable and affordable energy sources for the developing world—coal and natural gas—and putting far more expensive “Green” energy sources like wind and solar in their place.

As it happens, Kenya has an estimated 400 million tons of coal reserves and is about to begin mining them, making the coal available to generate electricity and deliver its people from the smoke that comes from burning wood and dried dung as primary cooking and heating fuels—smoke that causes high rates of illness and premature death, especially among women and children, from respiratory diseases.

In 2013, two other evangelicals made almost identical claims about climate change in Malawi. As I demonstrated then, those claims were false and based on the same kinds of mistakes shown in these videos.

I commend the good motives and lofty goals of those who make these claims. But motives and goals aren’t enough. Accurate facts are essential to wise decisions.

Ironically, and sadly, the climate policy the makers of these videos want will only bring further harm to the very people they long to help, by prolonging their poverty—the real threat to Kenyans’ health and life.

Here lies the problem with the current climate change debate. On the one side we have climate change activists who (probably rightly) see that something needs to be done to curtail climate change, reduce pollution, and protect the environment. On the other side we have climate change skeptics who (probably rightly) see the policies advocated by environmentalists as potentially harmful to economic growth and productivity. The debate has reached a point in which nothing productive is being discussed, even though the optimal answer (in terms of the economy and the environment) is not a compromise on either side but a simple realization in the importance of what each side is selling. Environmental sustainability on one side and economic viability on the other. Good environmental policies will not curtail Kenya’s economic growth, they will make Kenya’s future stronger.

Don’t believe me? Study some economics. (Or simply listen to this Freakonomics Podcast.)

I’d love to hear from you about the topic. Please comment or leave a question below.

One Year in Kenya [Lesson 4/5]

Effective organizational marketing does not equal ‘good’ development work.

One of my work responsibilities was to write stories for the marketing departments of the two organizations I was affiliated with. Initially I was excited about this responsibility, but I soon recognized frustrations and limitations.

Since people everywhere relate best to stories, the marketing departments wanted stories from me about individual people. Conversely, my job, as a researcher, was to analyze and evaluate impact in the aggregate, for a population of individuals. As a result, any single story I wrote about an individual felt weak, anecdotal, and potentially misleading. Who’s to know whether this individual is an extraordinary outlier or an average Joe? (For more on this see The Danger of a Single Story.)

Several months into my time in Kenya, I recognized the distinction at the heart of my troubles. It is sometimes difficult to differentiate, but the difference between someone doing advocacy and someone doing science is always crucial.

Organizational marketing, particularly for non-profits, is essentially advocacy work. In order to garner support (both financial and otherwise) marketing must ‘sell’ people on the idea of the organization’s work. In order to do this, marketing must be confident and convicted that the work the organization does is, in fact, ‘good’.

This attitude flies in the face of that of a scientist. Particularly in evaluation work, the default attitude is to be constantly critical and almost superficially unsure. The strongest conviction of a good evaluator is his or her own doubt. The three most important words for a scientist are: “I don’t know”.

It is often difficult to differentiate between the two, as good advocacy actually sites and quotes science. Good science does this too, but there is a key difference. Advocacy aims at pushing an idea or concept while science simply (or not so simply) aims at pushing the truth.

This distinction comes to the fore when making sense of this article by Michael Mathethson Miller of Acton Institute latently written about the work of the organization I work for. It is a brilliant form of advocacy and marketing, but it is certainly not science. It absolutely and unequivocally should not be used to inform or direct program strategy or policy decisions. The article is designed to get people, who usually don’t think about this sort of stuff, to think about poverty alleviation (or ‘wealth creation’, whatever you want to call it. It’s the same thing) more deeply. It is wonderful advocacy, but terrible science, and confusing the two can be fatal both for the work of the organization and, sadly, for the people the organization aims to serve.

The development blog WhyDev recently wrote about the juxtaposition between effective marketing and good development. Effective marketing raises attention and donations. Good development work should improve the lives of poor people. Here are five reasons why the two are incompatible:

  1. We have short attention spans
  2. There is no incentive to translate complexity
  3. Even if it offends some, on balance, dumb simple is better
  4. Money drives the work, not the need
  5. Effective marketing draws on herd mentality

[For a remarkably well written story (yes, story!) about the danger of confusing advocacy and science – and really much more, read Nina Munk’s brilliant book: The Idealist: Jeffrey Sachs and the End of Poverty.]

HT: Brett Keller on the advocate vs. scientist distinction

One Year in Kenya [Lesson 3/5]

‘Africa time’ is hokum.

This is not a rant about all the time I’ve waited for meetings to start or for events to end. I actually think the idea that a conception of time is geographically based is rife with lazy logic.

Yes, people from some parts of the world often seem to act based on so-called cultural perceptions of time. Yes, during my time in Kenya, I’ve waited for almost every meeting to start well after the agreed upon time. Yes, I’ve attended weddings where ‘lunch’ was served at ‘dinner time’. I think the reason for this is a bit deeper and more complicated than simple geographical differences.

An example:

At the Africa Theological Seminary, where I live and usually work, everything was on time. Always. The remarkable thing is that everything is always on time and there is no bell system or clocks on the classroom walls to remind teachers to wrap up their lectures. In fact, from my reckoning, the only room with a clock in it is the chapel.

Somehow, the same people can be very late for church or a meeting in town but can be on time, with almost freakish precision, when at the Africa Theological Seminary. So what is going on?

Clearly time is not necessarily a geographical phenomenon. It has more to do with social norms and institutional traditions.

Think of it this way: It is often believed that the United States is home to people who are always ‘on time’. It would be considered strange, however, for me to show up for a party at a friend’s house at exactly the time it was advertised it to begin. If the party is to start at 6:00, we will undoubtedly show up at 6:30 or later. Again, conceptions of time do not seem to be geographical.

More correctly, conceptions of time seem most fundamentally tied to social norms and the traditions set up by the various institutions in our lives. It may seem that conceptions of time are geographically based, because history plays out in specific geographical locations. Behind every seemingly geographical cultural characteristic I can think of, there lies a more fundamental reason for the perceived difference that actually influences the local culture. It usually points back to the influence of political, economic, social, and religious institutions on our daily lives.

P.S. This lesson, perhaps even more than others, is not completely settled with me. I’m very open to discussing this further.

One Year in Kenya [Lesson 2/5]

When reading anything start with the presumption that this is almost certainly wrong.

Being a research assistant really requires you to read and think for whomever’s research you are assisting. Research requires a certain level of non-emotional attitude. Reading must be done quickly and summaries must be with pith. Starting with this presumption helps, it may not be how you read in leisure, but it will make you a more efficient and a more appreciated research assistant.

There is an important caveat to this lesson: Starting with the presumption that everything is almost certainly wrong does not always lead to a contrarian conclusion, questioning everything no matter what is cliché and bullish. True critical thinking knows what to question, when, and where and what to appreciate and objectively support.

An example of this in the negative form comes from Pikettymania: Thomas Piketty, the now-famous French economist, published his take on wealth inequality in his best-selling book Capital in the 21st Century.

The book, as it should, made a huge splash upon its translation into English. Many (I mean, almost everybody who thinks about this stuff) wrote and published their thoughts on the book (and its famous equation: r>g) all over the economics blogosphere in the first half of 2014.

Chris Giles, an economics editor for the Financial Times, wrote his ‘critique’ on the so-called mathematical and statistical errors of Piketty’s analysis. His conclusion, Piketty is wrong, inequality is not actually rising.

Several weeks later, Piketty responded to Giles’s comments clearly (I mean with head spinning statistics) showing that his so-called mistakes were actually regular data maintenance methods accepted by anyone performing analysis on decades long time-series datasets.

While a careful reading of Piketty’s book starts with the presumption that it is almost certainly wrong, quality criticism should not follow the path Giles took: Starting with a pre-set conclusion and combing for a story to support that conclusion. Due to the weight of the implications of Piketty’s book much rigorous criticism must be deployed, but this must be done objectively and credit must be given where credit is due.

One Year in Kenya [Lesson 1/5]

Over the next couple weeks I will be posting on several of the most salient lessons I’ve learned while living and working in Kenya over the past year. These lessons will be skewed toward what I am currently able to articulate and write about. Certainly many of the most profound lessons will be realized in later months and years.

The more I learn about the world, the more I find myself saying, “I don’t know”.

This lesson is, perhaps, half due to reading a book (Aid at the Edge of Chaos) and half from experiences. booknew21

The book should be an important book for anyone interested in a career focusing on international issues. It’s influence, however, may not be as wide as it deserves. The book, particularly the middle section, is kind of a slog. Ben Ramalingam introduces the rich but young history of complexity science, the study of complex systems such as ecosystems, traffic patterns, deep-sea oilrigs, epidemiology, etc. Ben’s point is that not only are political and economic systems categorically complex, so are the aid and development systems of providing information, aid, assistance, knowledge, and resources. His conclusions are important, I recommend the book, but only if you are seriously pursuing an ID career.

Saying the world is complex is almost cliché now a days. This lesson, however, bypasses the cliché, as a complex world is now a ‘jumping off’ point for me rather than a landing area. One story encapsulates this well:

One of the components of the program I was a part of was providing loans to groups of borrowers. We had $25,000 to loan out during a one-year period. Due to the experimental nature of our project we chose to give out loans on a term of 6 months. This would allow us to make adjustments halfway through our time with the money.

The first round was complicated by the nightmare that is the global money transfer system. Due to several mistakes along the way by accountants entering faulty routing and account numbers, it took about two weeks for the money (when I say money, I mean numbers recorded in a computer) to travel from the United States to our bank account here in Kenya. After we had the money we had to transfer chunks of it to several groups of borrowers who have their own bank accounts. Again, accountants made mistakes entering account numbers and two of the transfers were reversed. One time the money was “lost” in cyberspace with record of one bank initiating the transfer but no record of the other bank receiving it. All this eventually was ironed out, but delayed the lending by another two weeks.

The second transfer has just recently occurred. In the weeks leading up to the loan application deadline, I found myself sitting with several of the leaders of the savings groups. They explained to me why taking a 6 month loan starting in July is a less than optimal option for them.

For those who would use the money to invest in farming, the seeds are already planted and harvest will not begin until November. Any investment in farming at this point is not financial.

Due to this, and because Kitale is primarily agricultural, the majority of the other businesses from a medical laboratory to a women’s clothing boutique are currently in a season characterized by low sales numbers. Not generally an optimal time for investment for individuals who often provide for their family and run their business out of the same pile of money.

Furthermore, there is an added dynamic of group lending. We lend to several groups, which are officially registered and have a set list of individual members. These groups then provide financial services to their members. The groups all have different methods for loaning money, but generally individuals will not take a loan until they are ready to use it. One group leader told me he’s worried that if his group takes a 6 month loan now, the money will simply sit in a bank account for 5 months until the harvest season begins and then the group will be required to repay the loan (with interest) before the majority of their most productive season has completed.

There are so many surface level conversations about loans being inherently ‘good’ or ‘bad’. The fact is, both of these opinions are right and both are wrong. More money does not necessarily mean more money. In what season was the money given or loaned? What are the terms? Is it one time transfers or multiple smaller transfers spread out over a longer period of time? Group lending is often considered less risky than individual lending, but what are the dynamics of the group? How did the group itself originally form? More generally, what are the other options for credit available in the area? How does the ‘market interest rate’ compare? What are the terms and conditions of the ‘competition’?

This is why, the more I learn about access to capital, microfinance, and financial products for the poor, the more I find myself saying, “I don’t know”.

One of my favorite movie quotes points to this idea of ‘the more you know, the less you know’.

“Conviction, as it turns out, is a luxury for those on the sidelines…”
A Beautiful Mind (2001)

I do not consider myself to be Hayekian, nor do I generally sympathize with Austrian Economics, this quote by F.A. Hayek, however, is really quite good:

“The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design.”
– Friedrich von Hayek

“Even Nothing Is Something”

Periodically an adjunct professor from the United States comes to teach a class here at the Africa Theological Seminary. Besides teaching, they really don’t have a whole lot to do, so (with some of these people) I end up spending evenings talking with them about stuff, in general. A couple weeks ago a new professor arrived on campus. I introduced myself after our weekly chapel service and when he learned that I had been in Kenya for almost a year, he said he had some questions for me.

Later that evening we sat outside and he told me about this village he “has a heart for”. Every time he teaches at ATS in Kitale he travels about an hour “into the bush” to visit this village. No water. No electricity. The school has dirt floors. He really feels like he needs to do something. He asked my opinions on several well known aid and development programs and agencies. World Vision (excellent theory of change, but no evidence proving it’s efficacy). Compassion International (weak theory of change, but rigorous evidence proving it’s efficacy). Hope International (fine work, but less than promising in a community with no water, electricity, and – likely – weak access to markets).

Like so many people who experience the reality of life in a village like the one he visits, he feels an inextricable (perhaps even spiritual) feeing to to do something. I went to bed that night thinking about everything I’ve learned through my studies in college and through my experiences in Panama, Ghana, and Kenya. I wrestled with this feeling to do something, felt by so many (including myself). As slumber set in, I lazily concluded: sometimes something is nothing.

Continue reading