A Torpedo Aimed Straight at H.M.S. Randomista

It can’t be that often that the American Journal of Agricultural Economics publishes a genuinely spectacular result, but their most recent issue is the exception that proves the rule.

Under the unassuming title of “Behavioural Responses and the Impact of New Agricultural Technologies: Evidence from a Double-Blind Field Experiment in Tanzania”, Erwin Bulte, Gonne BeekmanSalvatore Di FalcoJoseph Hellaand Pan Lei have delivered probably the most serious challenge yet on the theology of the Church of the Randomized Controlled Trial.

For years, people have known that RCTs in Development Economics have a serious external validity problem: just because poor farmers in Kenya react in a certain way to, say, a given incentive to use fertilizer, that’s no reason to suppose that farmers in Guatemala, or Nepal or anywhere else will.  As a result rigorous evidence isn’t, to quote Pritchett’s enviable knack for a snappy headline.

So long as that was the standard rap against RCTs, the randomistas just about got away with it via their claim to superior internal validity: they can say things about cause and effect that are true in a way other economists could only dream of.

And yet, there was always a discordant note in the RCT literature. The term itself, as well as its epistemic claims, are straightforwardly borrowed from medical research. But in medical research it’s not enough for a trial to be randomized and controlled. It generally has to be double-blind, as well, to aspire to “gold standard” status.

Of course, in most development settings, there’s no viable way to make trials double-blind: presumably, if you only pretend to give that Kenyan farmer a way to save for fertilizer, but you don’t really, the guy’s going to notice.

So the “double-blind” bit of the medical analogy is quietly dropped and swept under the epistemic rug, in hopes nobody will notice…or at least it was, until Bulte and his collaborators realized seeds have some interesting things in common with pills, and so if you can double-blind a medicine trial, you can probably double-blind a seed trial, too.

So they deviously ran an open RCT comparing traditional and improved cowpea seeds alongside a double-blind RCT testing the same thing. Their results are deeply troubling for Banerjee-and-Dufloites.

In the open RCT, Tanzanian cowpea farmers who knew they getting improved seed easily outperformed farmers who knew they were getting traditional seed. But in the double-blind study, farmers who weren’t told whether the seed they got was improved or not performed just as well whether that seed they got was improved or traditional.

In fact, farmers who used traditional seed without knowing it did just as well as farmers who used improved seed, whether they knew it or not. Only farmers who knew the seed they were given wasn’t improved lagged behind in productivity.

This gap between the results of the open and the double-blind RCTs raises deeply troubling questions for the whole field. If, as Bulte et al. surmise, virtually the entire performance boost arises from knowing you’re participating in a trial, believing you may be using a better input, and working harder as a result, then all kinds of RCT results we’ve taken as valid come to look very shaky indeed.

Of course, this is just one study, and so until it’s replicated we’d probably do best not to get too too excited. The pre-publication version of the paper came under some heavy fire, which was perhaps to be expected.

It does strike me as problematic that, while the study was done on cowpea seeds, cowpeas were a secondary crop for all farmers involved, and the productivity gap between improved and traditional cowpea seeds isn’t comparable to the gap in maize or wheat. Replication is badly needed here, and preferably in a study dealing with a crop more central to farmers’ livelihood strategies.

Still, the study is an instant landmark: a gauntlet thrown down in front of the large and growing RCT-Industrial Complex. At the very least, it casts serious doubt on the automatic presumption of internal validity that has long attached to open RCTs. And without that presumption, what’s left, really?


It’s one of the biggest things keeping Ugandan farmers poor. And nobody talks about it.

My piece on seed counterfeiting is out in The Guardian today. Have a look,

Of the many factors that keep small-scale Ugandan farmers poor, seed counterfeiting may be the least understood. Passing under the radar of the international development sector, a whole illegal industry has developed in Uganda, cheating farmers by selling them seeds that promise high yields but fail to germinate at all – with results that can be disastrous.

Counterfeiting gangs have learned to dye regular maize with the characteristic pinkish orange colour of industrially processed maize seed, duping farmers into paying good money for seed that just won’t grow. The result is a crisis of confidence in commercially available high-yield seed.


59 Reforms Later

Some books just fall into your hands at just the right time, and for me Matt Andrews’s The Limits of Institutional Reform in Development was just such a book. If you’re wondering why international institutions are so damn bad at helping developing countries govern themselves better, it really belongs on your reading list.

But actually, the facts reported are as much of an eye-opener as the argument. The sheer number of internationally-backed institutional reforms your average developing country has gone through startled me. Andrews cites the case of Honduras, which has undertaken fifty-nine separate World Bank backed reforms since 1988. Fifty-nine!

They cover just about everything. Banking, trade, privatization, civil service reform, public financial management, civil service payments, procurement, competition, external auditing, procurement monitoring, results-based management, anti-corruption, participatory budgeting, merit-based hiring and performance based compensation. Several of these, like civil service reforms, are “hardy perennials” that seem to crop up again decade after decade.

The bloat is multidimensional. And notice: that’s just the reforms backed by the World Bank, before we even start talking about the agendas pushed by IMF, IADB, WTO, USAID,  DFID, etc. etc.

It goes without saying that, 59 reforms later, Honduras is just as badly governed as before.

In Andrews’s telling, this volume of partner-driven reform activity is by no means unique to Honduras. The Bank requires all sorts of developing countries to jump through various reform hoops before it’ll approve a loan, so all kinds of countries makes a show of approving the desk-based portion of reforms as a signalling mechanism, with no real commitment or intention to really follow through.

That part sounds about right, though it does rather beg the question of why the Bank keeps falling for it again and again. Then again, bank staffers are compensated for the number of reforms “satisfactorily adopted”, not by the number that actually work, so maybe it’s not that big a mystery.

But to me the bigger question is normative. Readers know I don’t usually have much truck with handwringing about neoliberal imposition. But that probably betrays my biases as a guy from an upper-middle-income country with lots of oil that’s seldom had to jump through hoops to qualify for a desperately needed loan. But c’mon, Honduras is averaging well over two semi-imposed, voluntary-in-name-only reforms per year!

Can this really be right?

At the very least it looks like a colossal waste of resources. Who can put a cost on the wasted bureaucratic energy spent on approving reforms nobody ever seriously intended to implement?

But at worse, it looks like a hijacking of basic governance processes. At that pace of imposition, what can “democracy” really mean for a country like Honduras? And what chance for Honduras to strike out creatively and find its own ad hoc solutions for its own unique problems when all its institutional energy has to be devoted to dreaming up fake paper reforms designed to do nothing beyond get that next tranche of loan released?

The opportunity cost to outsider driven reform is enormous. Because whatever it is you’re doing when you’re chasing that next loan tranche, what’s clear what you’re not doing: Problem-Driven Iterative Adaptation.



Factories are Pro-Poor: A Meditation on Huajian in Ethiopia

A guest post by Frances Pontemayor

Can there be something less glamorous than a factory? Industrialization is gritty, sooty, sweaty – it just doesn’t align with the “tragically beautiful” poverty that development practitioners in Europe and North America dream of fighting with organic farms and $50 laptops. But at what point do implicit (never explicit) aesthetic preferneces start to blinker us from what really works?

Take the Huajian Shoe Company. It’s one of China’s leading shoemakers. I’m sure you’ve never heard of it, but if you’ve ever bought a pair of Naturalizers, Clarks, Guess or Tommy Hilfiger shoes, you’ve probably worn shoes made at a Huajian factory.

In January 2012, Huajian became one of the first Chinese manufacturing companies to launch large-scale operations overseas, opening a big-time shoe factory just outside Addis Abeba, in Ethiopia. The company produces 2,000 pairs of shoes everyday and employs over 1,750 workers at its factory.

With the media attention on the conditions of Chinese factories and all that horrible pollution, the knee-jerk reaction to Chinese factories in Africa is just more fuel to Western NGO’s white-knight complex.

“Race to the bottom!” “End modern day slavery!” said every development mission statement ever. How could we let these Chinese neocolonialists ruin all the hard work they’ve trying to accomplish for the last six decades?

To be clear, helping Ethiopia develop is not a goal of Huajian, anymore than it’s the benevolence of the butcher, the brewer, or the baker that we expect our dinner from. Business is business; development is the byproduct.

Given the rising costs of labor and appreciation of the RMB, the company chose Ethiopia because they needed to shave some costs.  The country has a fairly well developed local scene for leather inputs suppliers, and labor costs are much lower. When it comes down to it, they just wanted to make some shoes that Westerners will want to buy, even cheaper than they could make them in China. Does that offend your sensibilities? Look down; what shoes are you wearing again?

Huajian’s move to Ethiopia comes with a cherry on top: thanks to AGOA, exports from Ethiopia to the US and UK markets come tax-free.

Huajian Shoe Company on its own won’t make Ethiopia into Denmark. Of course. But it will help make it a more dynamic and productive economy.

Ethiopians working at a Huajian factory are vastly more productive than their counterparts in non-export oriented manufacturing companies. That may sound like a recondite fact, but we know that in the long-run average wages track average productivity, and there’s just no way to raise average productivity that doesn’t involve shifting people from low-productivity jobs to high-productivity jobs.

In the case of Huajian, the company has even gone above and beyond in sending a number of workers to China for training, giving these workers even more specialized skills.

But even in the short term, there are definite benefits: Huajian workers become integrated into the formal economy. They get dorm-style housing and food provided by the company. The management model has been described as “military-style” and yes, hours are long and the work is boring. Still, it’s easy to caricature management’s approach, and that’s not necessarily fair either: in China, Huajian’s boss made a point of giving out his cell phone to all 30,000 company workers so they could reach him directly if that was necessary.

More importantly, they get paid regularly, predictably, which is already a huge improvement on the uncertain, unpredictable earnings of informal sector workers.

Formal workers spend have cash to spend on their children’s tuition fees or books, healthcare, things for their house (possibly a cook stove that everyone keeps talking about), or at a restaurant to celebrate. And those shopkeepers and restaurant owners will then go out and spend their money. Money, even if in small increments, will flow throughout the economy, generating the kind of multiplier effects on community well being that so many pro-poor development projects talk about and so few actually achieve.

More people in the formal economy means more a bigger tax base for the Ethiopian treasury. Tax revenue feeds into infrastructure, into education, into social welfare. And the hard currency earned by Huajian’s export orientation finances imports of foreign manufacturing technologies to upgrade leather-making equipment, or more efficient irrigation systems for their farms.

The point is the Ethiopian government can use this money, their hard earned, pulled-up-by-the-bootstraps money, on whatever they feel would continue the cycle of wealth and to overcome aid dependence. Whether you’re a teenager or an African governments – no one wants to be managed by someone else.

Somewhere down the line, Huajian’s local managers will think to themselves: “Hey, I think I could run my own factory with all this know-how I’ve gotten from Huajian, and possibly make the production even more efficient!” This new class of entrepreneurial managers, with their local networks and priceless knowledge built over the years of running a factory will eventually lead to more factories being built. This kind of spill-over effect that takes hold as knowledge embodied in workers diffuses throughout an economy is anything but glamorous. But no country has ever really developed without it.

With more factories comes more revenue, and not just for the managers (now owners), but for their newly hired workers, who are also moving from low-productivity to high-productivity occupations and putting upward pressure on wages. The whole cycle is renewed anew, and before you know it, Ethiopia is developing the kind of export oriented industrial cluster that’s marked more or less every fast development experience of the last three centuries.

Somehow, though, when we think of development work we think of the kinds of “pro-poor” micro-interventions that make life more bearable for poor people even as they remain within low-productivity sectors. That’s not development. That’s managing life in poverty. Development is boring. And more often than not, it starts in a factory.



“Pull away the infrastructure and sophisticated just becomes complicated”

A great piece on why Medical Equipment ends up sitting unused in so much of Africa, over at WhyDev:

When an anaesthesia machine designed for use in an American hospital is used in a poorly-supported hospital in Uganda, things often go wrong – even when it’s a brand-new machine (and, as we all know, often it isn’t). The machine isn’t designed to work in low-resource hospitals, which violates the cardinal rule of medical device design: Know Thy Hospital.


Balloons for Baluchistan

It’s Silly Friday, a time to look at those development projects that transcend the ridiculous to reach a kind of clueless, navel-gazing glory.

This NYTimes article from May last year is a kind of classic of Development Bloat: what happens when the political imperative to exercise American soft power crashes head first into Afghanistan’s very limited capacity to absorb it.

The litany of development gimmicks here is grimly hilarious. For my money, the plan to “create a stream of shared instances of unexpected happiness” by handing out pink balloons to strangers on the streets of Kabul is unmatched in the annals of Development Bloat.

Continue reading


You Might be Through With the Gender Gap, but the Gender Gap Isn’t Through With You

Researchers have long known there’s a gender gap in African agriculture: women-run farms are less productive than similar farms headed by men. The reasons for this disparity have been only hazily understood, though. To the extent that researchers had looked into it, they’d tended to focus on the level of farm inputs: farms headed by men seem to have access to more pesticides, more fertilizer, more and better land, more and more productive labour. So long as that’s the case, it’s no real surprise why woman farmers produce less.

It’s not just about the level of inputs used. Even after adjusting for the level of inputs used, women get less bang for the input buck.

But new research out today by the ONE Campaign jointly with the World Bank paints a more nuanced and, in some ways, much bleaker picture. It’s not just that farms led by men use more inputs, it’s that they use those inputs better.

Using newly available household level data from the Living Standards Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA - all the data is freely available on the World Bank website, btw, so go and play), a team of researchers led by the bank’s Mike O’Sullivan and ONE Campaign’s Arathi Rao set out to decompose the sources of the agricultural gender gap in six African countries. Again and again, they find statistical evidence that it’s not just about the level of inputs used. Even after adjusting for the level of inputs used, women get less bang for the input buck.

What’s interesting around the research is that this isn’t just true of physical inputs like fertilizers and seed. It’s especially true when it comes to labour. Female-headed households are composed of fewer people than male-headed households, so there’s less labour around the house to begin with. They find that in Northern Nigeria and Uganda female-headed households also hire fewer farm workers than male-headed households, contributing to the gender gap. Elsewhere, like in Niger and Tanzania when women do hire farm hands those labourers are often less productive than when men are doing the hiring, which again boosts the farm gender gap. Similarly, extension services lead to bigger boosts in productivity when they’re made available to men, contributing to the gender gap in Ethiopia and Uganda.

Obviously, a multivariate regression can’t explain why these effects are there. Maybe cultural norms make it harder for women to supervise hired labourers than for men to do so. Maybe women who run farms spend so much time looking after children their agricultural productivity suffers. Maybe there’s some implicit gender bias in the way extension services are provided that makes them more useful to men than they are to women. There are major knowledge gaps at play with these issues, it’s not really possible to say at this point.

What the research makes clear is that a simple strategy of making sure women farmers have access to the same level of inputs as their male counterparts won’t erase the significant gender gap in farm productivity documented all over Africa. Some creative policy-making will be needed. Maybe women farmers need vouchers to hire extra seasonal workers. Maybe village day-care centers can free up their time to work more effectively on the farm. Maybe extension services need to be rethought to address the specific problems women farmers face. Maybe women need special encouragement to plant higher value crops.

A whole lot of experimentation is going to be needed to suss out which of these interventions can really make a difference and which can’t. And this, if nothing else, looks to me like the future of this line of research: RCTs. Lots of RCTs.



Conservation Agriculture as Bloat?

It’s now been decades since no-till farming became the first line of defense against top-soil erotion and land fertility degradation, especially in the Western Hemisphere.

The plough is the past: in one of those under-the-radar revolutions that’s somehow failed to filter through to urban people’s perceptions, farmers from Canada to Argentina have discarded 10,000 years of agricultural common sense. It turns out deep ploughing leaves your best soil exposed to the elements, where a hard rain can just wash it away, slowly degrading your land’s ability to sustain yields. Do this long enough and you can literally turn good farmland into desert. It’s just not a smart way to farm.

Hot on the heels of No-Till’s success in North and South America, Conservation Agriculture (CA) has quickly grown into development orthodoxy. The idea is simple but revolutionary: don’t plough, keep crop residues in place as mulch and, in general, organize your farm around the overarching need to prevent top soil erosion.

From FAO to any number of USAID contractors, Development Agencies have taken on the role of Conservation Agriculture evangelists, spreading the good news to the unenlightened peoples of Africa and South Asia. Work less, improve your soil, get better yields: who wouldn’t want to farm that way?

The thing is, when first world methods meet African realities, trouble is never far behind. Issues that just don’t show up in a large mechanized farm in Kansas or Rio Grande do Sul turn out to make a big difference in Malawi and Senegal.

The best compilation on the woes of CA in Africa is in a paper tellingly titled Conservation agriculture and smallholder farming in Africa: The heretics’ view, Ken E. Giller, Ernst Witter, Marc Corbeels and Pablo Tittonell.

So what are the big problems CA faces in Africa?

  1. One man’s residue is another man’s animal feed. Key to the CA dogma is the idea of using the residue from last year’s harvest as mulch: just leaving it where it falls to provide ground-cover that shields the soil from erosion by rain. This works great in big monoculture farms, where residue is just residue. But in Sub-Saharan Africa you typically find mixed farming systems, with people raising crops alongside animals. Those animals need to eat. In such settings, crop residue is already spoken for: that’s what you feed your cows/goats. In other settings, they’re burnt as fuel. Either way, leftover biomass isn’t perceived as “residue” at all: it’s a key farm asset. If that’s how you farm, just leaving the crop residue to gently rot in the field looks perverse, if not insane.
  2. The Weeds. Even Conservation Agriculture zealots acknowledge that you tend to get more weed pressure with CA methods than with conventional farming. In big mechanized farms, that’s no sweat: you just spray the bejeezus out of the field with herbicides. But in a smallholder setting in SSA, you gotta pull those weeds up by hand. Whether the labour you save by not having to till is more or less than the extra weeding load depends on local conditions – in some places you end up with more work after you switch to CA. And guess who’s generally stuck doing that extra work? The women. So CA ends up shifting the household division of labour away from men – who usually do more of the tilling – towards women – who often are in charge of weeding.
  3. The wait. Over the long term, it’s pretty well established that you get better yields with CA than with conventional methods: that’s why Conservation Agriculture is so popular among better resourced commercial farmers. But the long term is really long. As Giller and his team found in reviewing the literature, it can be 10 years before CA yields beat conventional yields in an African setting. For ten long years CA yields can be the same or even lower than conventional yields – an eternity for an edge-of-subsistence smallholder with a notoriously compressed time horizon. And there’s evidence that CA takes longest to establish its superiority in “the clay-poor, structurally weak soils of the (semi-) arid areas” which, as it happens, is a good description of quite a lot of Sub-Saharan Africa

For all these reasons – as well as others detailed in the paper – it turns out to be hard to get farmers in much of SSA to switch over to CA methods permanently.

A number of studies report enthusiastic adoption at first, but often it lasts only for as long as project support does, with farmers quickly reverting to conventional methods once the foreign experts leave. There seem to be some exceptions in Ghana, Zambia and Tanzania, but what researcher’s have never documented is any evidence of Conservation Agriculture going viral: spreading from one farmer to his neighbor via word-of-mouth in the absence of promotion efforts by development agencies.

Listen, if you read this blog you know I enjoy a counterintuitive development riff as much as anyone. But even for me, a wholesale denunciation of CA looks perverse. We know conventional farming degrades soils, we know it’s not sustainable, especially in areas with potential for desertification. We know we have to do better than that. So, to be clear, I absolutely understand the zeal to spread the one technology that’s proven to sustain soil health over time.

But along with missionary zeal comes a certain blindness to on-the-ground realities that quickly comes to look like the soft-imperialism of development bloat. In fact, in terms of my Development Bloat Checklist, the typical CA project scores a 5 out of ten:

  • It provides something most people in rich countries have/use, but few people in the recipient community have/use.
  • It’s likely to be abandoned once donor funding runs out
  • People in the recipient community have to significantly change their habits or reorganize their daily routines to make it work
  • If the cost of the project was just handed out to recipients in cash, they certainly wouldn’t spend it on CA
  • Its main target is soil health, not people’s incomes.

You’d have to be blind to miss the signs of bloat here.

The really worrying bit, for me, is how Giller et. al. say people in Development Agencies tend to react to their concerns:

We do not doubt that agriculture is possible without tillage, yet when we question whether CA is the best approach, or whether the suitability of CA in a given setting has been established, the reactions are often defensive. It seems as if we assume the role of the heretic – the heathen or unbeliever – who dares to question the doctrine of the established view.

This tendency to treat skeptics as heretics is a deeply worrying sign of the bureaucratization and institutionalization of the Saviour Complex.

And, when it comes down to it, what’s a bigger threat to African livelihoods: soil erosion, or the tyranny of experts?

TOMS Giving

Toms’ one-for-one: because children this cute can’t be wrong

Did you hear? For every pair of TOMS shoes a western customer buys, they give away a pair in a developing country. To children. Very very cute children. Like these.

Toms’ one-for-one is a really nice idea – but only as long as you don’t spend any time thinking about it at all.

If you do, you grasp why One-for-One has already carved out a cherished place for itself in the Annals of Bad Aid. A perfect synthesis of Bad Corporate Social Responsibility and Development Mission Creep, One-for-One destroys local livelihoods, drives cobblers out of business, in a quest to burnish a fancy brand’s image. (At least it did as originally designed, some say the most recent incarnation isn’t quite as bad.)

It’s the kind of thing that drives practitioners around the bend. Really, there are protest videos. (Though of course it could be worse: they could be giving away used shoes.)

None of which is to say giving a poor child nice shoes for free isn’t nice. Of course it is.

But the real question all these stuff-for-free programmes need to answer is simple: if you calculated the dollar amount it costs you to make and distribute these shoes and just gave it to your beneficiaries as cash, how many of them would turn around and use that cash to buy those shoes?

Continue reading


The Case for Microsavings

When it comes to nerding it up on how the world’s poorest actually manage on extremely small incomes, Portfolios of the Poor is a real achievement: a meticulous reconstruction of the entire financial life of several hundred people living on less than $2/day in three developing countries.

I haven’t quite finished it, so this is no review (you can find a good one here.) But it doesn’t take long to zero in on one amazing finding from the research: every poor household they studied had savings of one kind or another. Even the poorest. The instruments may almost always be informal, the sums may be tiny. But for people dealing with income that isn’t just small but also discontinuous and unpredictable, saving is an absolute necessity.

Portfolios of the Poor is yet another part of the mounting indictment on the conceptual muddle in the old-time microcredit movement, and another part of the now burgeoning case to reimagine credit as one part – and far from the most important part – of a much wider microfinance movement where improved access to microsavings mechanisms play the leading role.

Because not everyone needs a loan. But everyone needs a safe place to save.

A campaigning blog with a focus on rural Africa.


Get every new post delivered to your Inbox.

Join 1,945 other followers

%d bloggers like this: