The aid agency nobody knew existed is even worse than nobody realized

I recently saw a June 2009 World Bank (Independent Evaluation Group) evaluation of the Global Forum for Health Research, an 11-year-old international organization that had received $56 million (through 2007) in official aid funding, about half from the World Bank. The Global Forum’s mission is “demonstrating the essential role of research and innovation for health and health equity, benefiting poor and marginalized populations.” The evaluation report is 162 pages long, but two sentences in it would have sufficed:

About half of the respondents to a survey of 400 key researchers undertaken by the evaluation team indicated that they were unaware of the work of the Global Forum. For an organization concerned to raise awareness, this finding in itself raises questions about its effectiveness.

An 11-year-old health-research-awareness-raising organization of which half of the world’s leading health researchers are unaware – yes, I would agree that “raises questions.” I ground-truthed this finding with my own informal check of well-known people that I look up to on health issues – four of them had never heard of the Global Forum, and three others had heard of them but didn’t know enough about them to have an opinion about their effectiveness.

The evaluation has lots of other things to say about them, both positive (they have a lot of meetings!) and negative (“absence of an agreed results framework”).

So the World Bank seems to be following a disturbing trend. It is financing economic research publications that hardly anybody reads, and financing health awareness efforts that hardly anybody is aware of.

This also creates a new challenge for aid watchers – how can we hold accountable an aid agency we don’t know exists? What other dark matter in the aid world is awaiting discovery?

Smart rules and stupid outcomes: the Skip Gates teachable moment

This teachable moment is not only about race. It includes understanding why the Cambridge MA police department would arrest Skip Gates for breaking into his own home, and then continue to insist after a huge outcry that they did the right thing.

My guess is that Sergeant James Crowley was following an inflexible rule that you arrest anyone who shouts angrily at a cop. This may be a good general rule to identify dangerous persons, and having many such rules allows the department to cope with an enormous policing task with limited staff. The support that Sergeant Crowley attracted from other policeman elsewhere may reflect their sympathy with such rules (although the New York Times found wide variations in the extent to which this particular rule is followed).

All organizations have rules for their staff, whose purposes is saving on costs and staff time by prescribing routine responses to different situations. McDonalds makes a ton of money by having rules that can be implemented on a large scale by a relatively small and unskilled staff. As usual in economics, however, there are tradeoffs. Robotic rules may lead to stupid outcomes, outraging and driving away the customers.

I once had a customer service person insist that I could not return a bookcase because I had already opened the box. She admitted I had a valid reason for returning it -- that it was missing a crucial set of screws -- which I could only have discovered by opening the box. But no amount of argument could make her depart from the rule against open box returns. (After further persistence, I eventually got the company to give me the missing screws.)

So organizations choose rule policies that find the sweet spot trading off lower costs of inflexible rules against possibly even higher costs of outraging the customers with stupid outcomes. For private firms, the sweet spot is determined by supply and demand – consumers may be willing to put up with a small amount of stupid outcomes from rules that get them a cheaper product. So a rule is not automatically bad because it leads on a few occasions to a stupid result.

Obviously, the police rule in Gates’ case led to a stupid outcome. The question is what is the sweet spot for police departments? Public bureaucracies don’t respond directly to customer demand in finding their sweet spot, it’s politically determined. Since many of the suspect “customers” are poor and powerless, police departments likely choose to err on the side of sticking to the rules and putting up with the outraged suspects. And historically, they were more likely to perpetrate outrages on black suspects than on white suspects.

All of which suggests something more damning than stupid behavior by one policeman – it looks like the Cambridge, Massachusetts police department has chosen a sweet spot very easy on its own officers and very hard on its customers – and perhaps even harder on the black customers. This is morally and politically unacceptable; the Cambridge Police seem more insulated against democratic accountability than they should be.

Of course, there are a lot of parallels in unaccountable aid agencies. This blog has pointed out cases where USAID refuses to change even when outside critics point out egregious misbehavior. They follow the low-effort rule “just keep doing what you are already doing,” because the critics have little political power over them.

So perhaps as Obama, Gates, and Crowley share a Sam Adams, they can move beyond who said what and discuss making public bureaucracies more politically accountable to the citizens.

We try to make comprehensible the incomprehensible responses from those we criticize

Our targets for criticism have evolved a new tactic of writing longwinded unreadable responses (at least Vernon Smith had brilliant ideas underlying his unreadable book reviewed today). So the Global Development Network wrote us a bureaucratic reply to the charge that they were too bureaucratic. World Vision’s reply to our charge that they were inappropriately manipulating our feelings towards children generated a similarly long-winded reply. To avoid the “tl;dr” comment we got on the GDN response, we did World Vision the service of extracting the high points of their response below. (We posted their letter in full here.) On a far more positive note, we are always grateful when organizations take the time to respond and we think it is a good sign of organizational health and accountability.

Thank you for inviting us to respond…

…World Vision does not focus on children in order to 'tug at the heart strings' and gain greater support for our campaigning and fund-raising work. Rather, World Vision focuses on children as, globally, more children than adults by far live in poverty … and are generally much more

susceptible to the effects of poverty than adults, especially during infancy…

…In response to the comment concerning children's participation at international meetings such as the G8, World Vision, in line with the UN Convention on the Rights of the Child, strongly believes that children have a right to express their views in all decisions that will affect them….

…However, we recognise that children's participation, particularly at large national or international events can often amount to little more than tokenism. As such, in 2008, World Vision worked with Plan International, Save the Children, UNICEF and others to develop minimum standards for children's participation at these events in order to ensure that their engagement is both meaningful and beneficial to all present…

Yours sincerely,

Philippa Lei Senior Child Rights Policy Adviser World Vision UK

You’re rational after all: unconscious development insights from unreadable books

Vernon Smith is a Nobel Prize winner. You quickly realize on reading this book that he got it for economics, not literature. But if you can slog through this book (which took me about 4 months), you will be rewarded with some great insights about development. (But why I am working so hard when Tyler Cowen’s blog is about topless French sun-bathers?)

His big picture is familiar to readers of Hayek: societies develop NOT through the conscious design of some experts (Smith uses the horrible jargon “constructivist” for the design view), but through the “ecological” survival of institutions, norms, rules, firms, and products in a society of freely choosing individuals.

It gets more novel when Smith applies a similar insight to individual rationality. Unlike most of his experimentalist colleagues, Smith is not celebrating findings of human irrationality in the lab as the greatest thing to come along since Sacha Baron Cohen. Instead, Smith wants to look more deeply at “irrationality” and see if in some cases it might be rational after all. The big idea is similar to the above: as humans biologically and culturally evolved, unconscious ways of acting “rationally” passed “ecological” tests of contributing to survival and well-being. The standard “rationality” model that looks at only explicit constraints and logical reasoning (“constructivist”) is way too simplistic for Smith.

An example is the well-known two-player game, the prisoner’s dilemma, when each player gets a higher payoff by cheating if the other doesn’t. The payoff is still very high if neither cheat, and it is the lowest if both cheat. Rational behavior predicts that both players cheat and hence wind up with the lowest payoff. Yet laboratory experiments with real human subjects and real money find that both refrain from cheating surprisingly often.

So players are behaving “irrationally,” yet Smith points out that they have managed to get a better outcome than what “rational” behavior would achieve. He argues that players have unconscious social norms of “fair” behavior (and also they may find ways of “socially” signaling to each other these norms, since one thing we know about humans is that their social skills are highly advanced). Unconscious sociability allows humans to realize gains from social exchange that cannot be captured by the explicit “rational decision” model. He finds more evidence for this idea by subtle variations in the social context of the experiment.

Smith doesn’t address development differences, so the big question is why gains from social exchange are realized more in some societies than others. Maybe he will get to that in his next unreadable insightful book.

Shameless aid behavior awards of the month

3. Bono sings “Every generation gets a chance to change the world.” Another inspirational call to arms to fight African poverty? No, Bono is commercially exploiting his “save Africa” image to shill for Blackberry, who are sponsoring the latest U2 tour." 2. “Children trust adults to keep their promises.” A parental advice web site?

No, World Vision UK is manipulating our feelings about children to campaign for increased aid.

Children rely on G8 promises. Children are speaking out for change – and the G8 must listen.

As if that were not enough, World Vision UK calls for more: some children should attend G8 meetings, to be trotted out for G8 photo-ops:

Shouldn’t their voices be heard at the top table? When will we see a G8 that rolls out the red carpet to listen to someone young and who is not in a suit? After all, it is always much harder as an adult to tell a child that a promise made is not being kept.

1. The Save Darfur thong (from the great blog “Wronging Rights”)

Official Global Development Network Management Response to our post on GDN Research

The following entry was written by the management of the Global Development Network in response to our June 15 2009 post, A $3 million book with 8 readers? The impact of donor-driven research. The main point of our post was that GDN’s annual budget of $9 million had produced paltry results in publications, readers, or citations. We attributed this partly to a decision made early on to administer research through a bureaucracy rather than follow the meritocratic competition model of academic research.

The GDN response, which we received by email this week, is reproduced here verbatim:

A series of points raised the article merit clarifications for factual accuracy.

Objectives of GDN:

• generation and sharing of multidisciplinary knowledge for the purpose of development

capacity building

networking (particularly South-South)

policy outreachIn this context, the number of publications in international development journals measures outcomes in just one of the objectives.

Knowledge Creation, Publications and Career Development:

The GDN Independent Evaluation 2007 reported that:

• a median of two publication types per grant for Regional Research Competitions (RRCs) and for Global Research Projects (GRPs)

on average one of every two grantees published in an international journal, two of three in a national journal, two of three as a book chapter, and just over one working paper per person

these figures represent upper bounds on GDN’s effect on knowledge creation, as GDN cannot take credit in every case due to attribution

researchers surveyed also report enhanced knowledge of their subject, increased visibility, positive impact on careers and valuable networking opportunities through their affiliation with GDNA recent review of GDN conducted by the World Bank’s Independent Evaluation Group in 2008 stated that:

• GDN is supporting increased amounts of development research from within developing and transition countries

GDN-funded research has led to an increase in the dissemination of this research through papers, journal articles and books. (1)

In the 2006 capacity building pilot evaluation, external reviewers assessed much of GDN-supported research to be of publishable quality: (2)

with revision, 66% of papers in FY02 were judged to be publishable in refereed journals rising to almost 80% in FY05

a decline in number of papers not considered worthy of publication in current form from 36% in FY02 to 15% in FY05

demonstrated value of mentoring: if appropriately revised, 79% of papers were worthy of publication in refereed journals in FY05, a 30% increase from FY02.

Citations:

Several past studies noted evidence of higher numbers of citations in open access publications as opposed to expensive journals, although the evidence is not conclusive.

Nonetheless, there is a problem of access to the scientific literature in developing countries. In Gaule (2009), we find, controlling for the quality and field of research, that the reference lists of Indian scientists are shorter, contain fewer references to expensive journals, and contain more references to open access journals than the reference lists of Swiss scientists. (…) The goal of open access advocates to have all scientific publications freely available to the world from the day of publication is laudable. But in the short run, it is more important to make scientific publications freely available for developing countries, because this is where the problem really is. (Source)

GDN is committed to facilitating access, providing free of cost access to datasets and journals (J-STOR, Eldis); to all the GDN-funded research; as well as to almost 14,000 additional papers posted by researchers from developing and transition countries online in GDN’s Knowledge Base.

Cost-Effective Core Activities:

All GDN evaluations highlighted the cost effectiveness of its activities and the low overhead charged by the Secretariat (8-12%) compared to other similar organizations. The cost per study on average (2004-2007) in three noted GRPs have been:

• Bridging Research and Policy = $ 92,243

Impact of Rich Countries Policies on Poverty = $ 57,830

Multidisciplinary and Intermediation Research Initiative = $ 54,211Budget:

The budget for the 3 year Explaining Growth Global Research Project was, according to GDN’s financial records, less than $2 million dollars, including all thematic papers, case studies, capacity building workshops, mentoring fees (including Bill Easterly’s fee), publications and presentations at regional and global conferences.

Funding:

GDN is “not a World Bank-supported effort to promote development research.” GDN is only partly funded by the World Bank, with a declining share over time.

Process

GDN used regional networks to administer the project titled Explaining Growth (GRP) but grant allocation to researchers was competitive. Moreover, the Awards and Medals competition is an open competition with roughly 600-700 submissions annually. All RRCs and GRPs since Explaining Growth (the first one) have not been carried out in a “bureaucratic manner” but through open competitions, the modus operandi at GDN.

Footnotes:

(1) For example, GDN’s publication series, designed to give voice to researchers from developing and transition countries, has released 13 books to-date, with several edited volumes from GDN’s first Global Research Project on “Explaining Growth.” In partnership with the series’ publisher – Edward Elgar, GDN is able to makes these books available for half price to individuals from developing and transition countries registered on its Knowledge Base and copies of all publications under the series will soon be available electronically free of cost to registered users of GDN’s website.

(2) Papers reviewed included all RRC papers available for FY02 and FY05 as well as papers randomly selected by the World Bank, produced through the other major GDN activities (GRPs, Awards and Medals Competition and Annual Conferences) in FY02 and FY05.

What if we are just clueless whether we have a success story or not?

By Diane Bennett and Laura Freschi We had really intended this would be a feel-good story of success:

This month, while the G8 continued their vague pledges of billions in food aid, Eleni Gabre-Madhin was busy bringing a market-based solution to the problem of food security in Ethiopia, as the CEO of the Ethiopia Commodities Exchange (ECX). After years of studying Ethiopia’s agricultural markets, Gabre-Madhin hit upon a central commodities market as a way to end the country’s deadly boom and bust cycle, which has seen overproduction and low food prices one year and famine the next.

The ECX has been running for a little over a year now, and lists five commodities traded on its website—coffee, sesame, beans, maize and grains. With a central trading pit in Addis Ababa, eight warehouses throughout the country, and market data being transmitted through a network of electronic display tickers, text messages, TV, internet and radio, the ECX is designed to reduce the burden of high transaction costs and excessive risk on farmers and traders, and increase access to information for small-scale farmers in remote areas.

In 2008 when the ECX was launched, the Economist called it a “bold experiment to improve the efficiency of agricultural marketing,” and Eleni Gabre-Madhin herself has been lauded as a visionary and a pioneer. (Gabre-Madhin is being profiled tonight on the PBS program Wide Angle in an episode called “The Market Maker"—watch the preview here.)

Unfortunately, it’s not quite that simple.

From the start, the FT alluded to the potential difficulties of a market-based solution in Meles Zenawi’s aid-dependent Ethiopia, “one of Africa’s most state-dominated economies.” The Economist noted that 6 out of the 11 seats on the ECX board would be filled by government officials, which is a massive PR problem in a place where there is precious little trust in the government.

A quick trip down the rabbit hole of the Ethiopian (English-language) blogosphere unearths plenty of cynicism from bloggers distrustful of what they have come to see as a government scheme to get their hands into the farmers’ pockets. The government did little to quell these fears when they accused 6 major coffee exporters of “hoarding,” revoked their licenses, and grabbed 170,000 tons of coffee to sell off at auction to boost Ethiopia’s lagging forex supplies.

The ECX seemed like a great idea. But in the real world of complex politics and conflicting viewpoints, we really are at a loss to say whether ECX has already been a success, or whether it will succeed in the future. Unfortunately, there are a lot more development case studies that are in this category than in the well-trod “inspirational success” or “illustrious failure” genres.

--

Update: This post has been changed to correct a mistake in the second paragraph: "high food prices" was changed to "low food prices."

Didn't we try that in 1938? Why technical poverty fixes fall short

Is African poverty caused by lack of the necessary technical knowledge applied to disease, nutrition, clean water, and agriculture? Reading many discussions, like that of the recent food security initiative, or the UN Millennium Project (UNMP) on how to achieve the Millennium Development Goals, would make you think so. Would it change your mind if that technical knowledge already existed and there were attempts to apply it as long ago as 1938? The following table compares the technical recommendations from a prominent and exhaustive survey of African problems headed by Lord Hailey in 1938 to those of the UNMP in 2005.

African Problem to be Addressed	African Research Survey, 1938	UN Millennium Project, 2005
Malaria	“mosquito bed-nets …malaria control by the spraying of native huts with a preparation of pyrethrum”	“insecticide-treated nets…. insecticides for indoor residual spraying …{with} pyrethroids”
Nutrition	“…the African suffers from deficiency of Vitamin A”	“Malnutrition {is also} caused by inadequate intake of … vitamin A”
Soil fertility	“methods of improving soil fertility {such as} green manuring”	“using green manure to improve soil fertility”
Soil erosion	“increasing absorption and reducing runoff on cultivated land {through} the use of terraces”	“Contour terraces, necessary on sloping lands… when furnished with grasses and trees…{to avoid} soil erosion”
Land tenure	“… legal security against attack or disturbance can most effectively be guaranteed by registration”	“security in private property and tenure rights … registration of property”
Clean drinking water	sinking boreholes	“Increase the share of boreholes”

(A longer version of this table and the citations for the quotes appear in my recent article “Can the West Save Africa?” in the Journal of Economic Literature.)

Enthusiasts for technology fixes for poverty concentrate almost exclusively on the science and the technical design -- this is a characteristic fault of poverty solvers from Silicon Valley, the Gates Foundation, doctors, and natural scientists.

All of the above seem to forget that technology does not implement itself. Technical knowledge needs people to implement it – people who have the right incentives to solve all of the glitches and unexpected problems that happen when you apply a new technology, people who make sure that all the right inputs get to the right places at the right time, and local people who are motivated to use the new technology. The field that addresses all these incentives is called economics.

Are the best aid agencies the ones about to die?

Recently the acting CEO of the Millennium Challenge Corporation, Rodney Bent, invited a group of development bloggers for a congenial chat over breakfast pastries about their new monitoring and evaluation website and “results” portal. This is in contrast, mind you, with another aid organization which shall remain nameless but which has the initials U and N, which told us a couple weeks ago that they “didn’t have a communication policy for blogs” and weren’t sure whether they could give us the document we were requesting (to their credit, they eventually did.) To say nothing of another agency that emailed us: “Hello. I have received your emails and phone call. However, WHO does not participate in blog discussions. Thank you.” And let's not even mention the other US aid agency, USAID, who responds to requests for information by demanding that we talk only to the USAID press rep whose full-time job is not responding to requests for information.

So the bar here is low. And the MCC isn’t perfect. But hey, it’s nice to be given a cup of coffee every once in a while and treated like you exist.

The personable Bent started the meeting with a little story: he recently visited a university classroom and completely disarmed the students—who were eager to rip apart any self-serving propaganda he served up—by being the first to enumerate the MCC’s own failings and weaknesses. The original MCC design team was too tactless about implying that USAID had been a failure, he said, “too optimistic,” too “eager to sign things.” Not to mention the problems MCC has run into in Armenia, Nicaragua, Honduras and Madagascar…to name a few.

This little anecdote seems to represent the MCC’s current savvy outreach strategy: be honest about your failings before others can beat you to it! Hence, the chat and the coffee.

The MCC says many of the right things. Publishing economic rates of return for their projects, providing M&E data for each MCC country in two formats, adding in data visualization and more collaborative feedback tools (all currently available or in the works over the next few months) is MUCH more than many other aid agencies are willing or able to do at the moment.

You can visit the new site here, and use the feedback form to get in touch with Shiro Gnanaselvam, the MCC’s senior director for monitoring and evaluation.

Still the most interesting question to me is, why is the MCC so proactively courting bloggers when other aid agencies tell us to drop dead?

Maybe it is because the MCC—a Bush administration initiative that has never been fully funded—has been predicted to face extinction if it can’t get broad support from the public and from some key power brokers in the new administration.

Could this be a tiny piece of evidence in favor of the theory that effective accountability sometimes requires a threat to your very existence (as with private firms or with the political careers of elected officials)?

The organization with the least vested interests supporting it may be the one that will perform the best. At least, they actually invite their critics to breakfast.

No more African stereotyping that justifies military intervention

by Leonard Wantchekon, NYU Professor of Politics I would like to thank everyone for their comments on my previous post, Africans already got the idea: “Africa does not need strong men, it needs strong institutions.” This is very helpful and healthy.

How many countries that can be considered to be at war, in Africa? The two clear cut cases are Somalia, and Sudan. But you can add Chad and to some extent Congo. That is 4 out of 54 countries in Africa. That is 7 percent of Africa. You can throw in Nigeria or Kenya because of electoral violence. But would we call India a country at war?

It is important to have an accurate picture of conflict in Africa for at least three reasons.

First, policy response would radically differ if we think 20 percent of countries are at war or if it is only 7 percent. In one case, it might be useful to have a neutral rapid intervention force or a trusteeship of failed states. In the other case, all is needed might be technical assistance to courts, and police forces that can be handled locally.

Second, if we want to promote investment and tourism in Africa, it is really counter-productive to exaggerate the security situation. We also need to report progress, which has been significant in the past 10 years.

Third, we really need to underline the fact that political conflicts in Africa are increasingly peaceful despite economic hardship and the sometimes brutal repression as in Zimbabwe. Morgan Tsvangirai should be praised to not have responded to Mugabe’s violence by street violence. Zimbabwean opposition has shown incredible restraint and maturity in their struggle to establish democratic institutions in their country.

Regarding the strong man syndrome, I am not saying that it has have disappeared. Instead, I am saying that things are moving in the right direction. First, a majority of people reject it and second, the new “strong men” are nowhere near the “old” ones in terms of their autocratic style of government. Wade or Museveni are not, and can not possibly be, Mobutu or Eyadema. In addition, why call Obasanjo a strong man, even if he tried and failed to extend his term in office? But I entirely agree with Jeff Barnes when he says we now need to investigate “how do we translate this stated desire for strong institutions over strong men into reality”?

The development challenges in Africa are enormous. But we need to use serious empirical evidence to identify what the real problems are and to acknowledge progress. We need to be careful not to perpetuate this idea that it is “all the same everywhere and all the time in Africa.”

Africans already got the idea: “Africa does not need strong men, it needs strong institutions”

A reaction to President Obama’s speech in Ghana by Leonard Wantchekon, NYU Professor of Politics Overall, I like the theme of the President Obama’s speech in Ghana. Africans must own their future by strengthening democratic institutions and the rule of law in their countries, and by becoming less reliant on assistance. I also like the idea of a real partnership between Africa and other developed countries based on trade. It is very much in line with what most of us would think. He said:

America can also do more to promote trade and investment. Wealthy nations must open our doors to goods and services from Africa in a meaningful way.

As Africans reach for this promise, America will be more responsible in extending our hand. By cutting costs that go to Western consultants and administration, we want to put more resources in the hands of those who need it, while training people to do more for themselves.

What I find a bit questionable is this:

Africa is not the crude caricature of a continent at perpetual war. But if we are honest, for far too many Africans, conflict is a part of life, as constant as the sun ... These conflicts are a millstone around Africa's neck.

My sense is that in saying this he has helped to perpetuate, perhaps unwittingly, the very caricature that he questions. Conflict is NOT as constant as the sun in Africa. While this may have been the reality of the 1970s and the 1980s, it is certainly no longer the case. He forgot to add that many of these conflicts were proxy wars between the US and the former Soviet Union (such as that in Angola), or were manufactured by France (such as that in Congo Brazzaville).

The average African country is at peace. Moreover, it is a democracy, albeit one with relatively weak state capacity, such as Liberia, and Mali. Zimbabwe is the exception, not the rule. And even in Zimbabwe, where there is 90% unemployment, incredible hardships and repression, most people want democracy, not another war.

Freedom, especially freedom of the press, has also drastically improved in the majority of African countries, to the point where Reporters Without Borders have ranked several African nations above developed countries such as Italy and Japan.

Of course, democracy is—as Obama put it—“more than holding elections - it's also about what happens between them”: good governance, human rights, etc. But I see no path to good governance, human rights, and even conflict resolution in Africa unless elections are held regularly. In addition, elections are intrinsically valuable, beyond their potential effect on governance. I am sure Nelson Mandela would agree with that. In fact, various Afrobarometer surveys from 1999 to 2009 suggest that nearly 70% define democracy purely in terms of political rights, not in terms of governance. Perhaps surprisingly, there is no significant difference between rural and urban citizens, or between more educated and less educated citizens.

In terms of the strongman syndrome, things have changed for the better. All across Africa courts and unions have tried (most of the time successfully) to block and prevent constitutional changes that would allow the sitting president to run for an additional term (African presidents have therefore been less successful than the Mayor of New York City in this regard!). Afrobarometer surveys suggest that 75% of Africans reject military rule, 73% reject a one-party system, and 79% reject strongman rule.

I would like the President to acknowledge more clearly that Africans have already got the idea that “Africa doesn’t not need strong men, it needs strong institutions.”

Development Experiments: Ethical? Feasible? Useful?

A new kind of development research in recent years involves experiments: there is a “treatment group” that gets an aid intervention (such as a de-worming drug for school children), and a “control group” that does not. People are assigned randomly to the two groups, so there is no systematic difference between the two groups except the treatment. The difference in outcomes (such as school attendance by those who get deworming vs. those who do not) is a rigorous estimate of the effect of treatment. These Randomized Controlled Trials (RCTs) have been advocated by leading development economists like Esther Duflo and Abhijit Banerjee at MIT and Michael Kremer at Harvard. Others have criticized RCTs. The most prominent critic is the widely respected dean of development research and current President of the American Economics Association, Angus Deaton of Princeton, who released his Keynes lecture on this topic earlier this year, “Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development.” Dani Rodrik and Lant Pritchett have also criticized RCTs.

To drastically oversimplify and boil down the debate, here are some criticisms and responses:

1. What are you doing experimenting on humans?

Denying something beneficial to some people for research purposes seems wildly unethical at first. RCT defenders point out that there is never enough money to treat everyone and drawing lots is a widely recognized method for fairly allocating scarce resources. And when you find something works, it will then be adopted and be beneficial to everyone who participated in the experiment. The same issues arise in medical drug testing, where RCTs are mainly accepted. Still, RCTs can cause hard feelings between treatment and control groups within a community or across communities. Given the concerns of this blog with the human dignity of the poor, the researchers should be at least be careful to communicate to the individuals involved what they are up to and always get their permission.

2. Can you really generalize from one small experiment to conclude that something “works”?

This is the single biggest concern about what RCTs teach us. If you find that using flipcharts in classrooms raises test scores in one experiment, does that mean that aid agencies should buy flipcharts for every school in the world? Context matters – the effect of flipcharts depends on the existing educational level of students and teachers, availability of other educational methods, and about a thousand other things. Plus, implementing something in an experimental setting is a lot easier than having it implemented well on a large scale. Defenders of RCTs say you can run many experiments in many different settings to validate that something “works.” Critics worry about the feeble incentives for academics to do replications, and say we have little idea how many or what kind of replications would be sufficient to establish that something “works.”

3. Can you find out “what works” without a theory to guide you?

Critics say this is the real problem with issue #2. The dream of getting pure evidence without theory is usually unattainable. For example, you need a theory to guide you as to what determines the effect of flipcharts to have any hope of narrowing down the testing and replications to something manageable. The most useful RCT results are those that confirm or reject a theory of human behavior. For example, a general finding across many RCTs in Africa is that demand for free life-saving products collapses once you charge a price for them (even a low subsidized price). This refutes the theory that fully informed people are rationally purchasing low cost medical inputs to improve their health and working capacity. This would usefully lead to further testing of whether the problem is lack of information or the assumption of perfect rationality (the latter is increasingly questioned for rich as well as poor people).

4. Can RCTs be manipulated to get the “right” results?

Yes. One could search among many outcome variables and many slices of the sample for results. One could investigate in advance which settings were more likely to give good results. Of course, scientific ethics prohibit these practices, but they are difficult to enforce. These problems become more severe when the implementing agency has a stake in the outcome of the evaluation, as could happen with an agency whose program will receive more funding when the results are positive.

5. Are RCTs limited to small questions?

Yes. Even if problems 1 through 4 are resolved, RCTs are infeasible for many of the big questions in development, like the economy-wide effects of good institutions or good macroeconomic policies. Some RCT proponents have (rather naively) claimed RCTs could revolutionize social policy, making it dramatically more effective – this claim itself can ironically not be tested with RCTs. Otherwise, embracing RCTs has led development researchers to lower their ambitions. This is probably a GOOD thing in foreign aid, where outsiders cannot hope to induce social transformation anyway and just finding some things that work for poor people is a reasonable outcome. But RCTs are usually less relevant for understanding overall economic development.

Overall, RCTs have contributed positive things to development research, but RCT proponents seem to oversell them and seem to be overly dogmatic about this kind of evidence being superior to all other kinds.

The Soccer Theory of Globalization

by Daniel Kaplan, NYU graduate student in economics and supporter of Bafana Bafana Last season, when the British soccer team Liverpool FC played Real Madrid, the number of Spanish players in Liverpool’s team outnumbered those playing for Madrid. This is one illustration of an emerging trend: while soccer is already the most globalized of sports, it is also fast becoming one of the most globalized professions.

As labor mobility among soccer players has increased, there has been a decline in team inequality at the country level (documented by Branko Milanovic in this 2001 paper). Although traditionally dominant teams like Brazil, Italy, France and Germany continue to win, international tournaments are becoming far more competitive.

Cases in point: at the recent Confederations Cup in South Africa, Egypt beat Italy. And while Brazil eventually won the tournament, they had to fight back from two goals down against the USA (which had never before been in a major tournament final). What is behind the leveling of the international soccer landscape, and how could these lessons be applied to developing countries struggling to benefit from globalization?

Some “soccer economists” argue that nations that are worse at soccer have benefited from exporting their players to world-class foreign clubs, where they gain valuable skills and experience before returning to play for their home country This is similar to recent literature that questions the traditional Brain Drain fear, with the Brain Circulation alternative – skilled emigrants bring home skills and connections that could be as valuable to their home country as the skills brought back by exported soccer players.

But there is also a homegrown story. As Dani Rodrik points out, the Egyptian team that beat Italy had a majority of players with experience playing in domestic, rather than foreign clubs. The USA team that similarly surpassed expectations has key players from both domestic and foreign clubs. So taking advantage of globalization perhaps requires BOTH strong domestic capabilities and international links.

One nation’s strategy for developing a strong domestic soccer league will be very different from the next. American kids who play under the supervision of soccer moms are different from the street kids in a Brazilian favela. Perhaps the venerable theory of comparative advantage needs to become more complex as each country learns to play to its strengths and use more of whatever are its most abundant resources to compete globally. And just like in soccer, it’s hard to predict who will win the game at any particular time in any particular industry. Except the nice thing about trade, unlike soccer, is that both teams win in the end.

Links from Around the Web

Blood and Milk blog calls out a boneheaded Enough Project basketball jersey distribution at a refugee camp in Chad, starts a comment writing campaign, and gets a sincere apology. Nicely done. Oh, turns out the problem is that no one wants the job...Clinton blames lack of willing candidates and "nightmare" vetting process for delay in appointing USAID administrator.

Solid, practical advice from the Good Intentions are Not Enough blog on how to evaluate an NGO before giving them your money (like: check out what kinds of pictures they use to represent their beneficiaries, and look for honest evaluations of past projects, not just happy success stories). Or, you could ask Givewell, which has rated its top 5 international aid charities.

Funny-sad post from an aid worker abroad: Afghanistan: my part in its downfall.

Grading Obama's Africa Speech

Let’s grade Obama’s speech in Ghana on Saturday, July 11 relative to findings from the academic literature on aid & Africa. This is conveniently self-promoting as the current issue of the Journal of Economic Literature has a long literature survey article by yours truly, “Can the West Save Africa?” (Ungated version of the entire article here.) Of course, this is MY particular take on the literature, but it had to satisfy some hypercritical referees and editors who had no agenda.

“Africa’s future is up to Africans.”

A+. This is the big message of Obama’s speech and he nailed it. No, the West cannot save Africa. The JEL survey reviewed disappointments of the Big Push of foreign aid, structural adjustment, governance assistance, and post-conflict reconstruction, which makes it clear that Western actors do not have the knowledge or power to transform African societies, even if West can do some modest things to help African individuals (where there has already been success, especially in health and education).

“What we will do is increase assistance for responsible individuals and

institutions, with a focus on supporting good governance …. concrete solutions to corruption like forensic accounting, automating services, strengthening hotlines, and protecting whistle-blowers to advance transparency and accountability.”

B. On plus side, nice modesty of aid ambitions. On the minus side, seems to imply bad governance can be corrected with technical fixes, which overlook the systemic incentives that motivate the bad guys (as JEL lit review makes clear). Also assumes outsiders can tell good guys from bad guys, and surgically isolate them.

“Our $3.5 billion food security initiative is focused on new methods and technologies for farmers.”

C. Those who forget the past are condemned to hear a Santayana quote. The JEL survey makes clear the great efforts already expended on “new methods and technologies” for African agriculture (some going as far back as 1938), along with the universal agreement among observers that these efforts failed. Next step is to learn from this failure, try something new, not just repeat past attempts.

“Yet because of incentives – often provided by donor nations – many African doctors and nurses …work for programs that focus on a single disease. This creates gaps in primary care and basic prevention.”

A+, Obama nails another one. JEL survey makes clear the collapsing payoffs to the “vertical” approach to health (focusing on diseases rather than patients.)

“We welcome the steps that are being taken by organizations like the African Union and ECOWAS to better resolve conflicts, keep the peace, and support those in need. And we encourage the vision of a strong, regional security architecture that can bring effective, transnational force to bear when needed.”

D. Sigh. Obama seems to fall for the myth of the benevolent, neutral, outside, rapid-response “peacekeepers,” which is a leap of faith relative to the historical record that outside military intervention is rarely neutral and rarely available rapidly “when needed” (JEL article). Any given African country will not automatically see an outside force as neutral just because it is made up of other Africans.

“our Africa Command is focused not on establishing a foothold in the continent, but on confronting these common challenges to advance the security of America, Africa and the world.”

F. Double sigh. Why is Obama continuing Bush’s terrible idea of an Africa Command? Despite goodwill for Obama, goodwill for US military is nonexistent after a long history of Cold War Africa interventions, post-Cold War interventionist fumbles, reinforced by the more recent fiascos of Iraq and Afghanistan. Africans will never see US military (or any other Western force) as a neutral and benevolent force. They understand Great Power incentives better than the Powers themselves seem to.

“You can … make change from the bottom up…Freedom is your inheritance. Now, it is your responsibility to build upon freedom’s foundation. And if you do, we will look back years from now to places like Accra and say that this was the time when the promise was realized – this was the moment when prosperity was forged; pain was overcome; and a new era of progress began. This can be the time when we witness the triumph of justice once more. Thank you.”

A. Obama returns at the end back to his central idea. The speech seems written by different people with different views, which I’m sure it was. Let's hope the other advocates lose out to this closing inspiration of Bottom Up efforts of creative, free individuals -- contradicting the Top Down ideas like military intervention or expert fixes for corruption or agriculture -- hopefully the view closest to Obama’s personal vision.

The Pope, the G8, and the “Man in Charge” Fallacy

During the G8 meetings, Pope Benedict released a new encyclical saying “there is urgent need of a true world political authority.” This authority is needed:

To manage the global economy; to revive economies hit by the crisis; to avoid any deterioration of the present crisis and the greater imbalances that would result; to bring about integral and timely disarmament, food security and peace; to guarantee the protection of the environment and to regulate migration.

The Pope has fallen for the venerable “Man in Charge” fallacy. There is a “Man in Charge” of the global economy (or, according to the Pope, there should be). So all that we need is to get the right comprehensive set of recommendations to the Man in Charge to fix the global economy.

This a fallacy simply because: there is no Man in Charge of the global economy, There never has been, and there never will be. There is no Man in Charge of any national economy either. This is not about the debate about the role of government vs. markets, this is simply a statement of fact. There are government leaders, to be sure, but they are only one among many different power centers in the political system, society, and in the economy, all with sharply conflicting interests and tools to effect change, and so any individual leader has very limited power to change things. This is true of both authoritarian and democratic systems. You may want one of those leaders to act on a particular problem, which is fine, but you should not think that leader is the Man in Charge.

The Man in Charge fallacy contaminates much of the discussion in development economics. There is an endless search for the right comprehensive strategy to end global poverty, or to achieve national economic development. Such a search only makes sense if there is a Man in Charge, which there isn’t, who would have the power to implement The Strategy. Once you realize that all power is partial, you seek ways to achieve incremental beneficial changes and you end the unproductive search for the Complete Grand Plan to solve the problem all at once.

We pay excessive attention to G8 summits because we think the G8 is the eightfold Man in Charge. That this is a leap of faith is particularly obvious this year. The G8’s solution to the problems of the poor is apparently to bore them out of poverty through sheer wordiness. They reaffirm previous reaffirmations, amplify previous amplifications, and clarify previous clarifications.

Why do we all fall for the Man in Charge fallacy? We like to anthropomorphize a complex system of multiple power centers, bottom-up social norms, and spontaneous markets, innovators, and entrepreneurs, because it is scary to think of such a complex system with no Man in Charge. Pope Benedict XVI understandably thinks it’s possible to have a Man in Charge of the global economy since official church dogma says he himself is the Man in Charge of the church (although perhaps this is just as much a leap of faith). For the rest of us, maybe it goes back to our evolutionary caveman brain wiring, when there really was more of a Man in Charge of a simple hunting band during caveman times.

Once we free ourselves from the Man in Charge fallacy, all of us are set free to use whatever talents or levers of power are at our disposal to contribute our own tiny part of a solution to one problem – and there are so many of us, that many problems will be solved. We know this because, with the greatest improvement in standards of living and social indicators in the past half century in human history, we have already achieved an amazing amount with no Man in Charge.

Respecting local values: Western confusion about African orphans

When a remote area of South Sudan was resettling from the long-running civil war in 2001, tens of thousands of returnees were threatened by the upcoming rainy season without food. A small team was dispatched to assess and prioritize the needs of internally displaced people (IDPs) resettling in a corner of South Sudan. (Sudan continues to have the largest number of IDPs in the world, even without exact numbers from half the country.) I was part of the team in a village of 1,000 residents, where roughly 30% were orphaned children whose parents had been killed in the war or died without medical care.

Our team was horrified when we learned that lions actively hunted in this area, killing children daily without protection of shelter or family. As a result, protecting the most defenseless residents became our immediate concern. Elsewhere in Africa, top-down Western aid builds orphanages in such situations, but I knew the Western and African concepts of family and “orphans” are different. There are already so many orphanages that the website orphanage.org lists dozens to solicit donations and encourages readers to check back because “orphanages are being added frequently.” You can even be an orphanage tourist for a week!

Ubuntu is an African concept of interconnectedness, of a collective belonging and understanding of interdependence. In different regions of Africa it has varying rights and responsibilities, and I was unsure of the application in this remote place. I asked the village’s women’s council about who was caring for the orphans and they admitted there wasn’t enough food to share. We went on to discuss food security and our NGO’s role in helping the IDPs resettle. We promised food assistance for their transition and planting resources and asked if they would they take these children into their homes as part of their family, since the orphans were from their tribe. They readily agreed it was their responsibility, as these were the children of extended family members. With this agreement in place, within weeks all but five children were connected to extended family members that cared for them as their own children.

True to our word, our NGO brought in emergency food supplies, then seeds and agricultural tools. A year later, insufficient rain created a temporary food crisis and we again brought in supplemental food supplies to help them get through. Our commitment was to the village, that they survive the transition to self-sufficiency.

Within a short time of our first visit, there were no more lion attacks on helpless children and we never heard another word about the hundreds of orphaned children. The village has grown to about 15,000 people and today they grow the food they need. By making a commitment to this village, we helped the village to take care of its own challenge. And there was no expensive, Western-style orphanage institutionalizing them until adulthood and no long-term expense. Furthermore, the culture of this village has remained essentially undisturbed. The village leaders can be proud that these children have become part of their ubuntu connections and been raised by relatives speaking their own language and taught the ways of their people.

When aid solutions are empowering and consistent with local values and culture, in my experience they are less expensive, sustainable and more respectful. Interventions can help people get back on their feet to find their own way, not one imposed by aid organizations with agendas or expensive, delaying overhead. Let’s stop pretending that aid work can be ignorant of local values and culture; this is an essential part of constructive aid work.

Don’t say colonialism! the debate on Paul Collier

Boston Review has a special issue on Paul Collier’s ideas about how the international community can help “the Bottom Billion,” with some commentators including myself. For the first time, Professor Collier responds to some of the criticisms I have made of his arguments, so this is a good opportunity to see if we can advance the debate.

First, I have to remind those who don’t like debates that they do play a constructive role. Psychologists have documented the phenomenon of “groupthink,” in which every member of a group believes something becauses every other member of the group believes it, often prodded by an authority figure. A dissenter is very useful even if the dissenter is WRONG and the group is RIGHT as a challenge to make them independently justify their arguments, not just echo each other. Of course, when groupthink is WRONG, then you need dissenters far more. Dissenters can even prevent plane crashes when “groupthink” in the cockpit goes in a bad direction! (see the discussion in the great book Sway)

Second, I congratulate Prof. Collier on his success in convincing many influential people of his arguments. However, such success does make it even more essential that his dissenters are answered. It is understandable that Prof. C is a bit impatient with the few remaining dissenters after his success at persuading Downing Street and Beltway think tanks; he keeps citing eminent authorities who agree with him; he classifies the two dissenters in this Boston Review issue – Berkeley Professor Edward Miguel and myself – as outside “the core of the serious range for policy discussion.”

It is also understandable that the busy professor has not had enough time to fully read up on the arguments against him, the more serious of which he ignores and the less serious of which he misunderstands. He ignores the evidence against his image of catastrophic economic growth lasting forever – so central to his Bottom Billion concept – instead, movement back towards world averages is characteristic of previously poor growth performers on average. And as Miguel pointed out, and Collier acknowledges, Africa has already moved back to average world economic growth since the new millennium.

Collier is also upset that I “grossly misrepresented” him by saying he had a “poverty trap” theory of the Bottom Billion. The usual notion of a poverty trap is that poverty causes something bad to happen, and then that something bad prevents growth out of poverty, so you are trapped. Prof. Collier says that “poverty” is one of the factors that makes “rebellion easy,” as well as military coups. And “even the possibility of war is enough to deter investment and stunt growth,” and the same with unpredictable coups. So the “something bad” is war or coups, which makes growth impossible. Collier’s argument actually fits perfectly the definition of “poverty trap.” However, let’s honor his wish to call it something else, perhaps a “an income deficiency snare.”

I have for too long been harping on how unreliable is the “data mining” apparently used by Collier. Collier knows that few outside academia care about these statistical problems, so he plays to his audience by asserting hidden motives: “behind expressions of statistical fastidiousness lurks a recognizable philosophical hostility to public action.” Once again the harried Prof. Collier did not have the time to read my “New Data, New Doubts” paper that he cites (which partially applied to Collier’s own earlier work on aid and growth), so he missed the point – if an old result disappears with new data, that is consistent with the original result coming from “data mining” on the old sample of data.

I was foolish enough to refer to Collier’s recommended policies as “colonialism,” which he deems “coarse thought, not statistical rigor.” Collier understandably paints me as the wild-eyed extremist, while he is the reasonable advocate for democratically sanctioned international peacekeepers. Let’s double check, just to be sure. The UN Security Council decides on military intervention (“peacekeepers”) or a Great Power does it on their own. Two of the Council’s permanent members are authoritarian, most of the Great Powers follow their own geo-strategic interests most of the time, and none of them have any democratic rights for Bottom Billion citizens to make Security Council or Great Power foreign policy decisions. (Small caveat: There never has existed or will exist a benevolent and politically neutral international force that will rapidly deploy to surgically solve Bottom Billion problems.) Yet the Great Powers will decide according to Collier’s proposals whether an “area or people” are allowed to have elections, whether the elections are legitimate when allowed, and when to send in the military (which, despite the nice “peacekeepers” label, are in a purely technical sense made up of soldiers carrying guns that are aimed at people.)

The dictionary definition of “colonialism” is “Control by one power over a dependent area or people.” I agree that permanent colonies are a thing of the past, but the above description sure sounded a lot like “control” of “a dependent area” by outside powers. Many may indeed think me way out of line to call Collier’s proposals by the inflammatory word “colonialism” just because of the technicality that they actually fit the definition of “colonialism.” But us dissenters will persist anyway because the Bottom Billion deserve better than control by a development expert with an army, they deserve democratic rights just as much as all the other Billions.

Who’s in charge of global health spending?

There’s something I’ve noticed when talking about international aid to people who work outside the development/aid community: they always assume that there’s someone, somewhere who’s in charge. The concept they seem to have in mind is something like the Deist “Divine Watchmaker”—not a bearded fellow dictating our every move, but rather a benevolent force that set things in motion and is now generally keeping tabs on us.

The degree to which this is not in fact the case was driven home recently by a new study out of the Institute for Health Metrics and Evaluation at University of Washington which sought to follow every dollar—public and private—spent on development assistance in just one sector: health. One of the editors of the report, William Heisel, compared this colossal effort to counting the drops in a rain storm.

Lead researcher Nirmala Ravishankar and her team at UW uncovered several important stylized facts about global health spending over the last two decades (their results were published in the June 20 edition of the Lancet). For example, total health spending has almost quadrupled, from $5.6 billion in 1990 to $21.8 billion in 2007.

Now get ready for the bad news: Nearly one-third of the global health money spent by the very largest donor by far—the US government—is untraceable. The study highlights the large gaps in existing health data and comments, “Surprisingly, discussions about global health financing continue to take place in the absence of a comprehensive system for tracking [development assistance for health].”

The team plans to publish more results, delving deeper into transparency issues, towards the end of July.

Dr. Ravishankar put it better than we could: “If no one knows how exactly this money is being spent, then we will never know if it is making a difference.” Amen to that.

The Tragedy of the Millennium Development Goals

The United Nations today issued its Millennium Development Goals (MDGs) Report 2009. To make a long story short, the accompanying press release says:

The assessment, launched today by UN Secretary-General Ban Ki-moon in Geneva, warns that, despite many successes, overall progress has been too slow for most of the targets to be met by 2015.

Let’s face it: it’s over. The MDGs will not be met (the above statement was based on trends BEFORE the economic crisis hit, see crisis discussion below).

What went wrong? The UN was very successful in getting lots of people interested in global poverty who had not been interested previously. An enormous advocacy campaign resulted (see the video from YouTube above). The enthusiasm of the young and of many public figures was deeply inspiring.

What was the theory of social change behind the advocacy of the MDGs? Political advocacy is most successful when you can identify WHO is to blame for an injustice, and WHY (according to what principle) the situation is unjust. This points to WHAT the WHO should do.

The trinity of WHO/WHY/WHAT worked for positive social change, for example, for movements such as the American Revolution, the abolition of the slave trade, slave emancipation, the extension of the vote to the working class, the women’s rights movement, the end of colonialism, the civil rights movement, and gay rights.

WHO is to blame for missing the MDGs? Advocates enthusiastically advertised that 189 leaders signed the Millennium Declaration in 2000, but that was actually a sign of weakness rather than strength. Does an agreement have teeth when EVERYONE agrees – including many oppressive governments who had no more interest in alleviating poverty than in promoting Brussels sprouts? And if the agreement is broken, how can you find WHO is to blame, when 189 leaders (not to mention dozens of international organizations and NGOs) are COLLECTIVELY responsible?

The WHY and the WHAT were also murky, since there is little consensus on what causes poverty and how to end it. The responsibility is put on governments (see the YouTube video, for example), but the rest is unclear (WHICH one? WHAT should they do?)

The MDGs only content is that certain outcomes should be achieved by 2015, but all of these outcomes depend on many other factors besides government actions. The effect of the current crisis is a case in point. No doubt the crisis will be used as the excuse for the MDG failure (as the UN MDG 2009 report is already doing). But the MDGs’ attainment depended all along on global and national economic growth. How can you hold somebody accountable for something they don’t control? – that’s not true accountability at all. (Even someone as dense as yours truly pointed out this flaw long before the current crisis came along.)

The inspirational enthusiasm and increased efforts surrounding the MDGs probably did contribute to progress on specific efforts and some partial success stories (mainly in health and education), as pointed out in the UN MDG 2009 report. That can give some hope for the future and some solace to the hard-working and deeply committed participants.

But the point of the MDG campaign was that it precisely defined success and failure using specific goals. So on its own terms, it is a failure.

The MDGs will go down in history as a success in global consciousness-raising, but a failure in using that consciousness for its stated objectives. What a tragedy for all of those who contributed such effort and enthusiasm to the MDG campaign. And a much larger tragedy for the world’s poor.

Why waste any more effort on the MDGs, now that we know they will not be met? The next effort should get the WHO/WHY/WHAT clear. Here’s one suggestion for starters: the WHO is aid agencies, the WHY principle is that they are responsible for these funds entrusted to them to reach the poor, the WHAT is transparency on whether the funds did reach the poor. It is unjust that funds intended for the poorest of the poor wind up enriching somebody else not poor. Let’s have a movement protesting THAT injustice.