Do development indicators deceive us? Here is a better approach Seth Kaplan
November 5, 2012 | More on Conflict and security, Economics and development | 6 comments
Measuring how countries perform is all the rage. Everyone from the World Bank to Bertelsmann to Africa’s most famous entrepreneur does it, producing indices on things like how competitive economies are, how hungry populations are, how free the press is, how risky investments are, and how corrupt public sectors are.
Many of these indices are directly relevant for people working in development. They help countries determine how they compare with other states and where they ought to improve their performance. And they help aid agencies decide where and how to invest their resources.
Indicators tracking everything from GDP per capita to poverty to governance are ubiquitous across the field, especially among international professionals. Such numbers are used to determine need, priorities, and strategies (such as whether a government ought to be funded directly).
But do the indicators that have the greatest influence measure the right things? Are they focused on the issues that are most important to development? Can they predict how governments work or how countries will evolve in the future?
Too often, developing countries are assessed on a very narrow set of indicators, leading to an overemphasis on certain programs and “results” that have little to do with their prospects. Reducing poverty and hunger are worthwhile goals but may not reflect how well a country is doing (aid can reduce both without helping a state function better). “Good governance” may indicate good prospects, but bad governance certainly does not point to the reverse, as a long string of countries can attest to (including China, Indonesia, Cambodia, and Vietnam). GDP per capita is widely used to assess how well countries are doing (not least by the World Bank and many leading poverty analysts), but may actually be saying very little about the subject (such as when only elites benefit from natural resource wealth, as in Nigeria, Libya, and Angola).
Indicators on state fragility can easily miss the mark. The Failed States Index, for instance, completely failed to pick up the fault lines that threaten many Middle Eastern countries before the Arab Spring brought them into the open. The 2011 FSI ranked Syria as the 48th most fragile state in the world, but its complex ethnic and religious landscape has always made it far more fragile than it appeared. In 2012, Syria plummeted down to 23rd in the FSI. Next year, it will inevitably be much worse. Bahrain and Libya did not even make the ranking before 2011.
Many of the most important development issues are not included in major indices because they are not easily measured or are simply not considered as important as they ought to be.
In Fixing Fragile States, I wrote:
Development describes a complex process that transforms both the way people think and behave and the system of how they work together. Although economics drives development, politics plays a far greater role in the key take-off stages, with social, business, and government modernization inextricably linked as the process advances.
Do we ever measure how well a people work together? How institutionalized politics is (something quite different than democracy and “good governance”)? How cohesive a population is?
Assessing a country’s political dynamics may not be easy—especially if the goal is to measure it numerically—but is arguably more important than the majority of the indicators we currently use. The right kind of assessment ought to better gauge how resilient a country is, how prone to conflict it is, how stable its current political system is, how likely its elites are to work together to promote progress. All these things help us understand a country’s overall prospects in a way that few existing indicators can.
Measuring politics and political development requires creating a set of indices that reflect—or at least depend upon—the nature of sociopolitical dynamics, the degree of social / political / economic inclusiveness, the institutionalization of the state, the robustness of the rule of law, the level of social capital, the capacity of societies to create wealth (separate from natural resources), and the ability of government to get things done (which may not reflect existing governance scores).
What would these indicators look like? The new assessment criteria would seek to answer questions such as:
1) How great are group-based (ethnic, religious, caste, clan, etc.) economic, political, and cultural horizontal inequities?
2) How equitable is public spending?
3) How equitable are markets?
4) How equitable is the rule of law? Do elites or particular groups have systemic advantages over others?
5) How effective is public authority and the rule of law (taking into account a variety of mechanisms to achieve these)?
6) How inclusive is the concept of citizenship?
7) How equitable is the system of property rights?
8 ) How inclusive and poverty reducing is growth?
9) How diversified is the economy and exports (which depends on the robustness of institutions)?
10) Is political succession institutionalized and predictable?
11) How much does politics revolve around political parties and policies (rather than ethnicity and patronage)?
12) How much do political leaders depend on group identities to gain, hold onto, or compete for power?
13) How well do formal institutions (such as laws) reflect informal institutions? How widely accepted are these? How well do they penetrate society (as opposed to existing above it)?
14) How much investment is going into large factories (which are more risky than other investments)?
15) What is the level of political risk to invest in labor-intensive businesses (which require more effort and are more beneficial to a population)?
16) Is the economy producing an adequate number of jobs for young people?
17) How well can the government implement the policies it puts into place (if a road is supposed to be built, does it? How good is it?)?
18) How well can the government project authority across distance (is it as effective in outlying districts as it is in the capital)?
19) Are the government’s capacity and the country’s economic prospects keeping up with increases in education, urbanization, and the expectations of the population?
20) Are levels of dissatisfaction/frustration rising among powerful out of power actors (elites, identity group leaders, youth leaders, religious leaders, etc.)?
Some of these questions could be turned into indicators very easily (the data is available). Others could be turned into indicators by substituting another data source (for instance, tracking how well a government delivers public services at various distances from the capital will give a decent account of how well it projects authority). Many may be hard to assess, and require a more a concerted effort involving more spending on research and analysis.
If politics and political development mattered as much as they should, more effort would be made to create and use such indicators. Without these, we are flying blind, trying to understand the terrain using the wrong instruments.
Agree with much of the analysis but one small yet annoying point might be worth picking up – you use the terms "resilience" and "equitable" as if there is a universal understanding of what the mean. Sadly that is not the casee, and is one more reason why after meeting the High Level Panel last week I was left feeling that those of us wishing to see a radically different agenda that captured some of the complexity you describe need to do a bit of legwork first … starting with defining terms that might be common currency to us but few others.
I wrote an account of the HLP from my perspective here in case of interest.. http://www.chrisunderwoodsblog.com/2012/11/london…
1, 4, 6, 8, 12, 13, 18 r particularly interesting points. 19 is, i feel, as pertinent to the UK as to 'nations in need'. 20 is relevant to me – practically an elderly woman who is quite likely to express her feelings of being excluded from the world of work in a more radical way, soon (that is not a threat).
of course you're all 'flying blind' – in my opinion this is a conscious massive collusion in order that you all keep your jobs.
lastly, i believe that you development people would all vastly benefit from consulting some good old sociologists. i think they would have ALL the answers that you require.
I agree wholeheartedly with your comment about sociologists! These can be among the most insightful people when studying development, which is, after all, social change on a large scale.
I agree that there is no universal understanding of terms such as resilience and equitable, though there has been a lot of ink spilled discussing them. By "equitable," I mainly mean fair or roughly equal, though these things can of course lie in the eyes of the beholder. "Resilience" is harder to pin down, but can be interpreted as the ability to handle/manage shocks and change and come out reasonably well afterwards. Thanks for the link — I will check it out.
I like what you are proposing to measure; collective social resilience as viewed through the lens of the public sector, as I see it. That is not an exhaustive set of things to measure that measure real value, but it is a very real and comprehensive measure. I think lenses from other sectors would add further value to the metric. A private sector lens on creating value; what it looks like, what you should measure. A non profit sector lens, maybe, of what progress looks like toward their goals, or the world that they'd like to build.
I haven't read this, but it may be of interest to you, in the context of your article:
http://www.uncounted.org.uk/2012/11/a-rude-awaken…