Feeds:
Posts
Comments

Archive for the ‘Accountability’ Category

An experiment too far?

Posted in Accountability, Evaluation, Evolution, Innovation, Knowledge and learning, Public Policy, Research on May 3, 2013 | 6 Comments »

This week a breaking story in the UK focused on how unemployed jobseekers are being forced to complete bogus psychometric tests designed by the government’s Behavioural Insights Team (commonly known as the “nudge” unit). The story raises important issues for ethical experimentation which are very pertinent for aid efforts.

The Guardian reported the story as follows:

The test called My Strength… has been exposed by bloggers as a sham with results having no relation to the answers given. Some of the 48 statements on the DWP test include: “I never go out of my way to visit museums,” and: “I have not created anything of beauty in the last year.” People are asked to grade their answers from “very much like me” to “very much unlike me”. When those being tested complete the official online questionnaire, they are assigned a set of five positive “strengths” including “love of learning” and “curiosity” and “originality”. However, those taking the supposed psychological survey have found that by clicking on the same answer repeatedly, users will get the same set of personality results as those entering a completely opposite set of answers.

The Behavioural Insights Team, meanwhile, argue that their intentions were based on sound evidence and good intentions. This includes the finding from randomised controlled trials (RCTs) of the survey that it had led to ‘building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.’

For many critics, however, any potential positive benefits of the exercise were diminished by the fact that jobseekers were warned that the survey was compulsory and not filling it out would lead to allowances being curtailed. Instead of building wellbeing, this exercise simply gave the unemployed something else to worry about.

Clearly, there are some fundamental ethical problems with the way that this whole effort was designed and implemented. And of course, this is not unique to nudge efforts, but extends to all kinds of social policy interventions. But the experimental approach of nudge does open up a range of ethical quandaries that we need to be looking at more closely.

What the admirable efforts of the UK blogger community highlight for me is that aid recipients in developed countries do have some means for addressing their grievances about such experimental processes – even if (as in this case) they are indirect and work through informal rather than formal channels of accountability.

However, the poor in developing countries have few such channels for voicing their grievances and issues. As one statistician put it back in 2010 in a review of RCTs:

In conducting research with people, the need for guidance and adherence to ethical standards is of the utmost importance. Most areas of research involving human subjects have compulsory or voluntary codes of conduct and ethical rules, and many countries have strict processes in place to ensure that ethical standards
are met by any research involving human experimental units. There seems to be a gap, however, in research that involves human subjects carried out in the context of international development. We do not have a system of checks and balances that ensures adherence to high ethical standards. This may be because the jurisdiction of research committees does not extend
to the areas where some of this research is conducted.
And this, specifically on RCTs:
When RCTs are proposed for impact evaluation, the issue of consent from participants is not discussed. Telling a group of people that they will be included in an experiment, but not implementing a development intervention that might benefit them, is something that most people working in international development would find difficult.
There is a lot of talk about feedback mechanisms at the moment as a means of addressing the long-observed ‘broken feedback loop’ in foreign aid. But without “a system of checks and balances that ensures adherence to high ethical standards” such mechanisms will be prone to the same problems as other mechanisms used in development.

In a study I wrote on innovation in aid with Kim Scriven and Conor Foley a few years back, we argued that there was a need to find safe spaces for experimentation, and establish mechanisms to promote “honourable risk” if we wanted to see a more innovative, and yet still principled, aid system.

Even though other aspects of aid innovation have advanced considerably since that study, especially in terms of resources and policy attention, I am not sure we have really seen much progress on the issue of “honourable risk”. As a result, many of us in development aid run the risk of taking our experiments just a little too far.

In fact, we may be doing so already, and not know about it.

Read Full Post »

The Aid Explorer: Mapping the Hyperspace of International Aid

Posted in Accountability, Evaluation, Facilitation, Innovation, Institutions, Knowledge and learning, Leadership, Networks, Organisations, Public Policy, Reports and Studies, Research, Self organisation on May 2, 2013 | 1 Comment »

The team behind the Atlas of Economic Complexity (see my post on this here) have come up with a fascinating network-based approach for analysing the global aid system.

As they put it:

International development is a complex global goal that faces massive coordination barriers. The difference in income between rich and poor has expanded over the years from a four to one factor to a hundred to one. Where there once were only a handful of development agencies, thousands have now emerged. The system that connects donor agencies, recipient countries and development challenges is extremely complex and should not be managed with a top-down approach… The Aid Explorer was developed as a tool to facilitate better aid coordination. The Aid Explorer enables users to understand what issues face which countries and which aid organizations are aligned to address these issues.

Some specific pointers:

  • The Aid Explorer’s Profile pages enables us to see which issues face which countries and which organisations are best aligned to address them
  • The Network maps can be used to explore how issues, countries, and organizations relate to each other
  • The Rankings presents the findings and the best alignments of countries, issues and organizations

The process of developing the dataset, and how to use it, is described in more detail in the accompanying paper “The Structure and Dynamics of International Development Assistance“, published earlier this year in the Journal of Globalization and Development.

Read Full Post »

Lies, Damned Lies and Big Data

Posted in Accountability, Communications, Evaluation, Innovation, Knowledge and learning, Research on February 1, 2013 | 16 Comments »

This is a guest post by David Hales, a fellow associate of the new complexity think-tank, Synthesis. David specialises in computational social science and here he provides a thought-provoking response to the rise in big data, and some of the more outlandish claims made about it. For a good example of the latter, see Chris Anderson’s piece ‘The Data Deluge Makes the Scientific Method Obsolete‘. In this piece, David makes some very relevant points for development big data initiatives.

david-hales-chicheley-hall

  

Almost everything we do these days leaves some kind of data trace in some computer system somewhere. When such data is aggregated into huge databases it is called “Big Data”. It is claimed social science will be transformed by the application of computer processing and Big Data. The argument is that social science has, historically, been “theory rich” and “data poor” and now we will be able to apply the methods of “real science” to “social science” producing new validated and predictive theories which we can use to improve the world.

What’s wrong with this? On one level nothing. We know so little about the social world that anything is worth a try. Mining these huge databases will almost certainly lead to new ideas and insights. However, before we run headlong into this new world of big data, promoted as it is by corporations such as IBM and the large consultancies, perhaps we might benefit from a little critical reflection.

Firstly what is this “data” we are talking about? In it’s broadest sense it is some representation usually in a symbolic form that is machine readable and processable. And how will this data be processed? Using some form of machine learning or statistical analysis. But what will we find? Regularities or patterns (for a useful discussion of patterns within complex systems, see Greg Fisher’s post, Patterns Amid Complexity). What do such patterns mean? Well that will depend on who is interpreting them.

Given this level of generality, if someone tells you they are working on “big data” it tells you almost nothing. One way to approach the issue if confronted with a “big data” project is to ask the following question based on a thought experiment:

Imagine you had a massive computer database that contained all possible measurements that could ever be made over the entire span of all space and time. You could query it with any question and it would deliver the result instantaneously. All big data is merely a subset of this ‘the biggest data that could ever exist’.  What would your project ask it?”

If no coherent answer can be produced to this question then any such project is at best directionless and at worst not conscious of its aims.

One answer might be “looking for patterns or regularities in the data”. Looking for “patterns or regularities” presupposes a definition of what a pattern is and that presupposes a hypothesis or model, i.e. a theory. Hence big data does not “get us away from theory” but rather requires theory before any project can commence.

What is the problem here? The problem is that a certain kind of approach is being propagated within the “big data” movement that claims to not be a priori committed to any theory or view of the world. The idea is that data is real and theory is not real. That theory should be induced from the data in a “scientific” way.

I think this is wrong and dangerous. Why? Because it is not clear or honest while appearing to be so. Any statistical test or machine learning algorithm expresses a view of what a pattern or regularity is and any data has been collected for a reason based on what is considered appropriate to measure. One algorithm will find one kind of pattern and another will find something else. One data set will evidence some patterns and not others. Selecting an appropriate test depends on what you are looking for. So the question posed by the thought experiment remains “what are you looking for, what is your question, what is your hypothesis?”

It seems to me that one must at least try to answer this question if one is to pursue social science. Not just because it is good science but also because it has ethical and political implications.  The view one takes of social phenomena, either consciously or through algorithms and data, frames what is and is not conceivable for past and future social reality. If you doubt the importance of such ideas one should look that the history of the 20th century. Ideas matter. Theory matters. Big data is not a theory-neutral way of circumventing the hard questions. In fact it brings these questions into sharp focus and it’s time we discuss them openly.

Right now we are “data rich” and “theory poor”. We need new theory for the 21st century. That requires critical discussion, reflection, honestly and humility. It is not clear to me that such concerns are prominent within much of the “big data” movement.

Here is a more eloquent and playful take on these issues, by a colleague of mine, in the genre of that wonderful Orwell fable: https://scensci.wordpress.com/2012/12/14/big-data-or-pig-data/

Cross-posted from the Synthesis blog.

Read Full Post »

Questioning innovation

Posted in Accountability, Economics, Evaluation, Evolution, Influence, Innovation, Knowledge and learning, Leadership, Networks, Organisations, Public Policy, Reports and Studies, Research, Technology on January 30, 2013 | 5 Comments »

Innovation is popular in aid at the moment, so much so that there is a steady spate of articles which range from trashing its potential contribution to development through to challenging Western, donor, countries’ assumed roles as the ‘providers’ of innovation.

In this post I want to argue that there is a middle  ground between the unthinking mantras that are increasingly peddled by agencies and the growing number of entirely justifiable critiques.

An Economist article made the point succinctly over a decade ago, ‘what precisely constitutes innovation is hard to say, let alone measure.’  Some concluded, as a result, that innovation was a ‘new theology’.

innovation-prayer

However, with a growing chunk of economic growth being driven by industries and products that in fact didn’t exist ten years previously, such dismissals seem increasingly Luddite.

While clarity and precision in thinking about innovation is all-important, it is far from easy. We are not helped by the fact that many innovation stories are in fact apocryphal – retrospectively woven to lend the star protagonists much more agency and awareness than in fact they possessed. This is true of even the best known innovation stories. Take Alexander Fleming’s infamous and much-lauded discovery of penicillin – Fleming himself used to describe the conventional account of his contribution as the ‘Fleming Myth’.

Typically in business the market is the ultimate arbiter of innovation, and as we know, most products fail. In aid, however, the market does not provide an adequate indication of what is successful and what is not. This is largely done by the aid system itself. Bill Easterly made much of the fact that the market could get Harry Potter books anywhere there was a demand, which he compared unfavourably to the inability of the aid system to get simple treatments like vaccines to where they were most needed:

There was no Marshall Plan for Harry Potter, no International financing Facility for books about underage wizards. It is heartbreaking that global society has evolved a highly efficient way to get entertainment to rich adults and children, while it can’t get twelve-cent medicine to dying poor children.”

But as Amartya Sen subsequently argued:

the disparity in the results is indeed heartbreaking… [but] there is a radical difference… between the enterprise of supplying “what is in demand” — which is integrally linked to the buyers’ ability to pay — and that of supplying needed goods and services to people whose income and wealth do not allow a need to be converted into a market demand.

While Sen’s point applies more broadly to aid delivery, it is also relevant to new ideas and innovations within aid.

In any case, whether because of market failure or the wilful self-interest of aid agencies, innovation – which is an ambiguous enough concept in the business realm – becomes very murky territory indeed in development. It is hard to say  what innovation actually is, what it generates, and for whom. Like the famous ruling about pornography, many are of the view that ‘I know it when I see it.’

Such vagueness is the ideal seeding ground for development fads, and indeed, innovation is fast becoming the latest ‘fuzz-word’. Everything is being labelled ‘innovation’: as one blogger memorably put it, we seem to be suffering from Innovation Tourette’s.

Problem

Little wonder that growing numbers of thinkers and writers see the need to beat innovation with a big snarky stick. These criticisms play a vital role in highlighting the risks and downsides of  all shiny new aid agendas – and innovation is no exception.

Having observed such trends in the past, I think there is a danger that between the rise of the fad and the indignant reaction to it, we lose sight and sense of why the issue is question is actually important. Specifically, we risk learning the wrong lessons about what innovation actually is, and the potential it has to add to our work.

What we need is a more precise and accurate way by which to separate the innovation wheat from the faddish chaff. This was in fact one of the key motivations of a study I co-authored with Kim Scriven and Conor Foley while at ALNAP back in 2008-9.

So what did our study suggest in terms of getting more precision in innovation? We found it useful to ask some key questions to identify whether a particular idea or approach was in fact innovative.

  • Q1. Is the idea being proposed a new Product, a new Process or service, a new way of Positioning aid, or a new Paradigm or mental model? Or is it some combination of the four? (see more here)
  • Q2. What are the origins of the idea, and what does it aim to do differently to what is already out there? Where, exactly, is the novelty – is it a whole new thing, or is it new combinations of existing things? What, in partciular, are the implications for relationships with aid recpients? Did the idea involve re-thinking that age-old and much-critiqued relationship?
  • Q3. How disruptive is the innovation? Is it transactional, in that it enables existing efforts to work; incremental in that improves these efforts, or transformational in that it radically changes these efforts?
  • Q4. What precisely are the expected benefits the idea should confer? Can these benefits be framed in terms of existing evaluation criteria of enhanced relevance, efficiency, effectiveness, impact or sustainability of aid? Or are there other, newer, criteria that matter? How can the benefits be measured – qualitatively, quantitatively, or some blend thereof?
  • Q5. What are the potential risks and downsides of the idea for all parties – especially aid recipients – and how will these be mitigated against?
  • Q6. Where can the idea be located in an overarching innovation process? Is it at the early stages of recognition and invention, is it in need of development and implementation, or has it been tested and is now ready for wider diffusion?  (see more here)
  • Q7. What are the networks and relationships that will support and facilitate the innovation process? What capacities and competencies are necessary? Are these in place? How can they be built?
  • Q8. What is the potential scope of the innovation in terms of wider diffusion? Who might benefit, and in what ways? What is the route to scale, and who needs to be engaged to get there?

There are no doubt many more questions that could be asked, but the above provide a good starting point for what might be termed ‘innovation due diligence’. The key, in my opinion, is to use these and others questions to develop more honest, rigorous stories about ongoing and historical  innovations: about how they came about, why, and with what benefits. Such questions are useful because they help us look at innovation from a more systemic perspective: looking not just a idea, but the overall social, technological and institutional context from which it has emerged.

These questions should be relevant whether you are a donor bombarded with new proposals and ideas, an operational aid worker seeking to get funding for your exciting new idea, a blogger wanting to shine a light on the depressing excesses of innovation-speak, or a researcher wanting to investigate an supposed innovation in a systematic fashion (in fact I think we need far more of the last category, but that’s another story).

We will need to keep wielding the big stick as necessary, to curb against such excesses of aid ‘innovation-speak’. But we may also at times need a magnifying glass and ruler – metaphorically speaking – and asking these kinds of questions could help with this. My $0.02 is that if a would-be innovator can’t take a reasonable stab at these questions, they aren’t working hard enough, or they are over-selling something. A lot is spoken about creativity in innovation, but recent work suggests – in echo of the old 1% inspiration, 99% perspiration line – that the larger part of innovation lies in the proper execution of the idea.

I think there is a special role for the aid blogging community in asking such questions and demanding answers. We have seen in the past few years how bloggers have mobilised in a largely self-organised fashion to push back against various poorly considered ideas.

I know there are many bloggers who want to engage with innovation in a serious fashion, and who are dismayed by the current hype surrounding it. We should be able to highlight the good and bad of what we see emerging from the aid innovation agenda. And aid agencies should be willing to open their ideas up to the views and scrutiny of this emerging, globally networked, community of thinkers and analysts. This kind of effort has, in other distributed sectors, developed into new crowd-sourced marketplaces for innovation such as Innocentive. There’s no reason why the same couldn’t happen in our sector.

Our ultimate goal, I’d argue, should be to work to bringing the perspectives of aid recipients into the mix as part of our standard operating procedures. Now that in itself could be seen as a real innovation.

The need for such engagement goes beyond mere niceties. The most effective ideas we uncovered in our 2009 study were precisely those that re-thought and re-formulated this core aid relationship: cash, community approaches to malnutrition, transitional shelter. Put simply, these were the innovations that we found to be most worthy of the term.

Gamechanger

Read Full Post »

Reflections on bias and complexity

Posted in Accountability, Evaluation, Healthcare, Knowledge and learning, Reports and Studies, Research on May 29, 2012 | 6 Comments »

Earlier this month, Nature published a piece by Daniel Sarewitz on emerging challenges faced in science and research, which has some useful lessons for the aid system.
The greatest threat to science is not  due to the usual suspects of “inadequate funding, misconduct, political interference”, etc, etc. Instead, according to Daniel Sarewitz, the problem is more fundamental and relates to a widespread bias towards over-simplified models and positive results.

Bias is an inescapable element of research, especially in fields… that strive to isolate cause–effect relations in complex systems in which relevant variables and phenomena can never be fully identified or characterized…”

The field Sarewitz is writing here about is biomedicine, but he could easily be describing development or humanitarian work. The fundamental problem, as he sees it, is that biases are not random but systemic: “if biases were random, then multiple studies ought to converge on truth [but] evidence is mounting that biases are not random.”

This claim is not new, of course. As the piece argues, systematic positive bias was identfied in clinical trials funded by the pharmaceutical industry back in the mid-1990s. More recently, reviews of so-called ‘landmark’ studies in fields such as cancer research has shown that positive results could only be replicated in a minority of cases.

However, these previous assessments tended to assume that the problem was not with science per se, but rather with those forces that sought to co-op it: industry, government, special interests, and so on. Reduce the influence of these interests, the argument went, and you would eradicate such biases.

But it is now emerging that there are some serious underlying problems within science itself. The cases are wide-ranging across biomedicine: “evidence of systematic positive bias [is] turning up in research ranging from basic to clinical, and on subjects ranging from genetic disease markers to testing of traditional Chinese medical practices.”

The two major faultlines, according to Sarewitz, are the methodological narrowness of the approaches employed to generate evidence, and the culture and incentives of scientists and science funders.

The first one is pertinent for readers of this blog. Researchers seek to reduce bias “through tightly controlled experimental investigations. In doing so, however, they are also moving farther away from the real-world complexity in which scientific results must be applied to solve problems.” Ironically, “the canonical tenets of ‘scientific excellence’” are threatening to undermine the whole enterprise. One rather shocking (for me, at least) example relates to the latest developments in research on mice, where a lot of resources and funds have been poured into the cloning of genetically identical animals, in order to enable fully controlled, replicable experiments and rigorous hypothesis-testing. Any sense of moral repugnance aside, perhaps the worst thing about this endeavour is that the findings of the research subsequently undertaken have turned out to be useless when applied in the real world.

Sarewitz also writes about the lack of incentives to ‘report negative results, replicate experiments or recognize inconsistencies, ambiguities and uncertainties’. There are also challenges around the various cultural and attitudinal positions taken toward science among funders, scientists, the media and the public at large. Sound familiar?

It should – such issues are not a problem for biomedicine alone:

[they are] likely to be prevalent in any field that seeks to predict the behaviour of complex systems — economics, ecology, environmental science, epidemiology and so on. The cracks will be there, they are just harder to spot because it is harder to test research results through direct technological applications… and straightforward indicators of desired outcomes…

Sarewitz closes with one potential solution, which may also be of relevance for work in development and humanitarian fields:

Scientists rightly extol the capacity of research to self-correct. But the lesson coming from biomedicine is that this self-correction depends… on the close ties between science and its application that allow society to push back against biased and useless results.

So what can we in the aid sector do about such bias, if indeed it is present in our work?

The first idea is the one that Sarewitz suggests: “societal push back”. Sadly, despite the rhetoric and growing practice of participation, the scope for Southern stakeholders – especially aid recipients – to ‘push back’ against useless results in development and humanitarian research is still severely limited. This doesn’t mean we should stop the effort, however, and perhaps new technologies and feedback processes can help us here.

The second strategy might be to address issues of the incentives and cultures which perpetuate such biases. But we seem to be far too concerned with developing country actor incentives and motivations to look at those in our own organisations. As one participant at a recent ODI event put it: “why do we always say that developing country leaders have mixed motives at best whereas the motives of donors [and other aid actors] are always considered impeccable?” We should find a way to ensure that these aid “physicians” first heal themselves.

The final course of action is to try to expand and adapt the concepts and models used in our work. This effort (of which this blog is one small part) is still very much a work-in-progress, but the growing interest  among researchers and practitioners should give us some small cause for hope. After all, the key to paradigm shifts in science – and in other fields – is not just logical argument and experimental proof. In the words of Thomas Kuhn:

as in political revolutions, so in paradigm choice—there is no standard higher than the assent of the relevant community.”

Read Full Post »

JP Morgan and the Price of Complexity

Posted in Accountability, Biology, Chaos, Economics, Evolution, Financial crisis, Innovation, Institutions, Knowledge and learning, Leadership, Networks, Organisations, Self organisation, Strategy, Technology, Trade on May 15, 2012 | 1 Comment »

This is the text of an article in the Washington Post by Dominic Basulto about last week’s events in the financial markets. Great stuff.

When news first broke Thursday that JPMorgan’s credit derivatives portfolio had sustained a loss of $2 billion, and potentially as much as $5 billion, on trades gone awry, there was an immediate call for greater regulatory oversight over banks’ high-risk trading activities. The message was clear: “If you’re going to be a bank, then you can’t play at the casino,” as the Post’s Ezra Klein writes. At the same time that the market was lopping off billions of dollars in shareholder value, JPMorgan was purging top executives responsible for the bungled trades and facing awkward questions about its public stance in favor self-regulation. If banks can’t regulate themselves, though, who can?

Inevitably, the answer to that question depends on whether you view the financial markets as complicated or complex. If the financial markets are merely complicated, traditional approaches to regulation can be effective: regulators can turn their attention to individual actors within the market and systematically make the requisite changes to restore the market to equilibrium. In a complex system, however, traditional approaches to regulation can be woefully inadequate — small changes may end up having outsized effects, while big changes may end up having little or no effect. In a complex system, you need to focus on the interactions between each of the participants as much as the condition of individual actors.

The trading screens of Wall Street are, if nothing else, the perfect example of how computers are able to mask the complexity of an underlying system by being able to reduce the real world into 1’s and 0’s. There is no shortage of algorithms, formulas, sophisticated risk management models and quantitative trading models promising to reduce complex financial market interactions to something that can be studied, adjusted and tweaked. In theory, regulators should be able to look at a few numbers, compare them to a few benchmarks, and suggest the necessary adjustments.

But it is rarely that easy.

There is a big difference between complicated and complex. In a classic example, an automobile is complicated, but a transportation system with human drivers is complex. Ultimately, you can fix an automobile by lifting up the hood and checking that everything is working properly, no matter how sophisticated the parts. You can only fix a transportation system, though, by understanding how each of the drivers interacts with each other and understanding the distributions of dynamic traffic patterns.

One of the most innovative areas of public policy, in fact, involves the intersection of complexity theory with regulatory policy. Complexity theory, which has been used to model complex systems ranging from ant colonies to climate change, has also been applied to financial regulation. Practitioners within the financial markets are well-versed with complexity theory and its cousin: chaos theory. One of the entities consistently singled out in the academic literature is the CDC (Center for Disease Control), which is held up as a role model for how to deal with complex systems. For example, using complexity theory, the CDC was able to suggest that — contrary to what you might see in movies like “Contagion” — restricting air travel would have little or no impact on stopping the SARS outbreak. As the CDC recognizes, you simply can’t regulate away diseases. You need to deal with them with in real-time as they appear and find the right levers to stop them. Most importantly, you need to be able to spot potential flare-ups before they occur and understand the emergent behaviors that lead to outbreaks.

Certainly, the types of events that we observe in the financial markets, such as “flash crashes” and billion-dollar Black Swan events in derivative markets, are reminiscent of complex systems behaviors where small changes lead to unimaginable consequences. While the Volcker Rule, which would keep banks from engaging in risky trading behavior, could be effective in the short-term in avoiding certain types of adverse market effects, it may not be as effective in dealing with the full range of market fluctuations in the long-term. Implicitly, we recognize the complexity of financial markets by ceding power to computers and algorithms to price financial instruments properly. Now, we need to recognize that this complexity also has important consequences for the way that we think about regulating these markets.

Read Full Post »

Official Views, Doing the Wrong Thing ‘Righter’, and the World Bank Presidency

Posted in Accountability, Evaluation, Influence, Innovation, Knowledge and learning, Leadership, Public Policy, Strategy on March 5, 2012 | 14 Comments »

There’s been a lot of interest in the imminent vacancy of World Bank President, with numerous suggestions of qualified individuals who should be on the list. This post looks at one particular aspect of the role which seems to be missing from most of this debate, and which should be high on the list of criteria for a successful future leader of the Bank.

I: The Official Views of ‘Development Churches’

David Ellerman, currently a visiting scholar at the University of California in Riverside, and World Bank Staffer for over a decade (where his roles including being senior advisor to the Chief Economist Joseph Stiglitz), is one of the most original and innovative thinkers in development. In 2000 he published a paper entitled ‘Must the World Bank have Official Views?’ in which he argued that the Bank spent a lot of time and effort determining its Official Views on particular development issues, and that this practice undermined in efforts in a number of ways.

Specifically:

  • it impedes the open contesting of adverse opinions that is so crucial to the advancement of knowledge
  • it impedes the Bank as a learning organization since the overturning of an older view is all the more difficult if it has been branded and enshrined as an Official View
  • it impedes client countries being intellectually in the driver’s seat as they will inevitably be encouraged in a multitude of ways to accept an opinion because it is an Official View

(The fact that Ellerman wrote and published this while still at the Bank gives some indication of his intellectual courage – I don’t know the backstory so cannot say what impact this had on him personally or professionally.)

Ellerman expanded on this in a subsequent paper for Development in Practice Journal. Through its focus on Official Views, the World Bank and other aid agencies become, in effect, ‘Development Churches’:

“…giving definitive ex cathedra ‘official views’ on the substantive and controversial questions of development. As with the dogmas of a Church, the brand name of the organisation is invested with its views….”

Ellerman argues that in the face of these Official Views, adverse opinions and critical reasoning tend to give way to authority, rules and bureaucratic reasoning shaped by the hierarchies within the organisation. Moreover, these Official Views “short-circuit” and bypass the active learning capability of national and local actors, and substitute the authority of external agencies in its place.

…Once an ‘Official View’ has been adopted, then to question it is to attack the agency itself and the value of its franchise. As a result, new learning at the expense of established Official Views is not encouraged…”

II: Moving Away From Doing the Wrong Thing Righter

The conclusions of a recent, still draft study on the World Bank’s efforts in participatory development indicates that the issues Ellerman highlighted are still an issue within the agency:

Project structures need to change to allow for flexible, long-term engagement. Projects need to be informed more seriously by carefully done political and social analyses, in addition to the usual economic analysis, so that both project design and expected outcomes can be adapted to deal with the specific challenges posed by country or regional context… Most importantly, there needs to be a tolerance for honest feedback to facilitate learning, instead of a tendency to rush to judgment coupled with a pervasive fear of failure. The complexity of development requires, if anything, a higher tolerance for failure. This requires a change in the mindset of management and clear incentives for project team leaders to investigate what does and does not work in their projects and to report on it (emphasis added)

This general phenomena is not unique to aid agencies, of course. The late great Russell Ackoff, a systems thinking pioneer, used to argue that almost every problem confronting our society is a result of the fact that our public policy makers are doing the wrong things and are trying to do them ‘righter’.

The righter we do the wrong thing, the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.

Back in 2000, David Ellerman suggested a way of overcoming the addiction to Official Views, which involves presenting the following message to client countries, and then acting upon it:

“…To the best of our accumulated experience (which we deem to call “knowledge”), here is what works best in countries like yours. Why don’t you study these principles together with their corroboration to date, take a look at these case studies, contact these people who designed those reforms, set up horizontal learning programs with those best practice cases, and try some experiments to see what
works in your own country? After carrying out this learning process on your own, you might call us back if you feel we could help…”

III: Implications for World Bank Presidency Candidates: A Simple Questionnaire

Building on all of this, we might view the current candidates for the World Bank Presidency in a different, and hopefully useful, light. We need someone who can take Ellerman’s message and Ackoff’s philosophy make them part and parcel of the way the organisation works.

To test candidates suitability in this regard, we might sensibly ask them, and those who know of them, the following yes / no questions (which should take no more than fifteen minutes of their time).

  1. Does the candidates track record indicate they have the ability to be a leader who facilitates as well as one who directs?
  2. Is the candidate able to let go of the notion of selling to, or controlling, others using a set of predefined strategies and results? (Can they effectively manage the uncertainty and ambiguity that ensues?)
  3. Does the candidate instinctively seek out challenges to their institution’s  ideas and policies, and see their leadership role as catalysing ‘mutual learning’? (Does the candidate routinely present their viewpoints as ‘permanently provisional’ and ‘up for debate’?)
  4. Can they respectfully but purposefully elicit the insights, creativity, and wisdom from others? (Can they do this even when others disagree with them?)
  5. Can they encourage multi-stakeholder dialogue and debate as a route to experimentation and innovation?
  6. Are they courageous enough to say ‘I was wrong’, and enable their learning process to be public, to allow others in their organisation and more widely to follow suit? (Are they willing to hear about, and learn from, failure, even in high-profile programmes?)
  7. Can their leadership help diverse groups and constituencies accomplish the results that they want? (Are they willing to share the credit for successes with others?)

I would tentatively suggest that if we have ‘yes’ responses against most of these questions for a given candidate, we could be looking at a genuinely interesting appointment.

If we have mostly ‘no’s, then we should all get ready for another term or two of Official Views, and all that goes with them.

Read Full Post »

South Africa’s democracy: Complexity theory in action

Posted in Accountability, Economics, Evolution, Innovation, Institutions, Leadership, Public Policy, Resilience, Strategy on December 16, 2011 | 4 Comments »

Dr Brian Levy is a Public Sector Governance Advisor at the World Bank, andused to head up the unit responsible implementing the Bank’s governance and anti-corruption strategy.

In this guest post, cross-posted from here, he explores the relevance of complexity theory insights for South Africa. A fascinating read.

The edge of chaos is the balance point where the components of a system never quite lock into place, and yet never quite dissolve into turbulence, either…The edge of chaos is the constantly shifting battle zone between stagnation and anarchy, the one place where a complex system can be spontaneous, adaptive and alive…”
M. Mitchell Waldrop, Complexity.

We human beings do not like uncertainty. We seek to understand what events portend, taking comfort in coming up with an answer. So whenever something new in South Africa makes headlines – and that’s been happening quite often recently, the news not always happy – people ask me, “What do you think? Is the political settlement falling apart?”. And always, the expectation is for a definitive answer, one way or the other.

Yet sometimes there is more wisdom, and more comfort to be taken, in acknowledging a more humbling truth – that which of many alternative futures (including ones we cannot imagine) will come to pass is unknowable, is a product of decisions and actions that have not yet been made. This understanding of change as something ‘emergent’, evolving, which can unfold in far-reaching yet ex ante unpredictable directions, is the key insight of ‘complexity theory’ – an insight which can offer a useful dose of humility to governance prognosticators.

In a previous post, I described South Africa’s abiding tension between democracy and inequality. As that post suggested, one of the results of that tension could be a rise of patronage, personalized decision-making as to how to share the ‘spoils’ of power. Indeed, the strains on the country’s constitutional democracy are visible in the drumbeat of daily headlines, as I found when I recently spent some time in the country. Consider the following:

  • Over the past year, the country’s estimable Public Protector, Thuli Madonsela, investigated and made public compelling evidence of corruption on the part of two sitting cabinet ministers, and the chief of police. For many months, President Zuma did nothing – until October 24th  2011, when he unexpectedly fired the cabinet ministers, suspended the police chief, and announced the appointment of a judicial commission to re-investigate long festering allegations of corruption (including against himself) in a weapons procurement deal.
  • Then there’s the populist demagoguery of the leader of the ruling African National Congress’s Youth League, Julius Malema, whose political star, and polarizing influence on South Africa’s political discourse, seemed to be rising – until a disciplinary committee of the ANC announced on November 10th, 2011 that it had found Malema guilty of bringing the ANC into disrepute, and suspended him from the party for five years.
  • There’s the China connection. Across Africa, a context is raging between China and Africa’s erstwhile Western colonial (and post-colonial) hegemonies – for hearts, minds, natural resources, and business more generally. With its sophisticated institutions, and proud transition to constitutional democracy, it might be thought that South Africa would be unambiguously in the Western camp. So against that backdrop, the failure to grant the global human rights icon, the Dalai Lama, a visa in time to attend the October 2011 80th birthday celebration of his friend, Archbishop Desmond Tutu, is startling – and can best be explained by a desire to curry favor with the Chinese.
  • Finally, there’s the on-again, off-again – but then on again – Protection of Information Bill which (after numerous delays, and promises of consultation before finalization) was rushed through South Africa’s parliament on 22nd November 2011. Passage of the bill was seen as a catastrophe by many in South Africa’s civil rights community. Indeed, it signals a startling turn away from a post-apartheid commitment to openness. But no less an observer than The Economist, noting that the bill had been ‘vastly improved’ over earlier versions, suggested that some of the doomsday criticism ‘may be over the top’. And, in light of the ANC’s hitherto ironclad hold on the loyalties of its core constituencies, it is surely no small matter that the Congress of South African Trade Unions (a stalwart part of the ANC alliance) has threatened to take the bill to the Constitutional Court if it is not amended to incorporate a ‘public interest’ exception for whistleblowers.

In the face of all of these cross-currents, any claims of certainty as to where this interplay of forces will lead is surely illusory. Each of them could turn out to be a salutary example of checks and balances in action – of how democratic institutions can keep a country on track, even in the face of pressures to the contrary. But they could also be portents of a future that will become increasingly challenging for the country’s constitutional order.

But in South Africa’s case, and surely elsewhere, too.   perhaps the lack of certainty is a source of comfort. Knowing that the future is not preordained can free us from endless preoccupation with what is inherently unknowable, enabling us instead to direct energy towards action, in service of a hopeful future that we can, perhaps, yet help to create.

Read Full Post »

What are leaders really for?

Posted in Accountability, Campaigns, Conflict and peace building, Evolution, Financial crisis, Innovation, Knowledge and learning, Leadership, Networks, Self organisation on December 7, 2011 | 5 Comments »

A continuing theme on this blog has been the issue of leadership. Many reports and studies call for it, reforms are seen as impossible without it, critical challenges will not be met without it, and we are all ready to point out the lack of it (in others, at least).

Despite the fact that leadership is one of the most researched topics in management literature (or perhaps because of this fact) , our understanding of leadership remains vague and ambiguous.

A blog on HBR by Duncan Watts of Yahoo’s Human Social Dynamics research group eloquently explores the concept and ideas with reference to recent social movements.

His argument resonates strongly with ideas of complex adaptive leadership explored in a previous post here.

The Occupy Wall Street movement has both perplexed and frustrated observers and analysts by its persistent refusal to nominate an identifiable leadership who can in turn articulate a coherent agenda. What is the point, these critics wonder, of a movement that can’t figure out where it’s trying to go, and how can it get there without anyone to lead it?

It’s a reasonable question, but it says at least as much about what we want from our social movements as it does about the way movements actually succeed.

Typically, the way we think of social change is some variant of the “great man” theory of history: that remarkable events are driven by correspondingly remarkable individuals whose vision and leadership inspire and coordinate the actions of the many. Sometimes these individuals occupy traditional roles of leadership, like presidents, CEOs, or generals, while at other times they emerge from the rank and file; but regardless of where they come from, their presence is necessary for real social change to begin. As Margaret Meade is supposed to have said: “Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has.”

It’s an inspiring idea, but over 100 years ago in his early classic of social psychology, “The Crowd,” the French social critic Gustave LeBon, argued that the role of the leader was more subtle and indirect. According to LeBon, it was the crowd, not the princes and generals, that had become the driving force of social change. Leaders still mattered, but it wasn’t because they themselves put their shoulders to the wheel of history; rather it was because they were quick to recognize the forces at work and adept at placing themselves in the forefront.

Even before LeBon, no less an observer of history than Tolstoy presented an even more jaundiced view of the great man theory. In a celebrated essay on Tolstoy’s War and Peace, the philosopher Isaiah Berlin summed up Tolstoy’s central insight this way: “the higher the soldiers or statesmen are in the pyramid of authority, the farther they must be from its base, which consists of those ordinary men and women whose lives are the actual stuff of history; and, consequently, the smaller the effect of the words and acts of such remote personages, despite all their theoretical authority, upon that history.” According to Tolstoy, in other words, the accounts of historians are borderline fabrications, glossing over the vast majority of what actually happens in favor of a convenient storyline focused on the skill and leadership of the great generals.

Thinkers like Le Bon and Tolstoy and Berlin therefore lead us to a radically alternative hypothesis of social change: that successful movements succeed for reasons other than the presence of a great leader, who is as much a consequence of the movement’s success as its cause. Explanations of historically important events that focus on the actions of a special few therefore misunderstand their true causes, which are invariably complex and often depend on the actions of a great many individuals whose names are lost to history.

Interestingly, in the natural world we don’t find this sort of explanation controversial. When we hear that a raging forest fire has consumed millions of acres of California forest, we don’t assume that there was anything special about the initial spark. Quite to the contrary, we understand that in context of the large-scale environmental conditions — prolonged drought, a buildup of flammable undergrowth, strong winds, rugged terrain, and on so — that truly drive fires, the nature of the spark itself is close to irrelevant.

Yet when it comes to the social equivalent of the forest fire, we do in effect insist that there must have been something special about the spark that started it. Because our experience tells us that leadership matters in small groups such as Army platoons or start-up companies, we assume that it matters in the same way for the very largest groups as well. Thus when we witness some successful movement or organization, it seems obvious to us that whoever the leader is, his or her particular combination of personality, vision, and leadership style must have supplied the critical X factor, where the larger and more successful the movement, the more important the leader will appear.

By refusing to name a leader, Occupy Wall Street presents a challenge to this view. With no one figure to credit or blame, with no face to put on a sprawling inchoate movement, and with no hierarchy of power, we simply don’t know how to process what “it” is, and therefore how to think about it. And because this absence of a familiar personality-centric narrative makes us uncomfortable, we are tempted to reject the whole thing as somehow not real. Or instead, we insist that in order to be taken seriously, the movement must first change to reflect what we expect from serious organizations — namely a charismatic leader to whom we can attribute everything.

In the case of Occupy Wall Street, we will probably get our wish, for two reasons. First, if OWS grows large enough to deliver any lasting social change, some hierarchy will become necessary in order to coordinate its increasingly diverse activities; and a hierarchy by nature requires a leader. And second, precisely because the outside world wants a leader — to negotiate with, to hold responsible, and ultimately to lionize — the temptation to be that person will eventually prove irresistible.

Leaders, in other words, are necessary, but not because they are the source of social change. Rather their real function is to occupy the role that allows the rest of us to make sense of what is happening — just as Tolstoy suspected. For better and worse, telling stories is how we make sense of the world, and it’s hard to tell a story without focal actors around which to center the action.

But as we witness a succession of popular movements, from the Arab Spring to Occupy Wall Street, we can at least pause to appreciate the real story, which is the remarkable phenomenon of a great many ordinary individuals coming together to change the world.

Read Full Post »

The Complexity of Scaling Up

Posted in Accountability, Evaluation, Evolution, Healthcare, Innovation, Leadership, Malaria, Public Policy, Reports and Studies, Strategy on October 3, 2011 | 4 Comments »

Despite increased prominence and funding of global health initiatives, attempts to scale up health services in developing countries are failing, with serious implications for achieving the Millennium Development Goals. A new paper argues that a key first step is to get a more realistic understanding of health systems, using the lens of complex adaptive systems.

Much ongoing work in development and humanitarian aid is based on the idea  of ‘scaling up’ effective solutions. Healthcare is one of the areas where this idea has played a central role – from WHO’s Health for All in the 1960s to UNICEF’s child healthcare programmes, from rolling out HIV-AIDS, malaria and TB treatments to the package of interventions delivered to achieve the Millennium Development Goal on health.

However, despite the fact there are many cost-effective solutions to health problems faced in developing countries, many agencies are still frustrated in their attempts to deliver them at scale. This may be because of a widepread failure to understand the nature of health systems.

Melissa Leach, director of the superb STEPS Centre, has described health systems as:

complex systems made up of networks of many heterogeneous components that interact non-linearly. While pathways of change can be shaped by governance and are influenced by path dependencies, they are not entirely controllable or predictable; there will always be uncertainties and unintended consequences and new ‘emergent’ interactions and behaviours.’

If we accept this eminently sensible description, then it is little wonder that scaling up efforts continue to be frustrated. The paper by Ligia Paina and David H Peters, published in Health Policy and Planning in August, argues that there is a drastic need for a shift in thinking:

…from the current models around scaling up health services, which revolve around linear, predictable processes, to models that embrace uncertainty, non-linear processes, the uniqueness of local context and emergent characteristics.”

Their argument is supported by the fact that existing assumptions about the nature and hoped-for successes of scaling up have led to a lot of disappointments. Moreover, these efforts ‘offer little insight on how to scale up effective interventions in the future.’

The paper explores 5 concepts of complexity science, illustrated below.


All of these ideas carry relevant lessons for the design, planning, implementation and evaluation of health policy and programmes. As the authors conclude:

The implications include paying more attention to local context, incentives and institutions, as well as anticipating certain types of unintended consequences that can undermine scaling up efforts, and developing and implementing programmes that engage key actors through transparent use of data for ongoing problem-solving and adaptation.”

The authors close with a proposal that future efforts to scale up should adapt and apply complex systems models and methodologies which have been used in other fields but which remain underused in public health. These include network scinece approaches, modelling techniques, and tools to better understand systems dynamics.

The potential benefits are clearly stated:

This can help policy makers, planners, implementers and researchers to explore different and innovative approaches for reaching populations in need with effective, equitable and efficient health services.”

These ideas are already being applied in practice, The authors are involved in a capacity strengthening programme on complexity and health systems in China. Separately, the WHO has published a guide to using systems analysis in health systems strengthening, which builds on a number of the concept described above.

These are all fascinating developments, and suggest that health may be a key area where the ideas from complexity science can prove of tangible value for development and humanitarian work.

Interested readers can hear an podcast about the article here, with David Peters talking about his ideas and experiences (and me saying a few words about the history of complexity science and the relevance for health efforts.) David has also blogged about it here.

Previous Aid on the Edge posts relevant to this topic include MDGS and theories of change, Scan HIV-AIDS Globally, Reinvent Locally and How do you solve a problem like malaria?.

Read Full Post »

Older Posts »

Follow

Get every new post delivered to your Inbox.

Join 2,490 other followers