Image: Peaks through the clouds, pdvos Flickr.
Much of the governance and development literature looks at the role of institutions in securing progress. But cases where actors – whether leaders or coalitions – are working effectively against the odds are harder to identify. In a new paper we introduce an approach researchers can use to help uncover where such political agency might be at work, focusing on anti-corruption efforts.
Recently, researchers have identified ‘positive outliers’ as one way to study what works in low-quality governance environments. Research on such cases – also called ‘pockets of effectiveness’, examples of ‘positive deviance’, or ‘islands of integrity’ – highlights how institutions, reforms or individuals achieve developmental progress in incredibly challenging contexts. This research has had considerable impact: problem-driven iterative adaptation, for example, is one policy approach that has emerged.
However, the research often suffers from a methodological blind spot. In looking for cases to study, researchers and practitioners have relied on those that already have reputations as success stories. This is problematic, not just because ‘success’ may be misattributed or even misrepresented by charismatic development actors. It means that we miss hidden cases and can only learn from those that are relatively well known.
So how do we locate the positive outcomes that aren’t clearly attributable to a specific program or initiative? When positive development outcomes are unexpected, how do we ensure they don’t escape detection?
Our British Academy-funded anti-corruption research on ‘Islands of Integrity’ is inspired by work on positive outliers but has developed a new mixed-methods approach to case selection. In our pilot study, we looked for cases where bribery had unexpectedly – even unintentionally – been reduced. Then we went about discovering why.
To uncover hidden cases of bribery reduction, we first identified statistically significant positive outliers using a simple regression analysis. We examined sector-specific bribery rates from Transparency International’s Global Corruption Barometer to hone in on rates that had reduced far more than expected, given those in other sectors in the same country over the same period.
Second, we vetted 18 of these potential cases through a literature review and preliminary consultations with in-country experts to assess whether any should be excluded from further scrutiny. This enabled us to identify errors in the quantitative data that cause statistical tests to identify false-positives as outlying cases.
It also helped us to identify which cases involved reforms and which didn’t. We found, for instance, that bribery rates in schools in Sierra Leone and Liberia dropped significantly during the Ebola year – most likely the result of school closures rather than effective anti-corruption policy. Bribery also reduced significantly in the Mongolian land services sector between 2009 and 2013 – when Ulaanbaatar experienced a dramatic housing and land market slowdown.
Third, we used qualitative fieldwork to check two cases in detail – Uganda’s health sector and South Africa’s police – to see if the statistical method had indeed alerted us to previously unknown positive outliers. Over 11 weeks we visited hospitals and clinics, police stations and government offices and interviewed health care workers, patients, police, government bureaucrats, journalists and academics. This enabled us to examine whether bribery really had reduced in these sectors (spoiler alert: it had) and to identify and work towards understanding the complex, and often politically contentious, causes.
This methodology could be used in many other areas beyond anti-corruption work. It could also help identify surprising cases of subnational poverty reduction or service delivery progress, for example.
Using statistical analyses to identify unexpected improvements and then checking these cases through qualitative research can enable researchers to investigate the factors that lead to surprising developmental change much more fully.
And as we write up our pilot case studies, it strikes us that there may be another important reason not to rely on reputation alone for case study selection of positive outliers. Because we’ve not been primed to think of the cases as ‘successes’, it’s encouraged us to be critical of our findings, to see the failures that sit alongside the successes, and to begin to think through the (potential) negative consequences that might emerge.
