SPECIAL ISSUE: Evidence for Restorative Justice
20
make a difference? I return to that after first
balancing the narrative by pointing out that
not all literature reviews conclude that restorative justice is effective. Indeed restorative justice sceptics still abound. The most
recent important contribution of that kind
is by Weatherburn and Macadam.7 Weatherburn and Macadam do not consider my
own more wide ranging review of the literature,8 but begin their analysis by concluding
that many of the early studies have methodological limitations and that the earlier
reviews9 show only modest effects on reduced reoffending). No great disagreement there.
Having concluded that there is nothing up to 2007 to suggest that restorative
justice works very well, Weatherburn and
Macadam proceed to review studies since
2007. They found only eight of the fourteen studies from 2007 that passed their
tests of methodological adequacy reporting any statistically significant reduction
in reoffending.10 None of them concluded
that restorative justice made things significantly worse (a different result from earlier
reviews that concluded some interventions
had made things worse). If one added the
results of Weatherburn’s post-2006 studies
with the numbers from the earlier studies in
the meta-analyses of Latimer et al. (2001),
Bonta et al. (2006) and Sherman and Strang
(2007),11 the fundamental result would be
unchanged—a modest but statistically significant effect overall. That is, the pattern
of results in these studies from 2007 on is
a rather similar pattern to the earlier work.
Indeed, a higher proportion of these post2006 single studies are reporting a statistically significant effect and a lower proportion (zero) a counterproductive effect.
So I read Weatherburn and Macadam
as providing a broadly similar reading of
the facts on a narrower set of findings to
my own older more wide-ranging review.12
Weatherburn, Macadam, and I also share
some cynicism about meta-analysis in comparison with qualitative diagnosis of many
individual studies, which is why we review
literatures without doing a meta-analysis.
Perhaps I go even further than Weatherburn and Macadam in that regard, in that
I am prepared to interpret non-quantitative
data, such as that in Braithwaite and Gohar,13 as providing strong qualitative evidence that restorative justice can reduce
serious violence with high cost-effectiveness in the most difficult of conditions.
Those of us who see limits of a myopic
focus on meta-analysis of randomized controlled trials, as in the Cochrane and Campbell collaborations, must concede, however,
some important strengths to that approach.
One was revealed right at the beginning
when I first recruited Lawrence Sherman in
1993 to conduct an independent randomized controlled trial of the restorative jus-
tice innovations Terry O’Connell, John McDonald, David Moore, Peta Blood, and others were refining with me in Australia. Sherman asked what my theoretical predictions
would be about percentage impacts at different times of follow-up. It was twenty-one
years ago so I do not remember exactly the
numbers I proffered, though I am sure it
was a lower effect size than actually found
in Strang et al.14 What I remember is Sherman’s response—“If you only expect an impact as small as that, we will need to randomize many thousands of cases to deliver
the statistical power capable of detecting
such a small effect.” But of course that is
one way meta-analysis comes into its own
twenty years later. You can end up with a
situation decades on that is exactly the current state of play with the evidence on the
effectiveness of restorative justice. Many
studies are so methodologically flawed that
they should be simply dismissed; many useful studies show statistically insignificant
reductions of reoffending on sample sizes
too small to have the statistical power required; yet when these data sets are combined, the meta-analysis shows a modest statistically significant reduction of offending from the combined data sets. One
study at a time, the Strang et al. studies actually show a lower success rate for restorative justice than Weatherburn and Macadam when “vote counting” based on statistical significance is the approach adopted; it
is the combined data sets with their greater
statistical power that detects a significant
reduction of reoffending.15
Weatherburn and Macadam also implicitly agree with my16 view that the most important thing about restorative justice is whether it puts offenders (and victims) into followup rehabilitation programs that make things
better or worse.17 One of the many ways restorative justice can make things worse is
by putting young people into programs like
boot camps and scared straight programs
that worsen reoffending. The most [\ܝ[