“Despite the apparently plausible and widespread belief to the contrary, the evidence that levels of attainment in schools in England have systematically improved over the last 30 years is unconvincing. Much of what is claimed as school improvement is illusory, and many of the most commonly advocated strategies for improvement are not robustly proven to work. Even the claims of school effectiveness research – that we can identify good schools and teachers, and the practices that make them good – seem not to stand up to critical scrutiny. Recent growth of interest in evidence-based practice and policy appears to offer a way forward; but the evidence from past attempts to implement evidence-based approaches is rather disappointing. Overall, an honest and critical appraisal of our experience of trying to improve education is that, despite the best intentions and huge investment, we have failed – so far – to achieve it.

Nevertheless, we have reason to be hopeful, provided we are willing to learn from our experience. Specifically, I will argue that we need to do four things: to be clear what kinds of learning we value; to evaluate, and measure properly, teaching quality; to invest in high-quality professional development; and to evaluate robustly the impact of changes we make”.

As a former teacher himself, Professor Coe is not implying criticism of teachers, he realises how hard they work to improve things in schools.  He argues that there is insufficient data to robustly conclude that standards have risen despite all the efforts to drive the system harder.  Given what data is available he draws a bleak conclusions that our good intentions have failed to produce results. He lists some of the weaknesses of school improvement research methodologies and has a figure listing 7 things you could do to ‘mistake school improvement’ and make it look as if your improvement project has worked.

One might think that the content is unnecessarily cynical, but I believe that it presents a real wake up call to the mass of flimsy research and poor attribution of causation and effectiveness that is out there.  For example, value added is not necessarily the same as effectiveness, and the usual suspects in lists of characteristics of ‘good’ schools are often poorly defined and the correlations ‘generally pretty low’.

Many strategies that schools claim to be doing that ‘work’ (for example assessment for learning) are prone to a lack of proper contextual evaluation and subject to problematic intervention. In other words, there is a poor understanding of how supporting factors in one school may be different in another; also, “we do not know how to get large groups of teachers and schools to implement these interventions in ways that are faithful, effective and sustainable” (p.xi).