Extrapolating Policy Effects - Hopes, Assumptions, and the Extrapolator's Bind

Abstract

Evidence-Based Policy is the movement according to which policy should be based on high-quality evidence for ‘what works’. A major problem in using evidence to inform policy is concerned with extrapolation, i.e. using evidence of policy effectiveness from a study population to learn something about the effects of a policy in a novel target population. This thesis provides a critical discussion of extrapolation in Evidence-Based Policy and aims to make general contributions to improving both the theory and practice of extrapolation. It proceeds in three parts. Part I provides a comprehensive analysis of extrapolation, including what different kinds of extrapolation there are, what makes some of them highly challenging, and what successful extrapolation is. Part II critically examines existing strategies for extrapolation proposed by philosophers, econometricians, and computer scientists. Emphasis is put on the empirical assumptions about similarities and differences between populations that these strategies involve, and it is argued that supporting these assumptions is often over-demanding. In particular, the knowledge about the target population required to underwrite an extrapolation is often so extensive that we can learn the effect of interest in the target based on this knowledge alone. This is problematic, as it can render the evidence from which one extrapolates irrelevant to an envisioned conclusion, thus undermining the success of an extrapolation. Detailed investigations are provided to highlight the conditions under which existing strategies fall prey to this problem. Building on this critical investigation, Part III makes several positive proposals for how to improve the theory and practice of extrapolation in EBP and evade the central problems that it faces.