Impact assessment of STEM initiatives in improving educational outcomes

Abstract

Science, technology, engineering and mathematics (STEM) skills are globally valued. In United Kingdom, serious concerns have long been raised over the apparently insufficient number of young people studying science and maths beyond compulsory education. A range of STEM schemes have been introduced and sustained for over a decade now to raise attainment and improve attitudes of students in school towards pursuing STEM subjects and careers. These schemes call for huge investments of time, money, and resources. Over the same period, government reports have pointed out the pressing need for large scale evaluations to understand what works in public policy including education. This is important for accountability and to achieve better results by building on the best schemes for similar or reduced investments.

Increasing and widening participation in STEM are clearly priorities for UK’s education policy. However, in the absence of proper evaluations the impact of spending on STEM schemes on raising attainment or improving participation remains unclear. Addressing this gap in literature, using official datasets in the form of National Pupil Database (NPD), this project evaluates the impact of STEM enrichment and enhancement activities on all pupils, disadvantaged pupils and schools with a large share of such pupils. A part of this research project tries to understand the reasons linked to underachievement of disadvantaged pupils in school science and maths through a systematic review.

To ensure comparability across evaluations, the public sector guidance for evaluation issued by the National STEM Centre was followed. Using a prospective longitudinal (2007-2014) research design, a 1000 intervention secondary schools and 80,000 students exposed to STEM interventions were followed-up from the beginning of key stage-3 to A-levels. The study uses various deprivation measures such as eligibility for free school meals (FSM), speaking English as an additional language (EAL) and ethnic minority status. The outcome measures considered are school GCSE performances in science and maths, individual pupil attainment in GCSE science and maths, and continued post-16 participation in STEM subjects.

Correlation-regression approaches are used and a range of effect sizes have been calculated to estimate the impact. Results show overall science and maths results have improved for schools, students and disadvantaged pupils (since 2007). However, this success cannot be attributed to STEM enrichment and enhancement activities, because the improvements are not peculiar to schools known to have been involved in STEM interventions. Synthesising 771 research reports, the systematic review concludes that a range of individual, social, family and school related factors interact to hold back a child from realising their full academic potential. Recommendations from this evaluation research project should be of particular interest to policy makers, schools, educators, STEM activity providers and anybody working towards improvement of the learning trajectories of disadvantaged students.