Evaluating teachers’ use of research evidence to improve students’ literacy outcomes

Abstract

Policymakers, practitioners, and researchers from various sectors have long been curious about bridging the gap between research and practice, seeking to incorporate research evidence into real-life applications and policymaking across diverse domains. In education, it appears both logical and desirable to apply the best available research evidence to enhance teaching quality and effectiveness.

However, using research evidence in daily teaching in schools presents considerable challenges. There is growing but still incomplete knowledge of ‘what works’ in classroom interaction, and there is even less secure evidence on the most promising approaches to encourage and facilitate teachers’ use of research evidence. Moreover, even the feasibility of asking teachers to engage with and utilise research evidence is uncertain, let alone determining how to maintain high-quality implementation over time. Based on these research gaps, this study is a further exploration to understand the use of research evidence in teaching practice.

The study starts by examining teachers’ access to research information, their views of the usefulness of research findings to improve teaching and the strongest predictors of these views. These are investigated with TALIS data and the NFER Survey of Teachers 2010. The secondary data analysis of TALIS revealed that reading professional literature was a common way for teachers worldwide to access educational research. Over the past decade, there has been little change in teachers’ engagement with professional literature, research conferences, or conducting research, suggesting that analysing the Survey of Teachers 2010 remained meaningful. Findings from the Survey of Teachers suggested that using research findings was not a popular approach to improve teaching practice. Confidence in understanding and using research findings emerged as the strongest predictor of evidence-use. The data also highlighted the role of supportive school environments and teacher peer networks in facilitating the transition from accessing research to integrating it into teaching.

Next, the study summarises the best approach to disseminating and engaging teachers with research evidence via a structured review. 2,257 existing studies were screened, and 13 studies were found that evaluated the impact of teachers’ evidence use on students’ literacy outcomes. Despite the limited research on this topic, the structured review identified that having researchers engage in transferring and disseminating modified research evidence was the most tested and promising approach. Research evidence that is simplified to easy-understanding versions and activities, such as workshops and regular follow-ups, has the potential to encourage teachers’ use of research evidence and promote learning outcomes. Although the observed impact was modest, most high-quality studies highlight the cost-effectiveness of these approaches.

With the foundation of existing evidence on teachers’ use of research evidence, the current study then estimates the impact of a use-of-research-evidence intervention in the form of engaged dissemination of modified research evidence. The fieldwork was completed in China to fill key gaps in the exploration of using evidence in educational research so far in China. A pilot randomised controlled trial (RCT) was conducted in May 2023, involving 2,022 students and 46 teachers across two primary schools, to test the feasibility of the intervention materials. Lessons from the pilot highlighted the need for more detailed instructions, improved communication, and example lessons to support teachers in incorporating research evidence. Additionally, insights gained during the pilot led to better data collection methods to maintain accuracy and efficiency.

The main RCT expanded to 6,904 students and 154 teachers across four primary schools in the same region, with an additional school participating via a quasi-experimental approach. For the impact analysis, the study primarily focused on data from the four RCT schools, as the quasi-experiment school lacked an appropriate comparison group and its data were incomplete. An impact analysis including the quasi-experiment school can be found in the Appendix. Classroom observations from the quasi-experiment school were included in the descriptive analysis of the investigation process.

The impact analysis for the four RCT schools revealed a small positive effect size of +0.07 on students’ Chinese literacy scores. The use-of-research-evidence intervention also potentially fostered students’ positive attitudes towards reading. These positive effects were particularly pronounced among students with rural Hukou and those from migrant backgrounds. A slight reduction in homework time for Chinese literacy was also observed (-0.06). The use-of-research-evidence intervention influenced teachers’ decision-making and research engagement. It enhanced their awareness of the usefulness of research evidence (+0.26) and improved their ability to interpret research findings (+0.15). Although the impact on students’ Chinese literacy outcomes was marginal, the study underscores the feasibility of incorporating high-quality research evidence into teaching practice. It demonstrates that such interventions can enhance teachers’ awareness and skills, forming a foundation for sustainable, long-term implementation.

The process evaluation explored the process of using research evidence and teachers’ implementation quality based on the Complier Average Causal Effect (CACE) analysis, four headteacher interviews, five school research leads interviews, informal teacher conversations, and forty-four classroom observations. The role of teacher compliance in realising the intervention’s potential impact is reflected a CACE of +0.22 on students’ Chinese test scores among those whose teachers adhered to the implementation protocol. Factors associated with compliance included school-level planning, teachers’ confidence and skills, and collaboration among teachers. Classroom observations suggested that the adaptation of research evidence evolved iteratively, with initial attempts often being procedural and inaccurate. Teachers required time and effort to refine their approaches, incorporating classroom feedback and tailoring research-based materials to better meet their own students’ needs.

An implementation quality cycle was discussed at the school, individual teacher, and teacher interactive levels. At the school level, effective planning, monitoring, and clear communication with teachers were key to ensuring exposure to research findings. At the teacher’s individual level, skills and confidence in using research evidence were relevant to promote active engagement with research evidence. At the teacher interactive level, teacher-peer collaboration may help them identify and adjust ineffective practices based on research findings.

This study offers valuable insights into the impact of teachers’ use of research evidence on students’ literacy outcomes. It highlights the feasibility and sustainability of using research evidence in teaching practice through a use-of-research-evidence intervention. It also identifies essential factors for the high-quality implementation of evidence-informed teaching strategies, offering advice to further promote the effective use of research evidence in educational practice. Further research should focus on generating high-quality evidence on ‘what works’ in getting evidence into use with more open-access materials to facilitate educators’ use. Additional studies are needed to explore the impact of using research evidence across other subjects, with more diverse samples, broader age groups, and varied evidence-use approaches.