An investigation into the oral English language proficiency gain of pupils taught by native English-speaking teachers in Hong Kong secondary schools

Abstract

Purpose This study examines the extent to which Native English-speaking Teachers (NETs) have an impact on the oral English language proficiency gain of pupils taught in secondary schools in Hong Kong i.e. the so-called 'NET effect'. The principal aim was to determine whether the oral proficiency gain of subjects involved in this study was greater in students taught by NET teachers than it was in students taught by local teachers. Through the examination of Time one and Time two oral assessment data, this study also sets out to investigate the nature and strength of other predictor variables for the outcome variable 'Time two oral assessment'. Through a number of different statistical modelling techniques this study also sought to establish the model that would account for or 'explain' as much variance as possible between the Time one and Time two assessment scores. Procedures A randomised, stratified sample of secondary schools that was representative of the whole population secondary students in Hong Kong who are studying English was generated. From this sample, one thousand four hundred and twenty four students from forms one, three and four were selected and an English language oral proficiency test, specifically developed for this study was administered as a pre and post test. The period of data collection was a two year period, from the beginning of the 1998-1999 academic year to the end of the 1999-2000 academic year. The oral proficiency assessment instrument was designed, and piloted by a small team of trained assessors, and a standardised procedure was established for conducting the assessments. Hong Kong NET and local teachers were trained in the procedures and use of materials and techniques required to administer the assessments in specifically dedicated language assessment workshops. The assessments were then administered by the trained group of teachers who taped all of the interviews to allow monitoring to take place and to provide a data source for a second stage interview analysis (not covered in this thesis).The resulting pre-test and post-test data was then analysed through the use of a number of statistical techniques. In the first instance, a descriptive analysis was conducted in order to satisfy the assumptions on which traditional statistical analysis is based. The data analysis then proceeded with a number of scaling processes and was finally analysed to determine whether or not any significant 'NET effect' had been detected. In addition, the analysis also considered whether any of the other variables could be considered good predictors of the final post-test score. Major Findings Analysis of data produced from the Time one and Time two oral assessments revealed a number of important findings. Students did make significant oral English language proficiency gain as measured by the specially developed instrument. This gain was significant regardless of whether the students' results were analysed by whole sample or by separate year/age group. An analysis of means revealed that on average, the mean scores of students attending EMI schools were significantly higher than those attending CMI schools indicating that the medium of instruction is potentially a strong predictor of the Time two assessment score. In the post test analysis of means, students taught by NETs performed better than those taught by 'both' [NETs and local teachers] and in general, students taught by local teachers also performed better than those taught by 'both'. There was little difference between the scores of students taught by NETs and local teachers. The banding of the schools was also found to be an important predictor variable, with the average scores of students in high band schools significantly higher than their peers in medium and low band schools. Multiple regression analysis also revealed some important findings. When the modelling was conducted on the whole sample, the medium of instruction, the school level and NET teacher were all found to be significant predictor variables although in the case of the latter, the effect was small. When modelled by separate form/age group, similar results were found with Form one and with Form three students and again the medium of instruction, the school level and NET teacher were significant predictor variables although in the case of NET teacher, the effect was again small. The findings of this thesis suggest that in terms of measuring value-added between Time one and Time two, there are indeed strong predictor variables such as medium of instruction, school level and student level. However, in trying to evaluate the contribution of NETs to students' oral English language proficiency gain over a two-year period, there is some evidence of a so-called 'NET effect' although this is rather weak, suggesting that more research is required to investigate this question more thoroughly.