From evaluation to meta-evaluation of engineers' training in the automotive industry

Abstract

This thesis presents and analyses the evaluation of an European wide training programme aimed at engineers working in a large multinational automotive company. The training programme is unique in that it was conceived to address particular operational concerns and involved a multicultural workforce from six European countries. The evaluation of the training, which extends from the pilot stages of the programme through to its full implementation, where Kirkpatrick's four level evaluation framework is used, is the company’s first large scale attempt at systematic training evaluation. The evaluation of the programme is typical in its approach as reflected in the wide body of literature, however the use of meta-evaluation to determine the overall value of the evaluation approach in a commercial context provides originality and the basis for establishing an alternative approach to evaluating vocational training. The main body of the thesis is presented in three parts. Part I provides a critical review of the literature relating to; learning and training; conceptualisations of evaluation; and measurement and evaluation methodology, to establish the foundation for the empirical study. Part II is a detailed analysis of the evaluand, the evaluation methodology employed, and the results and outcomes from the evaluation. Part III provides directions for training evaluation based on a meta-evaluation of the empirical study. The thesis draws conclusions with respect to the role of evaluation in organisational training. The evaluation of training is largely conceptualised in the literature as being concerned with the assessment of value or worth of training to an organisation, which is the prevailing paradigm of Kirkpatrick's training evaluation framework. From the evidence obtained through the empirical study with regard to utility, feasibility, propriety and accuracy, it is concluded that the role of evaluation should be directed towards maximising value or worth of training through the systematic assessment, feedback and optimisation of the identifiable parameters of the training process, with the outcomes of training forming part of an overall evaluation of training framework.