Evaluating training effectiveness in the malaysian public service

Abstract

The National Institute of Public Administration (INTAN) is the main training institute for the Malaysian Public Service. It plays an important role in the development of the human resources in the Malaysian public sector. However, the current method of the evaluation of the training programmes are carried out at the reaction level of the Kirkpatrick's model of evaluation (Kirkpatrick, 1967), giving very little indication of the effectiveness of the training programmes. The main purpose of this study thus is to develop a tool to measure learning, which would indicate effectiveness by examining whether there have been any changes in the level of knowledge, skills, or attitude of the training participants. Data from a total of 760 training participants are used in this study, and several different statistical analyses are carried out, namely reliability tests, structural equation modeling (SEM), principal variables, tests of differences, and analysis of covariance (ANCOVA). Besides the main Learning Questionnaire, the Course Experience Questionnaire (CEQ) (Ramsden, 1987) and the General Health Questionnaire (GHQ) are also used. Findings indicate that the LQ needs to be modified. Model fits of the other two questionnaires are also found to be not very good. Work in this thesis continues with methods of comparing models graphically, based on the eigenstructures of the covariance matrices. The Learning Model which forms the basis of the Learning Questionnaire is applicable to other training institutes with appropriate modifications, while the statistical method of comparing eigenstructures proposed here is applicable to the general multivariate data analysis.