Identification and evaluation of factors affecting use of knowledge-based systems in a manufacturing environment

Abstract

A large amount of work has been carried out in the field of developing knowledge-based systems from initial analysis of the domain task, through formalisation and computerisation of knowledge to a completed knowledge-based system. However, issues relating to the use of such systems appear not to have been so clearly identified. It is essential to pay detailed attention to all of the human as well as technological issues which affect the practical use of such systems. The factors that influence the use of a knowledge-based system need to be identified to ensure that any systems developed will in fact be used by the intended end-users. In this thesis we propose a model for effective utilisation of knowledge-based systems. We will discuss how this model has been validated, then used as a basis for the identification of factors that affect system use. We will describe how we select and evaluate factors which we believe have a significant impact on the use of a system. We will present a set of initial findings based on experimental work we have performed as to the most significant factors. A set of conclusions are drawn on the approach we have adopted, the results we have obtained and the success of this work. We have identified a four phase model of system use, namely, acquisition, handover, operation and maintenance based on current literature. The model depicts the relationship between roles, functions and entities. By validation of the model, we have identified an initial set of 55 parameters that impact the effective use of a system. We have selected a subset of these parameters (those which we believe have significant impact and those which have less impact on the utilisation of knowledge-based systems) which we are able to control in order to evaluate a set of 17 hypotheses. The important parameters were: Role of system, Familiarity with system. Functionality, Robustness of system, Breadth of knowledge. Depth of knowledge, Method of displaying information (HCI), and Method of selecting options (HCI). The less important parameters were: Familiarity with domain tasks. User role. Fit with user requirement. Provision of system help. Provision of explanation. Response time. Security features. Error reporting, and Maintenance procedure. The experiments we performed allowed a systematic examination of the degree to which each parameter we selected impacts system use. From the data we obtained we identified a number of key parameters and the impact they have on effective use of a system. Specifically, from our experimental work we have identified the following factors as having the greatest degree of impact on system use: Role of system. Breadth of knowledge. Depth of knowledge. Provision of explanation. Provision of system help. Method of displaying information (HCI), Method of selecting options (HCI), Functionality, and Maintenance procedures. We also identified areas where additional work is required to further investigate the factors that impact on the effective use of knowledge-based systems.