Preliminary evaluation of schemes for predicting use satisfaction with the ability of system to meet stated objectives

In software engineering literature two most commonly investigated targets for prediction are development effort and software quality. This study follows the methodological advances of these studies but focuses on predicting user satisfaction in software project. Specific outcome variable investigated in prediction is user satisfaction with the ability of system to meet stated objectives (MSO). A total number of 288 prediction schemes have been evaluated in the ability to predict MSO. These schemes have been built as different combinations of their components, i.e. feature pre-selection, elimination of missing values, automated feature selection, and a classifier. Two best performing schemes achieved the accuracy measured as Matthews correlation coefficient of 0.71 in test subset. These schemes involved W-LMT and W-SimpleLogistic classifiers. Significant differences have been observed between different classifiers and selected other components, depending on the dataset (validation or test). Discussed results may serve as guidelines to design a scheme to predict user satisfaction.