On the impact of weighting informative features in the decision-making process

Abstract

The article serves as a methodological and educational note, viewing decision-making as a multi-stage process, encompassing phases of data preparation and normalization, weighing informative features, executing machine learning algorithms, and analyzing results. The work aims to show that assigning weights to informative features is an informal procedure that can significantly influence decision-making outcomes. Experiments were conducted on a test dataset using formal classification, clustering, and ranking algorithms with variations in the weight coefficients of informative features. It was assumed that changes in class labels, clusters, and ranks indicate changes in decision-making. It is visually demonstrated that decision-making based on data classification using discriminant function methods does not depend on the selection of weights for informative features. Conversely, decisions made by classification, clustering, and ranking algorithms that rely on calculating distances between objects are affected by the choice of weights and the data distribution in the feature space. Therefore, selecting machine learning algorithms and weights for informative features must be conducted thoughtfully, considering the context of solving a specific applied problem.