Semantic-Based Approaches for Automating Feature Engineering and Enhancing Machine Learning Interpretability

Soutenue par Mohamed BOUADI, au LIPADE de l’université de Paris Cité le 25-11-2024. Co-Encadrée par Salima BENBERNOU et Mourad OUZIRI, en partenariat avec Arta ALAVI de SAP France.
J’ai pu apprécier la grande qualité du manuscrit et de la présentation de Mohamed. Les contributions sont nombreuses et de qualité. [theses.fr]

Tremendous amount of data are being generated and saved in many complex engineering and social systems every day. Machine learning (ML) offers powerful tools to analyze this vast amount of data, helping organizations and individuals to make more informed decisions. The success of machine learning is often attributed to the expertise of data scientists, particularly in the area of feature engineering – a critical and time-consuming component of the machine learning workflow. To reduce the workload of data scientists, automated machine learning (AutoML) has received an increasing interest over the past decade. How- ever, recent studies have shown that feature engineering remains a significant bottleneck in the data science workflow, due to its labor-intensive nature and the need for deep understanding of the domain knowledge. Moreover, as machine learning systems become more prevalent, the need for interpretability becomes increasingly important, especially among domain experts who must understand the reasoning behind the model’s predictions to trust the outcomes. Therefore, automating the feature engineering process while simultaneously ensuring transparency and interpretability of the resulting models is crucial for the widespread adoption of machine learning across diverse applications.
This PhD thesis investigates the novel application of semantic web technologies to automate feature engineering and enhancing machine learning models’ interpretability. Specifically, We leverage knowledge graphs with reasoning mechanisms to generate meaningful features, enhancing both the performance and interpretability of the learning models, leading to more trustworthy and explainable AI systems.
First, we started by formalizing feature interpretability using description logics. Subsequently, we defined the feature engineering task as a Markov Decision Process and pro- posed KRAFT, a novel automated approach that combines deep reinforcement learning with symbolic reasoning to guide the generation of interpretable features.
Building on this, we introduced a novel metric for assessing feature interpretability, leveraging the semantics embedded in knowledge graphs. Alongside this, we developed SMART, a semantic-guided two-step approach for generating interpretable features. The first step infers domain-specific features using a reasoning algorithm, while the second employs a Deep Q-Network policy to explore and discover new, relevant features, effectively balancing the trade-off between model accuracy and feature interpretability.
Following this, we explored the potential of large language models (LLMs) in feature engineering, presenting ReaGen, a framework that combines the use of knowledge graphs with large language models to generate interpretable features, offering human-like explanations in the process.
In this thesis, we validated our approaches through extensive experiments, assessing both performance and interpretability, and comparing our methods against state-of-the-art approaches to demonstrate their effectiveness.