Study on Counterfactual Explanation for Practical Implementation

Reference No.	2024a032
Type/Category	Grant for Young Researchers and Students-Short-term Joint Research
Title of Research Project	Study on Counterfactual Explanation for Practical Implementation
Principal Investigator	Ken Kobayashi（School of Engineering, Tokyo Institute of Technology・Assistant Professor）
Research Period	September 24,2024. - September 27,2024. December 16,2024. - December 16,2024.
Keyword(s) of Research Fields	explainable machine learning, counterfactual explanation, mathematical optimization
Abstract for Research Report	Through the remarkable developments of modern machine learning methods such as deep learning, machine learning models have been applied to various critical decision-making in our society, for example, financial credit screening, medical diagnosis, and judicial decisions. However, prediction models such as neural networks have complex input-output relationships that are difficult for humans to interpret. Therefore, it is also important to provide additional information about the prediction results to improve the reliability of machine learning-based predictions. In this study, we focus on Counterfactual Explanation (CE) as one of the methods to improve the explainability of machine learning models. CE is a post-hoc explanation method that provides a perturbation vector of input to change the undesired prediction result given by the machine-learning model. Since the perturbation vector is viewed as an "action" to obtain the desired prediction results, CE can provide users with a more constructive explanation of the prediction results with CE. However, current CE methods ignore several requirements that should be met in practical use. First, machine learning models do not always have feasible actions. For example, if the prediction of a machine learning model relies on an unchangeable feature such as gender, there may be no action that the user can execute. Also, in real-world operations, machine learning models are regularly retrained, so the machine learning models have uncertainties that come from the retraining process. Therefore, even if the proposed action is executed, the desired output may not always be obtained from the retrained machine learning model. With these backgrounds, this research aims to develop practical CE methods that are applicable to more realistic situations. Specifically, we will proceed with the following studies: 1. Develop a learning algorithm to guarantee the feasibility of actions; 2. Develop a CE method that is robust to model uncertainty. We will develop a fundamental framework for highly interpretable and data-driven decision-making systems through these studies.
Organizing Committee Members (Workshop) Participants (Short-term Joint Usage)	Kentaro Kanamori（Fujitsu Limited・Researcher） Takuya Takagi（Fujitsu Limited・Principal Researcher） Yuichi Ike（Institute of Mathematics for Industry, Kyushu University・Associate Professor）