Exploring the ethical consequences of black-box medicine

Autonomy, responsibility and the prospects of explainable AI  

Artificial intelligence (AI) tools in medicine are increasingly opaque. This means that it becomes very hard – or simply impossible – to understand why an AI tool generated a certain outcome given the input it received. Moreover, the most promising techniques such as deep neural networks are the most opaque as well. This makes the ethical concerns related to the resulting black-box all the more pressing. For if a patient cannot understand why the AI tool proposes a certain therapy, how can she decide upon the adequacy of its proposal? In this project we will focus on the ethical consequences of black-box AI tools in medicine in terms of autonomy, informed consent, shared-decision making and responsibility.

Firstly, the literature is undecided with regard to the precise nature of the ethical problems arising from these black-box tools. Hence our first goal is to clarify how specific AI tools might problematize autonomy and related ethical concepts.

Secondly, the nascent scientific field of explainable artificial intelligence (XAI) develops techniques to foster (post-hoc) explanations for what happened in the black-box. Therefore we can ask which specific requirements these explanations should meet in order to solve the ethical problems we identified. In order to do this we will combine the bio-ethical literature on black-box AI, the literature in the philosophy of science on explainable AI and the corresponding scientific literature. Ultimately, we hope that our research will attribute to the development of a fine-grained ethical discussion about black-box AI in health care and constructively guide the development of the scientific field of explainable AI.