WHAT IS RIGHT FOR ‘THIS’ PATIENT? AUTONOMY, TRUST AND TRANSPARENCY IN THE ERA OF MEDICAL AI

In this paper we compare the capabilities and limitations of AI algorithms and human doctors in the context of treatment recommendations. Ethical concerns have been raised in the literature about transparency, accountability, and interpretability in relation to AI algorithms in the medical context. Patients’ autonomy is not likely to be respected if treatment recommendations are opaque or incomprehensible. Medical AI algorithms are programmed to prioritise predefined goals that may not align with patients’ preferences, potentially undermining their autonomy without their awareness. Yet human doctors often face similar challenges, such as reliance on clinical trial data with endpoints that may not be fully understood by patients. We conclude that there are opportunities now to integrate patient autonomy considerations into AI algorithms at the design stage, and that these goals and any relevant weightings are made explicit to end users, including both doctors and patients.
Keywords: Artificial intelligence. Autonomy Alignment. Medical ethics.