A Systematic Review of Trustworthy and Explainable Artificial Intelligence in Healthcare: Assessment of Quality, Bias Risk, and Data Fusion

  1. Albahri, A.S.
  2. Duhaim, Ali M.
  3. Fadhel, Mohammed A.
  4. Alnoor, Alhamzah
  5. Baqer, Noor S.
  6. Alzubaidi, Laith
  7. Albahri, O.S.
  8. Alamoodi, A.H.
  9. Bai, Jinshuai
  10. Salhi, Asma
  11. Santamaría, Jose
  12. Ouyang, Chun
  13. Gupta, Ashish
  14. Gu, Yuantong
  15. Deveci, Muhammet
Revista:
Information Fusion

ISSN: 1566-2535

Año de publicación: 2023

Tipo: Artículo

DOI: 10.1016/J.INFFUS.2023.03.008 GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Information Fusion

Objetivos de desarrollo sostenible

Resumen

In the last few years, the trend in health care of embracing artificial intelligence (AI) has dramatically changed the medical landscape. Medical centres have adopted AI applications to increase the accuracy of disease diagnosis and mitigate health risks. AI applications have changed rules and policies related to healthcare practice and work ethics. However, building trustworthy and explainable AI (XAI) in healthcare systems is still in its early stages. Specifically, the European Union has stated that AI must be human-centred and trustworthy, whereas in the healthcare sector, low methodological quality and high bias risk have become major concerns. This study endeavours to offer a systematic review of the trustworthiness and explainability of AI applications in healthcare, incorporating the assessment of quality, bias risk, and data fusion to supplement previous studies and provide more accurate and definitive findings. Likewise, 64 recent contributions on the trustworthiness of AI in healthcare from multiple databases (i.e., ScienceDirect, Scopus, Web of Science, and IEEE Xplore) were identified using a rigorous literature search method and selection criteria. The considered papers were categorised into a coherent and systematic classification including seven categories: explainable robotics, prediction, decision support, blockchain, transparency, digital health, and review. In this paper, we have presented a systematic and comprehensive analysis of earlier studies and opened the door to potential future studies by discussing in depth the challenges, motivations, and recommendations. In this study a systematic science mapping analysis in order to reorganise and summarise the results of earlier studies to address the issues of trustworthiness and objectivity was also performed. Moreover, this work has provided decisive evidence for the trustworthiness of AI in health care by presenting eight current state-of-the-art critical analyses regarding those more relevant research gaps. In addition, to the best of our knowledge, this study is the first to investigate the feasibility of utilising trustworthy and XAI applications in healthcare, by incorporating data fusion techniques and connecting various important pieces of information from available healthcare datasets and AI algorithms. The analysis of the revised contributions revealed crucial implications for academics and practitioners, and then potential methodological aspects to enhance the trustworthiness of AI applications in the medical sector were reviewed. Successively, the theoretical concept and current use of 17 XAI methods in health care were addressed. Finally, several objectives and guidelines were provided to policymakers to establish electronic health-care systems focused on achieving relevant features such as legitimacy, morality, and robustness. Several types of information fusion in healthcare were focused on in this study, including data, feature, image, decision, multimodal, hybrid, and temporal.

€ Ver financiación

Información de financiación

Financiadores

Referencias bibliográficas

  • Yu, (2018), Nat. Biomed. Eng., 2, pp. 719, 10.1038/s41551-018-0305-z
  • Albahri, (2021), J. Netw. Comput. Appl., 173, 10.1016/j.jnca.2020.102873
  • Rong, (2020), Engineering, 6, pp. 291, 10.1016/j.eng.2019.08.015
  • Amann, (2020), BMC Med. Inform. Decis. Mak., 20, pp. 1, 10.1186/s12911-020-01332-6
  • Hayden, (2014), Nature, 516, pp. 131, 10.1038/516131a
  • Santamaría, (2011), Comput. Vis. Image Underst., 115, pp. 1340, 10.1016/j.cviu.2011.05.006
  • Deo, (2015), Circulation, 132, pp. 1920, 10.1161/CIRCULATIONAHA.115.001593
  • Markus, (2021), J. Biomed. Inform., 113, 10.1016/j.jbi.2020.103655
  • Li, (2020), npj Digit. Med., 3, pp. 1, 10.1038/s41746-020-00318-y
  • Topol, (2019), Nat. Med., 25, pp. 44, 10.1038/s41591-018-0300-7
  • Commission, (2019), Eur. Comm., 9, pp. 1
  • Aria, (2017), J. Informetr., 11, pp. 959, 10.1016/j.joi.2017.08.007
  • Séroussi, (2020), Yearb. Med. Inform., 29, pp. 7, 10.1055/s-0040-1702029
  • Saheb, (2021), Comput. Biol. Med., 135, 10.1016/j.compbiomed.2021.104660
  • Muhammad, (2021), Inf. Fusion, 76, pp. 355, 10.1016/j.inffus.2021.06.007
  • Loh, (2022), Comput. Methods Programs Biomed., 226, 10.1016/j.cmpb.2022.107161
  • Yang, (2022), Inf. Fusion, 77, pp. 29, 10.1016/j.inffus.2021.07.016
  • Balagurunathan, (2021), Phys. Medica, 83, pp. 72, 10.1016/j.ejmp.2021.02.024
  • Rethlefsen, (2021), J. Med. Libr. Assoc., 109, pp. 174, 10.5195/jmla.2021.962
  • Holzinger, (2022), Inf. Fusion, 79, pp. 263, 10.1016/j.inffus.2021.10.007
  • Angerschmid, (2022), Machine Learning and Knowledge Extraction, 4, pp. 556, 10.3390/make4020026
  • Sohrabi, (2021), International Journal of Surgery, 88, 10.1016/j.ijsu.2021.105918
  • Khaw, (2022), Curr. Psychol., pp. 1
  • Setchi, (2020), Procedia Comput. Sci., 176, pp. 3057, 10.1016/j.procs.2020.09.198
  • Rostami, (2022), Informatics Med. Unlocked, 30, 10.1016/j.imu.2022.100941
  • Nicora, (2022), J. Biomed. Inform., 127, 10.1016/j.jbi.2022.103996
  • Z. Shi, W. Chen, S. Liang, W. Zuo, L. Yue, and S. Wang, “Deep Interpretable Mortality Model for Intensive Care Unit Risk Prediction,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11888 LNAI, no. 15th International Conference on Advanced Data Mining and Applications (ADMA). Jilin Univ, Minist Educ, Key Lab Symbol Computat & Knowledge Engn, Changchun 130012, Peoples R China, pp. 617–631, 2019. doi: 10.1007/978-3-030-35231-8_45.
  • A. Lucieri, M. N. Bajwa, A. Dengel, and S. Ahmed, “Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems,” arXiv Prepr. arXiv2011.13169, Nov. 2020.
  • P. Washington et al., “Selection of trustworthy crowd workers for telemedical diagnosis of pediatric autism spectrum disorder,” Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing, vol. 26, no. 26th Pacific Symposium on Biocomputing (PSB). Stanford Univ, Dept Bioengn, Palo Alto, CA 94305 USA, pp. 14–25, 2021. doi: 10.1142/9789811232701_0002.
  • Lucieri, (2022), Comput. Methods Programs Biomed., 215, 10.1016/j.cmpb.2022.106620
  • Anagnostou, (2022), Ethics Inf. Technol., 24, pp. 1, 10.1007/s10676-022-09634-1
  • A. S. Hussein, W. M. Omar, X. Li, and M. Ati, “Efficient Chronic Disease Diagnosis prediction and recommendation system,” in 2012 IEEE-EMBS Conference on Biomedical Engineering and Sciences, IECBES 2012, 2012, pp. 209–214. doi: 10.1109/IECBES.2012.6498117.
  • Martínez-Agüero, (2022), Futur. Gener. Comput. Syst., 133, pp. 68, 10.1016/j.future.2022.02.021
  • Kerasidou, (2021), J. Oral Biol. Craniofacial Res., 11, pp. 612, 10.1016/j.jobcr.2021.09.004
  • Carrington, (2023), IEEE Trans. Pattern Anal. Mach. Intell., 45, pp. 329, 10.1109/TPAMI.2022.3145392
  • Harerimana, (2021), J. Biomed. Inform., 118, 10.1016/j.jbi.2021.103778
  • Ben Yahia, (2022), Big Data Res, 27, 10.1016/j.bdr.2021.100286
  • Karim, (2022), IEEE Access, 10, pp. 54386, 10.1109/ACCESS.2022.3175816
  • Abdar, (2023), Inf. Fusion, 90, pp. 364, 10.1016/j.inffus.2022.09.023
  • Bania, (2021), Artif. Intell. Med., 114, 10.1016/j.artmed.2021.102049
  • Loey, (2022), Comput. Biol. Med., 142, 10.1016/j.compbiomed.2022.105213
  • Al Mamun, (2017), Futur. Gener. Comput. Syst., 66, pp. 36, 10.1016/j.future.2015.11.010
  • Chou, (2022), Inf. Fusion, 81, pp. 59, 10.1016/j.inffus.2021.11.003
  • Müller, (2022), N. Biotechnol., 70, pp. 67, 10.1016/j.nbt.2022.05.002
  • Deperlioglu, (2022), Futur. Gener. Comput. Syst., 129, pp. 152, 10.1016/j.future.2021.11.018
  • Barredo Arrieta, (2020), Inf. Fusion, 58, pp. 82, 10.1016/j.inffus.2019.12.012
  • Sachan, (2021), Expert Syst. Appl., 185, 10.1016/j.eswa.2021.115597
  • Du, (2022), Sci. Rep., 12, pp. 1170, 10.1038/s41598-022-05112-2
  • Ullah, (2021), Futur. Gener. Comput. Syst., 124, pp. 369, 10.1016/j.future.2021.06.012
  • Giordano, (2021), Front. Digit. Heal., 3, pp. 65
  • El-Sappagh, (2021), Sci. Rep., 11, 10.1038/s41598-021-82098-3
  • González-Gonzalo, (2021), Prog. Retin. Eye Res.
  • Müller, (2020), Comput. Graph., 91, pp. 1, 10.1016/j.cag.2020.06.004
  • Alanazi, (2022), Diagnostics, 12, 10.3390/diagnostics12123060
  • Zerka, (2020), IEEE Access, 8, pp. 183939, 10.1109/ACCESS.2020.3029445
  • Guiñazú, (2020), Inf. Fusion, 55, pp. 150, 10.1016/j.inffus.2019.08.006
  • Leal, (2021), Big Data Res, 24, 10.1016/j.bdr.2020.100172
  • Zarour, (2020), IEEE Access, 8, pp. 157959, 10.1109/ACCESS.2020.3019829
  • Ouyang, (2021), Inf. Sci. (Ny)., 570, pp. 124, 10.1016/j.ins.2021.04.021
  • Rahman, (2021), Sustain. Cities Soc., 72, 10.1016/j.scs.2021.103083
  • Abou-Nassar, (2020), IEEE Access, 8, pp. 111223, 10.1109/ACCESS.2020.2999468
  • Pal, (2020), Front. Robot. AI, 7, 10.3389/frobt.2020.00076
  • I. Barclay and W. Abramson, “Identifying Roles, Requirements and Responsibilitiesin Trustworthy AI Systems,” in UbiComp/ISWC 2021 - Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers, 2021, pp. 264–271. doi: 10.1145/3460418.3479344.
  • Collins, (2021), Eur. Urol. Focus
  • R. Larasati, A. De Liddo, and E. Motta, “AI Healthcare System Interface: Explanation Design for Non-Expert User Trust,” in CEUR Workshop Proceedings, 2021, vol. 2903.
  • Delacroix, (2021), Comput. Law Secur. Rev., 40, 10.1016/j.clsr.2020.105520
  • Holzinger, (2021), Inf. Fusion, 71, pp. 28, 10.1016/j.inffus.2021.01.008
  • Rieke, (2020), npj Digit. Med., 3, 10.1038/s41746-020-00323-1
  • Wenzel, (2020), IEEE Commun. Stand. Mag., 4, pp. 64, 10.1109/MCOMSTD.001.2000006
  • Sheikh, (2021), Lancet Digit. Heal., 3, pp. e383, 10.1016/S2589-7500(21)00005-4
  • Ho, (2021), Semin. Nephrol., 41, pp. 282, 10.1016/j.semnephrol.2021.05.009
  • Faris, (2021), Informatics Med. Unlocked, 23, 10.1016/j.imu.2021.100513
  • Oprescu, (2022), Inf. Fusion, 83–84, pp. 53, 10.1016/j.inffus.2022.03.011
  • Esposito, (2018), Eng. Appl. Artif. Intell., 67, pp. 136, 10.1016/j.engappai.2017.09.019
  • Rathi, (2021), Comput. Electr. Eng., 96, 10.1016/j.compeleceng.2021.107524
  • Rehman, (2021), Energies, 14, 10.3390/en14196414
  • Saba, (2020), J. Infect. Public Health, 13, pp. 1567, 10.1016/j.jiph.2020.06.027
  • Wang, (2022), Inf. Sci. (Ny)., 617, pp. 133, 10.1016/j.ins.2022.10.060
  • Alshehri, (2021), IEEE Access, 9, pp. 3660, 10.1109/ACCESS.2020.3047960
  • Shoeibi, (2023), Inf. Fusion, 93, pp. 85, 10.1016/j.inffus.2022.12.010
  • Alzubaidi, (2021), J. Big Data, 8, pp. 53, 10.1186/s40537-021-00444-8
  • Abolfazlian, (2020), 584, pp. 15
  • Ayhan, (2022), Med. Image Anal., 77, 10.1016/j.media.2022.102364
  • Saporta, (2022), Nat. Mach. Intell., 4, pp. 867, 10.1038/s42256-022-00536-x
  • Fan, (2021), IEEE Trans. Radiat. Plasma Med. Sci., 5, pp. 741, 10.1109/TRPMS.2021.3066428
  • Bergur Thormundsson, “Global explainable AI market revenues 2021-2030 | Statista.”
  • Goldberg, (2019), Not. Am. Math. Soc., 66, pp. 1, 10.1090/noti1912
  • Gunning, (2019), AI Mag, 40, pp. 44
  • Holzinger, (2022), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 13200, pp. 13
  • W. Samek, G. Montavon, S. Lapuschkin, C. J. Anders, and K. R. Müller, “Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications,” Proc. IEEE, vol. 109, no. 3, pp. 247–278, 2021, doi: 10.1109/JPROC.2021.3060483.
  • M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’ Explaining the Predictions of Any Classifier,” in NAACL-HLT 2016 - 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Demonstrations Session, 2016, pp. 97–101. doi: 10.18653/v1/n16-3020.
  • M. T. Ribeiro, S. Singh, and C. Guestrin, “Anchors: High-precision model-agnostic explanations,” 32nd AAAI Conf. Artif. Intell. AAAI 2018, vol. 32, no. 1, pp. 1527–1535, Apr. 2018, doi: 10.1609/aaai.v32i1.11491.
  • Huang, (2022), IEEE Trans. Knowl. Data Eng., 10.1109/TKDE.2022.3187455
  • Bach, (2015), PLoS One, 10, 10.1371/journal.pone.0130140
  • Montavon, (2017), Pattern Recognit, 65, pp. 211, 10.1016/j.patcog.2016.11.008
  • Robnik-Šikonja, (2008), IEEE Trans. Knowl. Data Eng., 20, pp. 589, 10.1109/TKDE.2007.190734
  • C. Frye, D. de Mijolla, T. Begley, L. Cowton, M. Stanley, and I. Feige, “Shapley explainability on the data manifold,” arXiv Prepr. arXiv2006.01272, 2020.
  • Frye, (2020), Adv. Neural Inf. Process. Syst., 2020, pp. 1229
  • Biecek, (2018), J. Mach. Learn. Res., 19, pp. 3245
  • J. Wang, J. Wiens, and S. Lundberg, “Shapley Flow: A Graph-based Approach to Interpreting Model Predictions,” in International Conference on Artificial Intelligence and Statistics, 2020, pp. 721–729.
  • Nadeem, (2020), Symmetry (Basel), 12, pp. 1
  • M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in 34th International Conference on Machine Learning, ICML 2017, 2017, vol. 7, pp. 5109–5118.
  • P. Madumal, T. Miller, L. Sonenberg, and F. Vetere, “Explainable reinforcement learning through a causal lens,” in AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, 2020, vol. 34, no. 03, pp. 2493–2500. doi: 10.1609/aaai.v34i03.5631.
  • R. C. Fong and A. Vedaldi, “Interpretable Explanations of Black Boxes by Meaningful Perturbation,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, vol. 2017 -Octob, pp. 3449–3457. doi: 10.1109/ICCV.2017.371.
  • Díaz-Rodríguez, (2022), Inf. Fusion, 79, pp. 58, 10.1016/j.inffus.2021.09.022
  • Crigger, (2022), J. Med. Syst., 46, pp. 1, 10.1007/s10916-021-01790-z
  • Gille, (2020), Intell. Med., 1–2
  • Yang, (2022), Electron. Mark, pp. 1
  • Stoger, (2021), Commun. ACM, 64, pp. 34, 10.1145/3458652
  • Ali, (2020), Inf. Fusion, 63, pp. 208, 10.1016/j.inffus.2020.06.008
  • Meng, (2020), Inf. Fusion, 57, pp. 115, 10.1016/j.inffus.2019.12.001
  • Zeng, (2022), IEEE Trans. Instrum. Meas., 71, pp. 1
  • Zhang, (2021), Inf. Fusion, 76, pp. 323, 10.1016/j.inffus.2021.06.008
  • Tang, (2020), Eur. Radiol., 30, pp. 823, 10.1007/s00330-019-06441-z
  • Al-Timemy, (2022), Cognit. Comput., 14, pp. 1627, 10.1007/s12559-021-09880-3
  • Li, (2022), Inf. Fusion, 79, pp. 229, 10.1016/j.inffus.2021.10.018
  • Yadav, (2020), Med. Biol. Eng. Comput., 58, pp. 669, 10.1007/s11517-020-02136-6
  • Wang, (2020), Remote Sens. Environ., 249, 10.1016/j.rse.2020.112009