Fairness Matters: Evaluating and Mitigating Bias in Skin Cancer Classification
DOI:
https://doi.org/10.47611/jsrhs.v13i3.7017Keywords:
Artificial Intelligence, Deep Learning, Skin Cancer DiagnosisAbstract
Almost 1 in 5 Americans develop skin cancer by the age of 70, and it is the most commonly diagnosed cancer in the United States, with most cases being preventable. Artificial intelligence (AI) has shown great potential to accelerate the diagnosis of skin cancer for early treatment. However, the fairness of using AI for skin cancer detection has raised concerns due to the lower accuracy of "darker" skin tone detection. This paper conducts a comprehensive study on the bias problem using the fitzpatrick dataset, while analyzing the effects of different methods towards mitigating this bias. Throughout our experiments, we found that not only was the darkest skin type biased, one of the specific light skin types also had a relatively low accuracy. Five AI models were evaluated and compared in terms of bias, and four augmentation techniques were applied to mitigate bias. Moreover, we studied the impact of training parameters (e.g. batch size, data splitting) on bias.
Downloads
References or Bibliography
References
Bethanney Janney, J., Krishnamoorthy, N. R., Divakaran, S., Sudhakar, T., Krishnakumar, S., & Akshya., V. (2021). Diagnosis of skin malignancy using deep learning approaches. 2021 International Conference on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA). https://doi.org/10.1109/icaeca52838.2021.9675722
Bissoto, A., Fornaciali, M., Valle, E., & Avila, S. (2019). (DE) constructing bias on skin lesion datasets. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://doi.org/10.1109/cvprw.2019.00335
Brancaccio, G., Balato, A., Malvehy, J., Puig, S., Argenziano, G., & Kittler, H. (2024). Artificial Intelligence in skin cancer diagnosis: A reality check. Journal of Investigative Dermatology, 144(3), 492–499. https://doi.org/10.1016/j.jid.2023.10.004
Buolamwini, J., & Gebru, T. (2018, January 21). Gender shades: Intersectional accuracy disparities in commercial gender classification. PMLR. https://proceedings.mlr.press/v81/buolamwini18a.html
Characteristics of publicly available skin cancer image datasets: A systematic review - The Lancet Digital Health. (n.d.). https://www.thelancet.com/journals/landig/article/PIIS2589-7500(21)00252-1/fulltext
Daneshjou, R., Vodrahalli, K., Liang, W., Novoa, R. A., Jenkins, M., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., Mukherjee, P., Phung, M., Yekrang, K., Fong, B., Sahasrabudhe, R., Zou, J., & Chiou, A. (2021, November 15). Disparities in dermatology AI: Assessments using diverse clinical images. arXiv.org. https://arxiv.org/abs/2111.08006
Faghihi, A., Fathollahi, M., & Rajabi, R. (2024, April 1). Diagnosis of skin cancer using VGG16 and VGG19 based transfer learning models. arXiv.org. https://arxiv.org/abs/2404.01160
Fitzpatrick scale at Emerge in Tulsa | Wellness, med spa, salon. (2023, March 28). Emerge Medical. https://emergetulsa.com/fitzpatrick/
Galdran, A., ∗, Alvarez-Gila, A., Meyer, M. I., Saratxaga, C. L., Ara´Ujo, T., Garrote, E., Aresta, G., Costa, P., Mendonc¸¸A, A. M., & Campilho, A. (2017). Data-Driven Color augmentation Techniques for Deep skin image analysis [Journal-article]. https://arxiv.org/pdf/1703.03702.pdf
Google Colab. (n.d.). Colab.google. colab.google. https://colab.google/
Groh, M., Harris, C., Soenksen, L., Lau, F., Han, R., Kim, A., Koochek, A., & Badri, O. (2021, April 20). Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17K dataset. arXiv.org. https://arxiv.org/abs/2104.09957
Hosny, K. M., Kassem, M. A., & Foaud, M. M. (2018). Skin cancer classification using Deep Learning and Transfer Learning. 2018 9th Cairo International Biomedical Engineering Conference (CIBEC). https://doi.org/10.1109/cibec.2018.8641762
Jain, S., Singhania, U., Tripathy, B., Nasr, E. A., Aboudaif, M. K., & Kamrani, A. K. (2021). Deep learning-based transfer learning for classification of Skin cancer. Sensors, 21(23), 8142. https://doi.org/10.3390/s21238142
Kaggle: your machine learning and data science community. (n.d.). https://www.kaggle.com/
Li, Z., & Xu, C. (2021). Discover the unknown biased attribute of an image classifier. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv48922.2021.01470
Mattgroh. (n.d.). GitHub - mattgroh/fitzpatrick17k. GitHub. https://github.com/mattgroh/fitzpatrick17k
Melanoma & Skin of Color - Melanoma Research Alliance. (n.d.). Melanoma Research Alliance. https://www.curemelanoma.org/about-melanoma/people-of-color
Melarkode, N., Srinivasan, K., Qaisar, S. M., & Plawiak, P. (2023). Ai-powered diagnosis of Skin cancer: A contemporary review, open challenges and future research directions. Cancers, 15(4), 1183. https://doi.org/10.3390/cancers15041183
Rezk, E., Eltorki, M., & El-Dakhakhni, W. (2022). Improving skin color diversity in cancer detection: Deep Learning Approach. JMIR Dermatology, 5(3). https://doi.org/10.2196/39143
Salman, H., Jain, S., Ilyas, A., Engstrom, L., Wong, E., & Madry, A. (2022, July 6). When does bias transfer in transfer learning?. arXiv.org. https://arxiv.org/abs/2207.02842
Skin cancer. (2022, April 22). https://www.aad.org/media/stats-skin-cancer#:~:text=The%20five%2Dyear%20survival%20rate,the%20lymph%20nodes%20is%2099%25.&text=The%20five%2Dyear%20survival%20rate%20for%20melanoma%20that%20spreads%20to,and%20other%20organs%20is%2030%25.
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2016, August 23). Inception-V4, inception-resnet and the impact of residual connections on learning. arXiv.org. https://arxiv.org/abs/1602.07261
Tan, M., & Le, Q. V. (2020, September 11). EfficientNet: Rethinking model scaling for Convolutional Neural Networks. arXiv.org. https://arxiv.org/abs/1905.11946
Waweru, A. K., Ahmed, K., Miao, Y., & Kawan, P. (2020). Deep learning in skin lesion analysis towards cancer detection. 2020 24th International Conference Information Visualisation (IV). https://doi.org/10.1109/iv51561.2020.00130
Published
How to Cite
Issue
Section
Copyright (c) 2024 Amy Zhang, Ana Zhao, Andrew Han
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright holder(s) granted JSR a perpetual, non-exclusive license to distriute & display this article.
