An ensemble-based Convolutional Neural Network model powered by genetic algorithm for melanoma diagnosis

Authors

Eduardo Pérez and Sebastián Ventura

Problem statement

Despite the expertise of dermatologists, the early diagnosis of melanoma remains as a tough task, because it is present in many different shapes, sizes and colors, even between samples of the same category. Morphological differences in images belonging to patients with different biological conditions (normal and malignant) can be seen in the following Figure (images taken from HAM10000 dataset).

Segmentation performance

Table 1: Evaluation of the segmentation methods in PH2 dataset. The best average U-Net score value was highlighted in bold typeface. No significant differences were encountered, the Friedman’s test didn’t rejected the null hypothesis with a p-value equal to 6.476E-1; Friedman’s statistic was equal to 0.86898 with two degrees of freedom. Last row shows the Friedman’s ranking, where the proposed segmentation method achieved the best average ranking, followed by R2U-Net.

Table 2: Evaluation of the segmentation methods in DERMLIB dataset. The best average U-Net score value was highlighted in bold typeface. Significant differences were encountered, the Friedman’s test rejected the null hypothesis with a p-value < 2.2e-16; Friedman’s statistic was equal to 108.3 with two degrees of freedom. Last row shows the Friedman’s ranking, where the proposed segmentation method achieved the best average ranking, followed by U-Net. Afterwards, the Hommel’s post-hoc test was conducted by considering the proposal as the control method, and the results showed that it significantly outperformed U-Net and R2U-Net.

Visualization of the individual predictions

To sum up, the results obtained through the experimental study revealed that GAPE-CNN was very effective for diagnosing melanoma, attaining better predictive performance with respect to the state-of-the-art models. However, the visualization of the individual predictions made by the model could add extra value when it is combined with experts. In this regard, several methods can be used to illustrate the predictions of black-box models, e.g. by using Accumulated Local Effects (Apley et al. 2016), Local Surrogate Models (Ribeiro et al. 2016), Counterfactual Explanations (Wachter et al. 2017), and Shapley Additive Explanations (SHAP) (Lundberg et al. 2017), among others. However, SHAP values are facing some criticism in recent articles. For instance, Kumar et al. (2020) stated that this method may not directly generate explanations, but when it is combined with domain knowledge, they can achieve quantitative comparisons and insights in individual predictions. As a result, in this work SHAP values are only applied to show which pixels are mostly activated during prediction in individual samples. After that, it should be complemented and interpreted by using dermatologists’ expertise. For example, next Figure shows how the proposal learned the features that allow to correctly predict whether a sample is melanoma or not. Now, the experts not only can analyze the probability that a test sample is melanoma or not, but also assess the rationale followed by the model to give such a conclusion. Red pixels increase the probability that a sample is melanoma, whereas the blue pixels do the opposite.

 

Apley, D. W. (2016). Visualizing the effects of predictor variables in black box supervised learning models. ArXiv Preprint ArXiv:1612.08468, 44.

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should {I} Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd {ACM} {SIGKDD} International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, 1135–1144.

Wachter, S., Mittelstadt, B., & Russell, C. (2017). Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GPDR. Harv. JL \& Tech., 31, 841.

Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Proceedings of the Advances in Neural Information Processing Systems, 4765–4774.

Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., & Friedler, S. (2020). Problems with Shapley-value-based explanations as feature importance measures. http://arxiv.org/abs/2002.11097