
==== Front
Indian J Dermatol
Indian J Dermatol
IJD
Indian J Dermatol
Indian Journal of Dermatology
0019-5154
1998-3611
Wolters Kluwer - Medknow India

IJD-69-296
10.4103/ijd.ijd_61_24
Original Article
Comparison of the Diagnostic Accuracy of Teledermoscopy, Face-to-Face Examinations and Artificial Intelligence in the Diagnosis of Melanoma
Yazdanparast Taraneh 1
Shamsipour Mansour 23
Ayatollahi Azin 1
Delavar Shohreh 14
Ahmadi Maryam 1
Samadi Aniseh 1
Firooz Alireza 1
1 From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
2 Department of Research Methodology and Data Analysis, Tehran University of Medical Sciences, Tehran, Iran
3 Center for Air Pollution Research (CAPR), Institute for Environmental Research, Tehran University of Medical Sciences, Tehran, Iran
4 Dermatology Department, Faculty of Medicine Tehran Medical Sciences, Islamic Azad University, Tehran, Iran
Address for correspondence: Dr. Alireza Firooz, Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, 415 Taleghani Avenue, Tehran, Postal Code: 1416613675, Iran. E-mail: firozali@sina.tums.ac.ir
Jul-Aug 2024
19 8 2024
69 4 296300
1 2024
3 2024
Copyright: © 2024 Indian Journal of Dermatology
2024
https://creativecommons.org/licenses/by-nc-sa/4.0/ This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.
Background:

Rapid diagnosis of melanoma is necessary for a good prognosis. Using teledermatology and artificial intelligence for this issue is developing, but its diagnostic accuracy is less measured in a clinical setting.

Objective:

The purpose of this study was to assess the diagnostic accuracy of the teledermoscopy method using the FotoFinder device as well as the Moleanalyzer Pro artificial intelligence (AI) Assistant and to compare them with the face-to-face clinical examination for the diagnosis of melanoma confirmed with histopathology.

Methods:

Thirty melanocytic moles of 29 patients were included in the study. Each mole was assessed face-to-face, using FotoFinder teledermoscopy and Moleanalyzer Pro software methods. The results obtained from each method were compared with the results of the gold standard (pathology). The sensitivity and specificity of the three methods were calculated for malignant and borderline versus benign lesions. Inter-method reliability between a gold standard and other methods was evaluated using per cent agreement and Cohen’s kappa coefficient.

Results:

Five moles had a histopathological diagnosis of melanoma, and six and 19 moles were diagnosed as borderline and benign, respectively. Sensitivities and specificities were, respectively, as follows: face-to-face (90.9%, 57.9%), FotoFinder teledermoscopy (63.6%, 78.9%), FotoFinder® Moleanalyzer Pro (36.4%, 42.1%). Agreement with biopsy-obtained diagnosis categories of benign, borderline and malignant for face-to-face was 63.33%, FotoFinder teledermoscopy 73.33%, and FotoFinder® Moleanalyzer Pro 40%.

Conclusions:

Teledermoscopy had the highest agreement with reference diagnosis as well as the highest specificities that caused a reduction of biopsy referrals. The FotoFinder® Moleanalyzer Pro had the lowest agreement. Therefore, it cannot replace dermatologist decision making.

KEY WORDS:

Artificial intelligence
melanoma
teledermoscopy
==== Body
pmcIntroduction

Malignant melanoma is the most dangerous type of skin tumour.[1] Although melanoma is a rare skin cancer, it is responsible for more than 70% of all skin cancer-associated deaths.[23] The incidence rates of skin cancers have increased over the last decades.[4] The resulting high number of skin cancer referrals from primary health care to dermatologists can result in a substantial delay until a face-to-face (FTF) appointment. As its prognosis is limited in advanced stages, early diagnosis is the most effective intervention to improve prognosis and reduce cost and mortality complications.[5] In recent years, teledermatology and artificial intelligence have been developed as a solution to this problem.[6]

Teledermoscopy is a form of teledermatology that involves the use of dermoscopic images to improve the clinical decision making of dermatologists.[7] Several smartphone applications for melanoma detection have been released, but there has been little clinical validation.[8] A poorly designed, inaccurate or misleading consumer application could harm a patient.[9] However, with appropriate development and evaluation, modern electronic technology can increase diagnostic accuracy. In fact, artificial intelligence (AI) algorithms recently demonstrated that they can classify photographs of lesions with a level of accuracy comparable to those provided by dermatologists to classify melanoma.[1011]

One of the new teledermoscopy technologies is the FotoFinder System. This device impressively visualises the colour, borders, symmetry axes and structure of the lesion and indicates the size, perimeter and diameter of the lesion.[12] Moleanalyzer Pro-AI Assistant of this system is the first market-approved Convolutional neural network in Europe. Moleanalyzer Pro software system uses a complex machine algorithm that determines the probability of the lesion being melanoma without the aid of a dermatologist.[13]

As the aforementioned system and software are less tested in clinical settings and the diagnostic accuracy of teledermoscopy and its effectiveness in the referral rate of benign lesions is yet unclear[1415] in this study, we investigated the diagnostic accuracy of the teledermoscopy method using the FotoFinder® system as well as Moleanalyzer Pro software and compared them with the FTF clinical examination for the diagnosis of melanoma confirmed with histopathology.

Method

This analytical cross-sectional study was conducted at the Center for Research and Training in Skin Disease and Leprosy (Tehran University of Medical Sciences) between December 2019 and May 2022. A total of 30 melanocytic moles from 29 patients (age > 18) were included in the study. One expert dermatologist established a diagnosis of melanoma and biopsy needed based on clinical detection. Patients were excluded if they had mucosal lesions, lesions located on scars, psoriasis, eczema or other skin conditions, lesions covered by thick hair, lesions with a history of previous biopsy or any type of treatment, ulcerated lesions and patients under 18 years. The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Tehran University of Medical Sciences (ethics code: IR.TUMS.VCR.REC.1397.012). Informed consent was obtained from each participant. The data of patients were kept confidential and all assessments were non-invasive and performed free of charge.

Convolutional neural network and dermatologists

We have used the latest version of a convolutional neural network (CNN), which has been approved as a medical device for the European market (Moleanalyzer Pro®, FotoFinder Systems GmbH, Bad Birnbach Germany). The neural network architecture and training procedures are described in detail elsewhere.[10] A trained medical doctor captured the dermoscopic images (Resolution of 1920 × 1080 pixels and 20 times magnification) of the melanocytic skin lesions. Moleanalyzer Pro software analysed the dermoscopic data and reported the results, classifying the lesions into one of the following three groups: green (benign lesion), red (malignant lesion), and yellow (borderline lesion). Moreover, to perform the teledermoscopy method, the same set of dermoscopic images, without the patient history, were forwarded to a certain dermatologist. View-only software, which displayed dermoscopic images in the same resolution, had been installed on the dermatologist’s computer. The dermatologist investigated dermoscopic data and reported the results, as previously mentioned, by classifying the mole into one of these three groups: benign, malignant, and borderline (Atypical melanocytic moles were considered borderline). In the FTF method, all subjects underwent a comprehensive dermatologic evaluation. The examining dermatologist reported the probability of the lesion malignancy as the same classification as the other techniques, with a naked eye and having access to the patient’s history. Afterwards, patients were referred for a biopsy approval of lesions as a gold standard of diagnosis. The clinical images, dermoscopic features and corresponding results of four cases are presented in Figure 1.

Figure 1 Clinical images and dermoscopic features of four cases. Case 1: Pathology Result: Benign, Teledermoscopy Result: Benign, Face-to-Face Clinical Result: Borderline, Moleanalyzer Pro Result: Benign. Case 2: Pathology Result: Benign, Teledermoscopy Result: Benign, Face-to-Face Clinical Result: Benign, Moleanalyzer Pro Result: Benign. Case 3: Pathology Result: Benign, Teledermoscopy Result: Malignant, Face-to-Face Clinical Result: Malignant, Moleanalyzer Pro Result: Malignant. Case 4: Pathology Result: Malignant, Teledermoscopy Result: Benign, Face-to-Face Clinical Result: Borderline, Moleanalyzer Pro Result: Benign

Statistical analysis

Data were analysed using IBM SPSS Statistics 24 (SPSS Inc., Chicago, IL, USA). Baseline characteristics of participants were shown as mean ± SD or median (interquartile range) for continuous variables and frequency (percentage) for categorical variables. To determine the sensitivity and specificity of the assessment methods, the results obtained from each method were compared with the results of the gold standard (pathology). The sensitivity and specificity of the three methods were calculated for malignant and borderline versus benign lesions. Inter-method reliability between the gold standard and other methods was evaluated using Cohen’s kappa coefficient. Statistical significance was taken as P < 0.05.

Result

Thirty lesions were analysed from 29 patients. Eleven lesions from 11 patients had a histopathological diagnosis of melanoma (5 cases) or borderline (6 cases), while 19 lesions from 19 patients were diagnosed as benign. Patient demographics and lesion diagnosis are described in Table 1. The mean age of the patients was 44.9 years (range: 19-79). There were 12 men and 17 women. Melanocytic lesions were located mainly in the head and face (43.3%).

Table 1 Demographic and clinical features of study groups

Variable	Measurment	
Study population		
 Patients	29	
 Lesions	30	
Age (mean range)	44.9 (19-79)	
Sex		
 Male	12	
 Female	17	
Anatomic site		
 Face	13 (43%)	
 Trunk	6 (20%)	
 Extremities	7 (23%)	
 Neck	3 (10%)	
Histopathology diagnosis		
 Benign	19 (63%)	
 Borderline	6 (20%)	
 Malignant	5 (16%)	

The FTF method had a sensitivity of 90.9% and a specificity of 57.9%. The agreement between the pathological diagnosis and the FTF method was 63.33% (kappa: 0.41, P > 0.001). This method reported 15 cases as borderline, seven of which were benign, two were malignant, and six were correct. Three malignant lesions were reported by the FTF method as benign or borderline.

The FotoFinder teledermoscopy method had a sensitivity of 63.6% and a specificity of 78.9%. The agreement between the pathological diagnosis and the teledermatology diagnosis was 73.33% (kappa: 0.49, P > 0.001). This method reported seven lesions as borderline, four of which were correct, and the others were benign. The teledermatology missed two melanomas.

FotoFinder® Moleanalyzer Pro had a sensitivity of 36.4% and a specificity of 42.1%. The percentage of agreement was 40.00% (kappa: -0.01, P = 0.53). This method reported six cases as borderline, one of which was correct, and the other five were benign. It missed two malignant lesions. The results of the statistical analysis are shown in Table 2.

Table 2 Diagnostic performance of methods

	FotoFinder Teledermoscopy	Face-to-face Examination	FotoFinder Moleanalyzer Pro	
Sensitivity	63.6%	90.9%	36.4%	
Specificity	78.9%	57.9%	42.1%	
False-negative rate	36.4%	9.1%	63.6%	
False-positive rate	21%	42.1%	57.9%	
Per cent agreement	73.33%	63.33%	40.00%	
Value of Kappa	0/49	0/41	-0/01	

Discussion

We report an independent, prospective, diagnostic concordance study of the detection of melanoma using clinical/dermoscopic examinations, teledermoscopy and AI algorithm based on a non-invasive imaging system. The results of the study showed that the level of agreement between the teledermatology, FTF, and FotoFinder® Moleanalyzer Pro software methods with the pathology method in the diagnosis of melanoma based on kappa index was moderate, moderate, and none, respectively.

The teledermoscopy method had an acceptable and higher diagnostic per cent agreement rate, so it could be a successful technique in diagnosing melanoma. Also, teledermoscopy had the highest specificity compared to other methods, which means dermoscopy, by providing high-resolution photos of melanocytic moles for dermatologists, reduces the number of unnecessary biopsies of benign lesions. The sensitivity of this method has been reported to be lower than the FTF method; the reason appears to be the dermatologist not having access to the patient’s history.[10] Moreover, studies confirm that the diagnostic accuracy of teledermoscopy is related to the doctor’s experience. Also, proper training and knowledge of melanoma-specific structures by any physician who engages in skin cancer screening have an important role in the capacity to use dermoscopy.[1617] In addition, dermoscopic signs of some melanomas become visible late, so even with dermoscopy, it is difficult to diagnose some of the melanomas.[18] Therefore, with proper training and access to the patient’s history, the sensitivity of the teledermoscopy method can be improved as much as the sensitivity of the FTF method.

The FotoFinder® Moleanalyzer Pro had the lowest agreement rate with the reference diagnosis compared to other methods, as well as the lowest sensitivity and specificity. MacLellan et al.[13] performed a comparative study of the diagnostic accuracy of teledermoscopy and FTF methods and showed that Moleanalyzer Pro software of FotoFinder System, in comparison with other software, has the highest sensitivity and specificity and could be a valuable tool to assist, but not replace dermatologists. Also, the highest specificity was reported for the teledermoscopy method (82.6%), and the highest sensitivity for the FTF method (96.6%).

Our research supports a previous study in 2022 by Winkler and Haenssle,[19] which claimed that cooperation of “man with a machine” has the best result in the assessment of pigmented and non-pigmented skin lesions, but that using Moleanalyzer Pro software of FotoFinder System has some limitations in clinical settings as well.

A meta-analysis of 22 studies reported that dermoscopy with an expert dermatologist reaches a sensitivity of 89% and specificity of 79%, which is very close to the results of our study, especially about specificity and also mentioned that dermoscopy increases diagnostic accuracy in the clinical diagnosis of a questionable lesion.[20]

Börve A. et al.[21] obtained a diagnostic accuracy of 60.9% using the FotoFinder Handyscope® dermatoscope, which is slightly lower than the diagnostic accuracy of our study (73.33%). They, however, considered all skin cancer types in calculating the diagnostic accuracy of FotoFinder teledermoscopy.

Sies K. et al.,[22] in a non-interventional study, reported a sensitivity of 44.2% for using Moleanalyzer Pro to detect melanoma, which is slightly higher than the result of our study. The reason could be that retrospective studies in this field are usually prone to selection and confirmation bias.[23]

The current study encountered some limitations. Firstly, the study included only lesions requiring biopsy or excision, resulting in a selection bias. Dermatologists were aware of this fact, which may have influenced their management decisions. The small sample size might lessen the power of our results. Besides, our patients were recruited from one single centre, which might affect our data’s generalizability. These drawbacks make it necessary to implement future studies with larger sample sizes to validate our results.

Overall, it is shown that successful computer-aided diagnosis supports dermatologists in the early detection of melanoma, and using teledermoscopy for screening melanoma will be useful.[24] The teledermoscopy method, because of greater agreement with biopsy diagnosis and reduction of biopsy referrals, besides saving money and time for doctors and patients, is an effective method for early diagnosis of melanoma.

Conclusion

In conclusion, Using FotoFinder in the teledermoscopy method is valuable in early diagnosis of melanoma. Moleanalyzer Pro software in comparison with FTF method and teledermoscopy method is a poor technique. Therefore, it cannot replace dermatologist decision making.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.
==== Refs
1 Davis LE Shalin SC Tackett AJ Current state of melanoma diagnosis and treatment Cancer Biol Ther 2019 20 1366 79 31366280
2 Johnston L Starkey S Mukovozov I Robertson L Petrella T Alhusayen R Surveillance after a previous cutaneous melanoma diagnosis: A scoping review of melanoma follow-up guidelines J Cutan Med Surg 2023 27 516 25 37489919
3 Haggenmüller S Maron RC Hekler A Utikal JS Barata C Barnhill RL Skin cancer classification via convolutional neural networks: Systematic review of studies involving human experts Eur J Cancer 2021 156 202 16 34509059
4 Dahlén Gyllencreutz J Paoli J Bjellerup M Bucharbajeva Z Gonzalez H Nielsen K Diagnostic agreement and interobserver concordance with teledermoscopy referrals J Eur Acad Dermatol Venereol 2017 31 898 903 28150389
5 Bruce AF Mallow JA Theeke LA The use of teledermoscopy in the accurate identification of cancerous skin lesions in the adult population: A systematic review J Telemed Telecare 2018 24 75 83 28056600
6 Foahom Gouabou AC Collenne J Monnier J Iguernaissi R Damoiseaux J-L Moudafi A Computer aided diagnosis of melanoma using deep neural networks and game theory: Application on dermoscopic images of skin lesions Int J Mol Sci 2022 23 13838 doi: 10.3390/ijms232213838 36430315
7 Vestergaard T Andersen MK Bygum A Acceptance of teledermoscopy by general practitioners and dermatologists in Denmark Dermatol Pract Concept 2021 11 e2021033 doi: 10.5826/dpc. 1102a33 33954016
8 Chuchu N Takwoingi Y Dinnes J Matin RN Bassett O Moreau JF Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma Cochrane Database Syst Rev 2018 12 Cd013192 doi: 10.1002/14651858.CD013192 30521685
9 Wolf JA Moreau JF Akilov O Patton T English JC 3rd Ho J Diagnostic inaccuracy of smartphone applications for melanoma detection JAMA Dermatol 2013 149 422 6 23325302
10 Haenssle HA Fink C Schneiderbauer R Toberer F Buhl T Blum A Man against machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists Ann Oncol 2018 29 1836 42 29846502
11 Esteva A Kuprel B Novoa RA Ko J Swetter SM Blau HM Dermatologist-level classification of skin cancer with deep neural networks Nature 2017 542 115 8 28117445
12 Fink C Blum A Buhl T Mitteldorf C Hofmann Wellenhof R Deinlein T Diagnostic performance of a deep learning convolutional neural network in the differentiation of combined naevi and melanomas J Eur Acad Dermatol Venereol 2020 34 1355 61 31856342
13 MacLellan AN Price EL Publicover-Brouwer P Matheson K Ly TY Pasternak S The use of noninvasive imaging techniques in the diagnosis of melanoma: A prospective diagnostic accuracy study J Am Acad Dermatol 2021 85 353 9 32289389
14 Del Rosario F Farahi JM Drendel J Buntinx-Krieg T Caravaglio J Domozych R Performance of a computer-aided digital dermoscopic image analyzer for melanoma detection in 1,076 pigmented skin lesion biopsies J Am Acad Dermatol 2018 78 927 34.e6 29678380
15 Uppal SK Beer J Hadeler E Gitlow H Nouri K The clinical utility of teledermoscopy in the era of telemedicine Dermatol Ther 2021 34 e14766 doi: 10.1111/dth. 14766 33421232
16 Marghoob NG Liopyris K Jaimes N Dermoscopy: A review of the structures that facilitate melanoma detection J Am Osteopath Assoc 2019 119 380 90 31135866
17 Nervil GG Ternov NK Vestergaard T Sølvsten H Chakera AH Tolsgaard MG Improving skin cancer diagnostics through a mobile app with a large interactive image repository: Randomized controlled trial JMIR Dermatol 2023 6 e48357 doi:10.2196/48357 37624707
18 Darmawan CC Jo G Montenegro SE Kwak Y Cheol L Cho KH Early detection of acral melanoma: A review of clinical, dermoscopic, histopathologic, and molecular characteristics J Am Acad Dermatol 2019 81 805 12 30731177
19 Winkler JK Haenssle HA Artificial intelligence-based classification for the diagnostics of skin cancer Dermatologie (Heidelb) 2022 73 838 44 36094608
20 Garbe C Amaral T Peris K Hauschild A Arenberger P Bastholt L European consensus-based interdisciplinary guideline for melanoma. Part 2: Treatment–update 2019 Eur J Cancer 2020 126 159 77 31866016
21 Börve A Terstappen K Sandberg C Paoli J Mobile teledermoscopy—there's an app for that! Dermatol Pract Concept 2013 3 41
22 Sies K Winkler JK Fink C Bardehle F Toberer F Buhl T Past and present of computer-assisted dermoscopic diagnosis: Performance of a conventional image analyser versus a convolutional neural network in a prospective data set of 1,981 skin lesions Eur J Cancer 2020 135 39 46 32534243
23 Dick V Sinz C Mittlböck M Kittler H Tschandl P Accuracy of computer-aided diagnosis of melanoma: A meta-analysis JAMA Dermatol 2019 155 1291 9 31215969
24 Snoswell CL Caffery LJ Whitty JA Soyer HP Gordon LG Cost-effectiveness of skin cancer referral and consultation using teledermoscopy in Australia JAMA Dermatol 2018 154 694 700 29801161
