Published on in Vol 8, No 1 (2022): Jan-Dec

Preprints (earlier versions) of this paper are available at, first published .
Interobserver and Human–Artificial Intelligence Concordance in Differentiating Between Invasive and In Situ Melanoma

Interobserver and Human–Artificial Intelligence Concordance in Differentiating Between Invasive and In Situ Melanoma

Interobserver and Human–Artificial Intelligence Concordance in Differentiating Between Invasive and In Situ Melanoma

Authors of this article:

Sam Polesie 1, 2 Author Orcid Image ;   John Paoli 1, 2 Author Orcid Image


1Department of Dermatology and Venereology, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden

2Department of Dermatology and Venereology, Sahlgrenska University Hospital, Region Västra Götaland, Gothenburg, Sweden

Corresponding Author:

Sam Polesie, MD, PhD

Department of Dermatology and Venereology

Institute of Clinical Sciences, Sahlgrenska Academy

University of Gothenburg

Medicinaregatan 3

Gothenburg, 41390


Phone: 46 31 786 00 00


Background: Machine learning algorithms including convolutional neural networks (CNNs) have recently made significant advances in research settings. Even though several algorithms nowadays are targeted directly to the consumer market, their implementation in clinical practice is still pending. Most melanomas are easy to recognize even without the aid of dermoscopy. Nonetheless, it is often more challenging to discriminate between invasive melanoma and melanoma in situ (MIS) in a preoperative setting even with the assistance of dermoscopy. Although several dermoscopic features suggestive of MIS and invasive melanomas have been presented, their usefulness in a larger setting is limited by how well physicians agree on their presence or absence.

Objective: The overarching aims of this research project are to identify useful dermoscopic features to help dermatologists predict melanoma thickness and to develop CNNs that can assist dermatologists in the preoperative assessment of melanoma thickness. The ultimate aim is to develop algorithms that can strengthen patient care, improve clinical decision-making, and be used in routine health care.

Methods: We have included dermoscopic images as well as clinical close-up images of invasive melanomas and MIS from our department during the time period of January 1, 2016, to December 31, 2020. Using this image material, we have trained, validated, and tested two separated CNNs based on dermoscopic and clinical close-up images. We have also invited dermatologists to review the test sets and, for a subset of the dermoscopic images, asked them to specify the presence of prespecified dermoscopic features. Subsequently, we compared CNN outputs to the combined dermatologists’ output for all lesions and assessed the interobserver agreement for several dermoscopic features.

Results: The CNN developed using dermoscopic images performed on par with the invited dermatologists whereas the CNN using clinical close-up images was outperformed by the group of dermatologists. Two dermoscopic features (atypical blue-white structures and shiny white lines) both displayed a moderate to substantial interobserver agreement and were both indicative of invasive melanomas >1.0 mm.

Conclusions: CNNs used to differentiate between invasive melanomas and MIS might be an example of a clinically relevant machine learning application, but they need further refinement and evaluation in prospective clinical trials. Only a few dermoscopic features are helpful in distinguishing melanoma thickness.

Conflicts of Interest: None declared.

iproc 2022;8(1):e36895



Edited by T Derrick; This is a non–peer-reviewed article. submitted 28.01.22; accepted 28.01.22; published 08.02.22


©Sam Polesie, John Paoli. Originally published in Iproceedings (, 08.02.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Iproceedings, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.