Assessment of a Diagnostic Classification System for Management of Lesions to Exclude Melanoma

IMPORTANCE: The proposed MOLEM (Management of Lesion to Exclude Melanoma) schema is more clinically relevant than Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MATH-Dx) for the management classification of melanocytic and nonmelanocytic lesions excised to exclude melanoma. A more standardized way of establishing diagnostic criteria will be crucial in the training of artificial intelligence (AI) algorithms. OBJECTIVE: To examine pathologists’ variability, reliability, and confidence in reporting melanocytic and nonmelanocytic lesions excised to exclude melanoma using the MOLEM schema in a population of higher-risk patients. DESIGN, SETTING, AND PARTICIPANTS: This cohort study enrolled higher-risk patients referred to a primary care skin clinic in New South Wales, Australia, between April 2019 and December 2019. Baseline demographic characteristics including age, sex, and related clinical details (eg, history of melanoma) were collected. Patients with lesions suspicious for melanoma assessed by a primary care physician underwent clinical evaluation, dermoscopy imaging, and subsequent excision biopsy of the suspected lesion(s). A total of 217 lesions removed and prepared by conventional histologic method and stained with hematoxylin-eosin were reviewed by up to 9 independent pathologists for diagnosis using the MOLEM reporting schema. Pathologists evaluating for MOLEM schema were masked to the original histopathologic diagnosis. MAIN OUTCOMES AND MEASURES: Characteristics of the lesions were described and the concordance of cases per MOLEM class was assessed. Interrater agreement and the agreement between pathologists’ ratings and the majority MOLEM diagnosis were calculated by Gwet AC1 with quadratic weighting applied. The diagnostic confidence of pathologists was then assessed. RESULTS: A total of 197 patients were included in the study (102 [51.8%] male; 95 [48.2%] female); mean (SD) age was 64.2 (15.8) years (range, 24-93 years). Overall, 217 index lesions were assessed with a total of 1516 histological diagnoses. Of 1516 diagnoses, 677 (44.7%) were classified as MOLEM class I; 120 (7.9%) as MOLEM class II; 564 (37.2%) as MOLEM class III; 114 (7.5%) as MOLEM class IV; and 55 (3.6%) as MOLEM class V. Concordance rates per MOLEM class were 88.6% (class I), 50.8% (class II), 76.2% (class III), 77.2% (class IV), and 74.2% (class V). The quadratic weighted interrater agreement was 91.3%, with a Gwet AC1 coefficient of 0.76 (95% CI, 0.72-0.81). The quadratic weighted agreement between pathologists’ ratings and majority MOLEM was 94.7%, with a Gwet AC1 coefficient of 0.86 (95% CI, 0.84-0.88). The confidence in diagnosis data showed a relatively high level of confidence (between 1.0 and 1.5) when diagnosing classes I (mean [SD], 1.3 [0.3]), IV (1.3 [0.3]) and V (1.1 [0.1]); while classes II (1.8 [0.2]) and III (1.5 [0.4]) were diagnosed with a lower level of pathologist confidence (≥1.5). The quadratic weighted interrater confidence rating agreement was 95.2%, with a Gwet AC1 coefficient of 0.92 (95% CI, 0.90-0.94) for the 1314 confidence ratings collected. The confidence agreement for each MOLEM class was 95.0% (class I), 93.5% (class II), 95.3% (class III), 96.5% (class IV), and 97.5% (class V). CONCLUSIONS AND RELEVANCE: The proposed MOLEM schema better reflects clinical practice than the MPATH-Dx schema in lesions excised to exclude melanoma by combining diagnoses with similar prognostic outcomes for melanocytic and nonmelanocytic lesions into standardized classification categories. Pathologists’ level of confidence appeared to follow the MOLEM schema diagnostic concordance trend, ie, atypical naevi and melanoma in situ diagnoses were the least agreed upon and the most challenging for pathologists to confidently diagnose.