A deep multiple instance learning framework improves microsatellite instability detection from tumor next generation sequencing

Authors

John Ziegler

Jaclyn F Hechtman

Satshil Rana

Ryan N Ptashkin

Gowtham Jayakumaran

Sumit Middha

Shweta S Chavan

Chad Vanderbilt

Deborah DeLair

Jacklyn Casanova

Jinru Shia

Nicole DeGroat

Ryma Benayed

Marc Ladanyi

Michael F Berger

Thomas J Fuchs

A Rose Brannon

Ahmet Zehir

Doi

10.1038/s41467-024-54970-z

PMID: 39746944 · DOI: 10.1038/s41467-024-54970-z · Journal: Nature Communications (2025)

TL;DR

Ziegler et al. develop MiMSI, a deep multiple-instance learning (MIL) classifier that detects microsatellite instability (MSI) from targeted next-generation sequencing reads at microsatellite loci. Trained on 741 MSK-IMPACT cases (including deliberately difficult low-purity / low-coverage samples) and evaluated on a held-out test set of 317 samples plus a prospective cohort of 5,037 tumors with orthogonal MMR IHC, MiMSI achieved sensitivity 0.895 and auROC 0.971 — outperforming MSISensor (sensitivity 0.67, auROC 0.907). The advantage was largest for tumors with <30% purity (MiMSI 85.1% vs MSISensor 72.8%, McNemar’s chi-squared P=8.244 × 10⁻⁷). On a global comparison across 45,112 prospectively sequenced MSK-IMPACT tumors, MiMSI cut the indeterminate rate from 3.8% to 0.47% and showed 96% concordance with MSISensor for definitive MSS/MSI-H calls. The training/test microsatellite vectors and the secondary cohort somatic data are released as the cBioPortal study pancan_mimsi_msk_2024. PMID:39746944

Cohort & data

  • Training/test cohort (n=1,058): orthogonally labeled by MMR IHC and/or MSI-PCR, deliberately enriched for difficult cases (low purity, low coverage, or discordant MMR status). Split into 741 training samples (396 MSS, 345 MSI-H) and 317 held-out test samples (160 MSS, 157 MSI-H). PMID:39746944
  • Prospective secondary cohort (n=5,037) from 42 cancer types with paired MMR IHC: 4,195 MMR-proficient and 842 MMR-deficient (580 MLH1 loss, 166 MSH2 loss, 60 MSH6 loss, 36 PMS2 loss). Released as cBioPortal study pancan_mimsi_msk_2024. PMID:39746944
  • Global MSK-IMPACT comparison cohort (n=45,112 samples, 40,414 consented patients): prospective clinical sequencing between January 2014 – April 2020 with the MSK-IMPACT panel (clinical pipeline from PMID:28481359, see msk_impact_2017). Concordance analysis between MSISensor and MiMSI. PMID:39746944
  • WES validation set (n=582): NGS libraries from a subset of the test cohort were re-captured with whole-exome sequencing probes and reanalyzed with MiMSI. PMID:39746944
  • Cancer types most represented in the prospective cohort sensitivity analysis: Colorectal Cancer (COAD) (n=2,448), Endometrial Cancer (UCEC) (n=1,212), Esophagogastric Carcinoma (EGC) (n=475), Cancer of Unknown Primary (CUP) (n=114), Bladder Cancer (BLCA) (n=73), Small Bowel Cancer (SBC) (n=60), Prostate Cancer (PRAD) (n=55). PMID:39746944
  • Assay: targeted hybridization-capture sequencing on MSK-IMPACT (panels including IMPACT468); typical mean coverage ~600×, downsampled to 100×, 200×, 300×, 400× for MiMSI vector inputs. Microsatellite locus list: 1,755 loci covered by MSK-IMPACT (generated using msisensor scan v0.2). PMID:39746944
  • Orthogonal reference: MMR immunohistochemistry confirmed by a board-certified pathologist; MSI-PCR. Ground-truth labels at the bag (sample) level. PMID:39746944
  • Software: PyTorch implementation, Adam optimizer, ResNet-style CNN feature extractor, MIL pooling (mean or attention), sigmoid classifier with threshold 0.4 and 95% CI from 10 random downsamples. Released under GPLv3 at https://github.com/mskcc/mimsi (Zenodo DOI 10.5281/zenodo.13357948). PMID:39746944

Key findings

  • MiMSI vs MSISensor on the held-out test cohort (n=317): MSISensor — accuracy 0.835, sensitivity 0.67, specificity 1.00, auROC 0.907. MiMSI 200× attention model — accuracy 0.942, sensitivity 0.895 (95% CI 0.884–0.905), specificity 0.988, auROC 0.971. The 100× and 200× models were the top performers; the 200× attention model was used for downstream analyses. PMID:39746944
  • Threshold and indeterminacy rule: cut-off probability 0.4 for MSS vs MSI-H; cases whose 95% CI crosses 0.4 are MSI-indeterminate. CI is computed from 10 random microsatellite-downsampling runs per sample. PMID:39746944
  • Discordant cases on test cohort (17/317, 5%): 15 orthogonally MSI-H but called MSS by MiMSI; 1 indeterminate (Sample_18831, low-purity endometrial cancer with 27 mutations and median VAF 5%); 2 orthogonally MSS but called MSI-H by MiMSI. Two of the 15 false-negatives carried a POLE exonuclease-domain deficient mutational signature with associated high TMB; among the false-positives, Sample_54409 was a colon adenocarcinoma with 303 somatic mutations dominated by POLE deficiency (V411L, 72% of mutations) plus 15% MMR-attributable mutations and an MSH2 E580* nonsense variant. PMID:39746944
  • Sample dilution experiment (Supplementary Table 2): an MSI-H tumor diluted with matched normal DNA. MSISensor score dropped from 36.7 and fell below the MSI-H threshold of 10 at the 6% dilution; MiMSI continued to classify the sample as MSI-H even at the lowest dilution point. PMID:39746944
  • Effect of microsatellite-loci downsampling: MSI-indeterminate rate increased from 1.2% (900 loci used) to 12% (100 loci used), demonstrating that the number of evaluable microsatellite sites is a primary driver of confidence. PMID:39746944
  • Prospective cohort (n=5,037), MMR-D vs MMR-P genomic features: TMB significantly higher in MMR-D (median 39 mut/Mb vs MMR-P median 5.27 mut/Mb, Mann-Whitney P<2.2 × 10⁻¹⁶). Among MMR-D subgroups, MSH2 loss had higher TMB than MLH1 loss (median 46.5 vs 37.7, P=0.0013). PMID:39746944
  • Indel/SNV ratio: median 0.18 in MMR-P vs 0.5 in MMR-D (Mann-Whitney P<2.2 × 10⁻¹⁶). MLH1 loss had the highest indel/SNV ratio (median 0.57); MSH6 loss had the lowest (median 0.09). PMID:39746944
  • MMR mutational-signature contribution (samples with ≥15 mutations, n=898): median MMR-signature contribution 0.036 in MMR-P vs 0.62 in MMR-D (P<2.2 × 10⁻¹⁶). MSH6 loss → median 0.42 (lowest); MLH1 loss → median 0.67 (highest). PMID:39746944
  • Sensitivity in the prospective cohort: MiMSI 91.6% (95% CI 89.5–93.4%) vs MSISensor 86.1% (83.3–88.6%). MSISensor failed to report 118 cases (2.3%) due to low coverage / purity. Among MSISensor-indeterminate cases (n=247), MiMSI correctly classified 226 (91%). PMID:39746944
  • Tumor-purity stratified sensitivity: ≥30% purity — both methods comparable; <30% purity (n=2,238) — MiMSI 85.1% (95% CI 81.0–88.5%) vs MSISensor 72.8% (67.3–77.8%), McNemar’s chi-squared P=8.244 × 10⁻⁷. PMID:39746944
  • Per-cancer-type sensitivity advantage for MiMSI: Small Bowel Cancer (100% vs 94.1%), Prostate Cancer (93.8% vs 85.7%), Endometrial Cancer (89.7% vs 79%); comparable in Bladder, Colorectal, and Esophagogastric cancers; in Cancer of Unknown Primary MSISensor was nominally higher (92.3% vs 87.5%, overlapping CIs). PMID:39746944
  • Global MSK-IMPACT concordance (n=45,112): 96% concordance for MSS and MSI-H calls; MiMSI reduced indeterminate calls from 3.8% (n=1,724 by MSISensor) to 0.47% (n=210). Confusion table: 42,059 MSS/MSS, 1,027 MSI-H/MSI-H; 215 MSI-H by MiMSI but indeterminate by MSISensor; 1,459 MSS by MiMSI but indeterminate by MSISensor. PMID:39746944
  • WES out-of-the-box transfer (n=582): 98.6% concordance with MSK-IMPACT classifications; only 5 discordant cases. MiMSI generalized to a different capture without retraining. PMID:39746944
  • Tumor-only mode: with an unrelated normal as comparator, attention pooling produced many false-positive MSI-H calls (likely driven by ancestral differences amplified by attention weights). Replacing the unrelated normal with a pooled control (equimolar mix of 10 blood samples) and using mean (no-attention) pooling produced the best balance; authors caution that thresholds should be re-validated for tumor-only deployment. PMID:39746944

Genes & alterations

  • MLH1 — loss in 580/842 MMR-D tumors in the prospective cohort; associated with the highest indel/SNV ratio (median 0.57) and highest MMR-signature contribution (median 0.67) among MMR proteins. PMID:39746944
  • MSH2 — loss in 166/842 MMR-D tumors; significantly higher TMB than MLH1 loss (median 46.5 vs 37.7 mut/Mb, P=0.0013). One false-positive MSI-H discordant case (Sample_54409, colon adenocarcinoma) carried an MSH2 E580* nonsense variant alongside a dominant POLE signature. PMID:39746944
  • MSH6 — loss in 60/842 MMR-D tumors; lowest indel/SNV ratio (median 0.09) and lowest MMR-signature contribution (median 0.42) among MMR proteins. PMID:39746944
  • PMS2 — loss in 36/842 MMR-D tumors; intermediate genomic features, with TMB not significantly different from MSH6 loss (P=0.62) or MLH1 loss (P=0.96). PMID:39746944
  • POLE — exonuclease-domain mutations (e.g. V411L) explain several discordant MiMSI calls: two MiMSI false-negatives in the test cohort had a POLE-deficient mutational signature plus high TMB; one MiMSI false-positive (Sample_54409) attributed 72% of mutations to POLE deficiency. The authors highlight that MiMSI can flag MMR phenotype even when it co-occurs with POLE-driven hypermutation. PMID:39746944

Clinical implications

  • Biomarker for immune checkpoint inhibition: the FDA has approved pembrolizumab for MSI-H / MMR-D solid tumors of any histology, making robust pan-cancer MSI calling on routine clinical NGS a direct treatment-selection tool. MiMSI extends the eligible patient pool by recovering cases that MSISensor miscalls due to low tumor purity. PMID:39746944
  • Reduces orthogonal-testing burden: by collapsing the indeterminate rate from 3.8% to 0.47% in the 45,112-sample MSK-IMPACT cohort, MiMSI reduces the need for follow-up MMR IHC or MSI-PCR on indeterminate NGS results — relevant for tissue-limited specimens. PMID:39746944
  • Lynch-syndrome screening: MSI-H detection is predictive of Lynch syndrome regardless of primary tumor site (citing PMID:30376427 for the pan-cancer Lynch–MSI relationship); a more sensitive NGS classifier widens the screening net, especially in low-purity FFPE samples typical of routine practice. PMID:39746944
  • Cross-platform portability: 98.6% concordance between MSK-IMPACT and WES on matched DNA libraries supports deploying the same trained model across capture designs without retraining, lowering the bar for adoption at sites already running WES. PMID:39746944
  • POLE-MMR co-occurrence: MiMSI flags MMR phenotype even when paired with POLE-ultramutator phenotype, where TMB-based heuristics may obscure MMR contribution; clinically relevant for endometrial and colon cancer subgroups where dual POLE+MMR deficiency occurs. PMID:39746944

Limitations & open questions

  • Single-institution training data: model trained entirely on MSK-IMPACT samples from MSKCC; authors explicitly note potential single-institution bias and ship a re-training tool with the package. External-site validation on independent NGS pipelines is open. PMID:39746944
  • Tumor-only deployment is fragile: with an unrelated normal comparator, attention-pooled MiMSI over-calls MSI-H, likely from ancestral SNP differences. Best results required swapping in a pooled normal and dropping attention; this configuration must be re-validated and the 0.4 threshold re-tuned per dataset. PMID:39746944
  • MMR IHC ground truth has its own ~94% sensitivity ceiling (with documented retained-MMR-protein cases driven by missense MMR mutations, PMID:32198466), limiting the ceiling of any classifier evaluated against it. PMID:39746944
  • Interpretability of attention scores for individual microsatellite loci is shown qualitatively but not validated against any biological prior — the attention map is a model artifact, not a biological feature ranking. PMID:39746944
  • Indeterminate cases with very low locus counts: when fewer than ~200 microsatellite sites pass coverage, indeterminate rate climbs to >12%; deployments on smaller panels than IMPACT468 may need supplementary loci or different thresholds. PMID:39746944
  • Raw sequencing data unavailable (patient-consent restrictions); only the precomputed three-dimensional microsatellite vectors and the secondary cohort somatic alterations (cBioPortal study pancan_mimsi_msk_2024) are public. Independent reanalysis of read-level features is not possible without an institutional data-transfer agreement. PMID:39746944

Citations from this paper used in the wiki

  • “MiMSI showed higher sensitivity (0.895) and auROC (0.971) than MSISensor (sensitivity: 0.67; auROC: 0.907)…” (Abstract) PMID:39746944
  • “In a separate, prospective cohort, MiMSI confirmed that it outperforms MSISensor in low purity cases (P=8.244e-07).” (Abstract) PMID:39746944
  • “Of the 5037 samples, 842 were MMR-D (580 with MLH1 loss, 166 with MSH2 loss, 60 with MSH6 loss, and 36 with PMS2 loss…).” (Results — Comprehensive features of MMR-D cancers) PMID:39746944
  • “Loss of MMR machinery leads to the formation of indels as opposed to single nucleotide variations (SNVs) … median indel/SNV in MMR-P=0.18, median indel/SNV in MMR-D=0.5, Mann-Whitney Wilcoxon p-value=<2.2x10−16.” (Results) PMID:39746944
  • “Comparison of MiMSI results with MSISensor showed an overall 96% concordance for cases identified as MSS and MSI-H with both methods … MiMSI reduced the number of samples in the MSI-Ind category identified by MSISensor from 3.8% (n=1724) to 0.47% (n=210).” (Results — Global comparison) PMID:39746944
  • “WES results were highly concordant with MSK-IMPACT classifications (overall concordance: 98.6%) with a total of 5 discordant cases.” (Results — WES analysis) PMID:39746944
  • “Clinical and somatic alteration data for the secondary cohort are publicly available on cBioPortal: https://www.cbioportal.org/study/summary?id=pancan_mimsi_msk_2024.” (Data availability) PMID:39746944
  • “The software, along with the fully trained model is hosted at https://github.com/mskcc/mimsi (https://doi.org/10.5281/zenodo.13357948).” (Code availability) PMID:39746944

This page was processed by crosslinker on 2026-05-04.