Three-Dimensional Deep Noninvasive Radiomics for the Prediction of Disease Control in Patients With Metastatic Urothelial Carcinoma treated With Immunotherapy

Published:March 18, 2021DOI:


      • To predict disease control to immunotherapy in 42 patients with metastatic urothelial cancer.
      • We investigated an innovative approach based a 3-dimensional deep radiomics pipeline.
      • The predictive accuracy of the pipeline was 82.5% by the baseline computed tomography images.
      • The predictive accuracy increased to 92.5% by adding other baseline clinical factors.
      • The 3-dimensional deep radiomics pipeline deserves validation in larger series.



      Immunotherapy is effective in a small percentage of patients with cancer and no reliable predictive biomarkers are currently available. Artificial Intelligence algorithms may automatically quantify radiologic characteristics associated with disease response to medical treatments.


      We investigated an innovative approach based on a 3-dimensional (3D) deep radiomics pipeline to classify visual features of chest-abdomen computed tomography (CT) scans with the aim of distinguishing disease control from progressive disease to immune checkpoint inhibitors (ICIs). Forty-two consecutive patients with metastatic urothelial cancer had progressed on first-line platinum-based chemotherapy and had baseline CT scans at immunotherapy initiation. The 3D-pipeline included self-learned visual features and a deep self-attention mechanism. According to the outcome to the ICIs, a 3D deep classifier semiautomatically categorized the most discriminative region of interest on the CT scans.


      With a median follow-up of 13.3 months (95% CI, 11.1-15.6), the median overall survival was 8.5 months (95% CI, 3.1-13.8). According to disease response to immunotherapy, the median overall survival was 3.6 months (95% CI, 2.0-5.2) for patients with progressive disease; it was not yet reached for those with disease control. The predictive accuracy of the 3D-pipeline was 82.5% (sensitivity 96%; specificity, 60%). The addition of baseline clinical factors increased the accuracy to 92.5% by improving specificity to 87%; the accuracy of other architectures ranged from 72.5% to 90%.


      Artificial Intelligence by 3D deep radiomics is a potential noninvasive biomarker for the prediction of disease control to ICIs in metastatic urothelial cancer and deserves validation in larger series.

      Graphical abstract


      To read this article in full you will need to make a payment


        • Ferlay J
        • Soerjomataram I
        • Dikshit R
        • et al.
        Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012.
        Int J Cancer. 2015; 136: E359-E386
        • Vaidya A
        • Soloway MS
        • Hawke C
        • Tiguert R
        • Civantos F.
        De novo muscle invasive bladder cancer: is there a change in trend?.
        J Urol. 2001; 165 (discussion 50): 47-50
        • De Santis M
        • Bellmunt J
        • Mead G
        • et al.
        Randomized phase II/III trial assessing gemcitabine/carboplatin and methotrexate/carboplatin/vinblastine in patients with advanced urothelial cancer who are unfit for cisplatin-based chemotherapy: EORTC study 30986.
        J Clin Oncol. 2012; 30: 191-199
        • Seront E
        • Machiels JP.
        Molecular biology and targeted therapies for urothelial carcinoma.
        Cancer Treatment Rev. 2015; 41: 341-353
        • Grande E
        • Galsky M
        • Arranz Arija JA
        • et al.
        LBA14_PR - IMvigor130: efficacy and safety from a phase III study of atezolizumab (atezo) as monotherapy or combined with platinum-based chemotherapy (PBC) vs placebo + PBC in previously untreated locally advanced or metastatic urothelial carcinoma (mUC).
        Ann Oncol. 2019; 30: v888-v889
        • Powles T
        • Park SH
        • Voog E
        • et al.
        Maintenance avelumab + best supportive care (BSC) versus BSC alone after platinum-based first-line (1L) chemotherapy in advanced urothelial carcinoma (UC): JAVELIN Bladder 100 phase III interim analysis.
        J Clin Oncol. 2020; 38: LBA1
        • Powles T
        • Duran I
        • van der Heijden MS
        • et al.
        Atezolizumab versus chemotherapy in patients with platinum-treated locally advanced or metastatic urothelial carcinoma (IMvigor211): a multicentre, open-label, phase 3 randomised controlled trial.
        Lancet. 2018; 391: 748-757
        • Bellmunt J
        • de Wit R
        • Vaughn DJ
        • et al.
        Pembrolizumab as second-line therapy for advanced urothelial carcinoma.
        N Engl J Med. 2017; 376: 1015-1026
        • Sternberg CN
        • Loriot Y
        • James N
        • et al.
        Primary results from SAUL, a multinational single-arm safety study of atezolizumab therapy for locally advanced or metastatic urothelial or nonurothelial carcinoma of the urinary tract.
        Eur Urol. 2019; 76: 73-81
        • Balar AV
        • Galsky MD
        • Rosenberg JE
        • et al.
        Atezolizumab as first-line treatment in cisplatin-ineligible patients with locally advanced and metastatic urothelial carcinoma: a single-arm, multicentre, phase 2 trial.
        Lancet. 2017; 389: 67-76
        • Balar AV
        • Dreicer R
        • Loriot Y
        • et al.
        Atezolizumab (atezo) in first-line cisplatin-ineligible or platinum-treated locally advanced or metastatic urothelial cancer (mUC): long-term efficacy from phase 2 study IMvigor210.
        J Clin Oncol. 2018; 36: 4523
        • Balar AV
        • Castellano D
        • O'Donnell PH
        • et al.
        First-line pembrolizumab in cisplatin-ineligible patients with locally advanced and unresectable or metastatic urothelial cancer (KEYNOTE-052): a multicentre, single-arm, phase 2 study.
        Lancet Oncol. 2017; 18: 1483-1492
        • Vuky J
        • Balar AV
        • Castellano DE
        • et al.
        Updated efficacy and safety of KEYNOTE-052: a single-arm phase 2 study investigating first-line pembrolizumab (pembro) in cisplatin-ineligible advanced urothelial cancer (UC).
        J Clin Oncol. 2018; 36: 4524
        • Sharma P
        • Bono P
        • Kim JW
        • et al.
        Efficacy and safety of nivolumab monotherapy in metastatic urothelial cancer (mUC): results from the phase I/II CheckMate 032 study.
        J Clin Oncol. 2016; 34: 4501
        • Massard C
        • Gordon MS
        • Sharma S
        • et al.
        Safety and efficacy of durvalumab (MEDI4736), an anti-programmed cell death ligand-1 immune checkpoint inhibitor, in patients with advanced urothelial bladder cancer.
        J Clin Oncol. 2016; 34: 3119-3125
        • Apolo AB
        • Infante JR
        • Balmanoukian A
        • et al.
        Avelumab, an anti–programmed death-ligand 1 antibody, in patients with refractory metastatic urothelial carcinoma: results from a multicenter, phase Ib study.
        J Clin Oncol. 2017; 35: 2117-2124
        • Aggen DH
        • Drake CG.
        Biomarkers for immunotherapy in bladder cancer: a moving target.
        J Immunother Cancer. 2017; 5: 94
        • Banna GL
        • Passiglia F
        • Colonese F
        • et al.
        Immune-checkpoint inhibitors in non-small cell lung cancer: a tool to improve patients' selection.
        Crit Rev Oncol Hematol. 2018; 129: 27-39
        • Paratore S
        • Banna GL
        • D'Arrigo M
        • et al.
        CXCR4 and CXCL12 immunoreactivities differentiate primary non-small-cell lung cancer with or without brain metastases.
        Cancer Biomarkers A Dis Markers. 2011; 10: 79-89
        • Rundo F
        • Conoci S
        • Banna GL
        • Ortis A
        • Stanco F
        • Battiato S.
        Evaluation of Levenberg-Marquardt neural networks and stacked autoencoders clustering for skin lesion analysis, screening and follow-up.
        IET Computer Vision. 2018; 12: 957-962
        • Addeo A
        • Banna GL
        • Weiss GJ.
        Tumor mutation burden-from hopes to doubts.
        JAMA Oncol. 2019;
        • Banna GL
        • Di Quattro R
        • Malatino L
        • et al.
        Neutrophil-to-lymphocyte ratio and lactate dehydrogenase as biomarkers for urothelial cancer treated with immunotherapy.
        Clin Transl Oncol. 2020; 22: 2130-2135
        • McGranahan N
        • Swanton C.
        Clonal heterogeneity and tumor evolution: past, present, and the future.
        Cell. 2017; 168: 613-628
        • Banna GL
        • Olivier T
        • Rundo F
        • et al.
        The promise of digital biopsy for the prediction of tumor molecular features and clinical outcomes associated with immunotherapy.
        Front Med. 2019; 6: 172
        • Wang X
        • Girshick R
        • Gupta A
        • He K.
        Non-local neural networks.
        in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018: 7794-7803
        • Cha KH
        • Hadjiiski L
        • Samala RK
        • Chan HP
        • Caoili EM
        • Cohan RH.
        Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets.
        Med Phys. 2016; 43: 1882
        • Eisenhauer EA
        • Therasse P
        • Bogaerts J
        • et al.
        New Response Evaluation Criteria in Solid Tumours: revised RECIST guideline (version 1.1).
        Eur J Cancer. 2009; 45: 228-247
        • Rundo F
        • Spampinato C
        • Banna GL
        • Conoci S.
        Advanced deep learning embedded motion radiomics pipeline for predicting anti-PD-1/PD-L1 immunotherapy response in the treatment of bladder cancer: preliminary results.
        Electronics. 2019; 8: 1134
        • Chua LO
        • Yang L.
        Cellular neural networks: theory.
        IEEE Trans Circuits Systems. 1988; 35: 1257-1272
        • Chua LO
        • Yang L.
        Cellular neural networks: applications.
        IEEE Trans Circuits Systems. 1988; 35: 1273-1290
        • Yang L-B
        • Chua LO
        • Krieg KR.
        VLSI implementation of cellular neural networks.
        IEEE Trans Circuits Systems. 1990; 2423: 2425-2427
        • Krieg KR
        • Chua LO
        • Yang L-B.
        Analog signal processing using cellular neural networks.
        IEEE Trans Circuits Systems. 1990; 952: 958-961
        • Rundo FO A.
        • Battiato S.
        • Conoci S.
        Advanced bio-inspired system for noninvasive cuff-less blood pressure estimation from physiological signal analysis.
        Computation. 2018; 6: 46
        • Battiato S
        • Rundo F
        • Stanco F.
        Self organizing motor maps for color-mapped image re-indexing.
        IEEE Transactions on Image Processing. 2007; 16: 2905-2915
        • Williams H
        • Cattani L
        • Li W
        • et al.
        3D convolutional neural network for segmentation of the urethra in volumetric ultrasound of the pelvic floor.
        in: 2019 IEEE International Ultrasonics Symposium (IUS). 2019: 1473-1476
        • Vaswani A
        • Shazeer N
        • Parmar N
        • et al.
        Attention is all you need.
        Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., Long Beach, CA2017: 6000-6010
        • Havel JJ
        • Chowell D
        • Chan TA.
        The evolving landscape of biomarkers for checkpoint inhibitor immunotherapy.
        Nat Rev Cancer. 2019; 19: 133-150
        • Addeo A
        • Banna GL
        • Metro G
        • Di Maio M.
        Chemotherapy in combination with immune checkpoint inhibitors for the first-line treatment of patients with advanced non-small cell lung cancer: a systematic review and literature-based meta-analysis.
        Front Oncol. 2019; 9: 264
        • Moradi P
        • Jamzad M.
        Detecting lung cancer lesions in CT images using 3D convolutional neural networks.
        in: 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA). 2019: 114-118
        • Dromain C
        • Beigelman C
        • Pozzessere C
        • Duran R
        • Digklia A.
        Imaging of tumour response to immunotherapy.
        Eur Radiol Exp. 2020; 4: 2
        • Alsaab HO
        • Sau S
        • Alzhrani R
        • et al.
        PD-1 and PD-L1 checkpoint signaling inhibition for cancer immunotherapy: mechanism, combinations, and clinical outcome.
        Front Pharmacol. 2017; 8: 561
        • Zhou TC
        • Sankin AI
        • Porcelli SA
        • Perlin DS
        • Schoenberg MP
        • Zang X.
        A review of the PD-1/PD-L1 checkpoint in bladder cancer: from mediator of immune escape to target for treatment11MPS is an investor in and consultant for Urogen. SAP is consultant and advisor for Vaccinex. The remaining authors have nothing to disclose.
        Urol Oncol. 2017; 35: 14-20
        • Chardin D
        • Paquet M
        • Schiappa R
        • et al.
        Baseline metabolic tumor volume as a strong predictive and prognostic biomarker in patients with non-small cell lung cancer treated with PD1 inhibitors: a prospective study.
        J Immunother Cancer. 2020; 8e000645
        • Butner JD
        • Elganainy D
        • Wang CX
        • et al.
        Mathematical prediction of clinical outcomes in advanced cancer patients treated with checkpoint inhibitor immunotherapy.
        Sci Adv. 2020; 6: eaay6298