Advertisement

Benefit of Metastasectomy in Renal Cell Carcinoma: A Propensity Score Analysis

Open AccessPublished:March 24, 2022DOI:https://doi.org/10.1016/j.clgc.2022.03.010

      Abstract

      Introduction

      To quantify the magnitude of benefit of metastasectomy as compared to medical treatment alone in patients with metastatic renal cell carcinoma (mRCC).

      Patients and Methods

      We therefore conducted a propensity score analysis of overall survival (OS) in 106 mRCC patients with metachronous metastasis, of whom 36 (34%) were treated with metastasectomy, and 70 (66%) with medical therapy alone.

      Results

      The most frequent metastasectomy procedures were lung resections (n = 13) and craniotomies (n = 6). Median time-to-progression after metastasectomy was 0.7 years (25th-75th percentile: 0.3-2.7). After a median follow-up of 6.2 years and 63 deaths, 5-year OS estimates were 41% and 22% in the metastasectomy and medical therapy group, respectively (log-rank P = .00007; Hazard ratio (HR) = 0.38, 95%CI: 0.21-0.68). Patients undergoing metastasectomy had a significantly higher prevalence of favorable prognostic factors, such as fewer bilateral lung metastases and longer disease-free intervals between nephrectomy and metastasis diagnosis. After propensity score weighting for these differences and adjusting for immortal time bias, the favorable association between metastasectomy and OS became much weaker (HR = 0.62, 95%CI: 0.39-1.00, P = .050). Propensity-score-weighted 5-year OS estimates were 24% and 20% in the metastasectomy and medical therapy group, respectively (log-rank P = .001). In exploratory analyses, the benefit of metastasectomy was confined to patients who achieved complete resection of all known metastases.

      Conclusion

      Within the limitations of an observational study, these findings support the concept of metastasectomy being associated with an OS benefit in mRCC patients. Metastasectomies not achieving complete resection of all known lesions are likely without OS benefit.

      Keywords

      Introduction

      Systemic antineoplastic therapies such as tyrosine kinase inhibitors and immune checkpoint inhibitors have significantly improved outcomes of patients with metastatic renal cell carcinoma (mRCC).
      • Rini BI
      • Plimack ER
      • Stus V
      • et al.
      Pembrolizumab plus Axitinib versus sunitinib for advanced renal-cell carcinoma.
      Although these treatments can induce tumor responses, delay disease progression,
      • Choueiri TK
      • Escudier B
      • Powles T
      • et al.
      Cabozantinib versus Everolimus in advanced renal cell carcinoma (METEOR): final results from a randomised, open-label, phase 3 trial.
      preserve quality-of-life,
      • Motzer RJ
      • Hutson TE
      • Cella D
      • et al.
      Pazopanib versus sunitinib in metastatic renal-cell carcinoma.
      and improve overall survival (OS),
      • Motzer RJ
      • Escudier B
      • McDermott DF
      • et al.
      Nivolumab versus Everolimus in advanced renal-cell carcinoma.
      few patients achieve long-term remissions and mRCC still remains a usually fatal disease.
      • Ko JJ
      • Xie W
      • Kroeger N
      • et al.
      The international metastatic renal cell carcinoma database consortium model as a prognostic tool in patients with metastatic renal cell carcinoma previously treated with first-line targeted therapy: a population-based study.
      Metastasectomy, ie surgery with the intent to remove most or all metastatic lesions, is a frequently practiced strategy in selected patients with mRCC.
      • van der Poel HG
      • Roukema JA
      • Horenblas S
      • van Geel AN
      • Debruyne FM.
      Metastasectomy in renal cell carcinoma: a multicenter retrospective analysis.
      Retrospective series have demonstrated that adequately selected patients can experience long-term remissions and favorable survival following metastasectomy. For example, in a meta-analysis by Zhao and colleagues pooling 16 studies on pulmonary metastasectomy in mRCC, a favorable 5-year OS estimate of 43% was observed.
      • Zhao Y
      • Li J
      • Li C
      • Fan J
      • Liu L.
      Prognostic factors for overall survival after lung metastasectomy in renal cell cancer patients: a systematic review and meta-analysis.
      Moreover, a recent systematic review of 56 retrospective studies by Ouzaid and colleagues reported a better OS after metastasectomy as compared to treatment concepts without metastasectomy.
      • Ouzaid I
      • Capitanio U
      • Staehler M
      • et al.
      Surgical metastasectomy in renal cell carcinoma: a systematic review.
      However, due to the retrospective, mostly uncontrolled, and non–comparative design of the currently available data as well as the absence of any randomized data,
      • Ouzaid I
      • Capitanio U
      • Staehler M
      • et al.
      Surgical metastasectomy in renal cell carcinoma: a systematic review.
      the magnitude of benefit of metastasectomy versus treatment without metastasectomy for overall survival in patients with mRCC remains unclear. To improve the understanding about the potential role of metastasectomy for treating patients with mRCC, more data are needed.
      Analysis of observational data is an increasingly popular way to explore potential treatment benefits in the absence of randomized data or in settings where randomized studies are infeasible for ethical or logistical reasons.
      • Moik F
      • Riedl JM
      • Winder T
      • et al.
      Benefit of second-line systemic chemotherapy for advanced biliary tract cancer: a propensity score analysis.
      However, a comparison of patients with mRCC with and without metastasectomy using retrospective data has a high inherent risk of bias, because patients are likely selected by their treating uro-oncologists for metastasectomy according to favorable prognostic criteria, such as low metastatic load, technical feasibility of resection, or good performance status. Consequently, a naive comparison of OS outcomes between mRCC patients who did, and did not undergo metastasectomy has a high risk of overestimating the “true” effect of metastasectomy in this setting. Biostatistical research has brought forward comparative effectiveness research methods such as propensity score analysis to address this problem.
      • Austin PC.
      The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments.
      In this observational study, we perform a propensity score analysis of patients with metachronous metastasis from RCC to quantify the potential benefit of metastasectomy toward OS as compared to treatment strategies without metastasectomy.

      Patients and Methods

      Study Population and Design

      In this single-center, observational study, we retrospectively ascertained baseline and outcome data for all patients who underwent either partial or total nephrectomy at the Department of Urology, Medical University of Graz, Austria, between January, 2005 and November, 2018 (n = 1190). These data were collected from our electronic health record system as previously described.
      • Posch F
      • Leitner L
      • Bergovec M
      • et al.
      Can multistate modeling of local recurrence, distant metastasis, and death improve the prediction of outcome in patients with soft tissue sarcomas?.
      • Posch F
      • Silina K
      • Leibl S
      • et al.
      Maturation of tertiary lymphoid structures and recurrence of stage II and III colorectal cancer.
      • Seles M
      • Posch F
      • Pichler GP
      • et al.
      Blood platelet volume represents a novel prognostic factor in patients with nonmetastatic renal cell carcinoma and improves the predictive ability of established prognostic scores.
      Subsequently, all patients who developed metachronous metastasis were included in the current analysis (Supplementary Paragraph 1, Supplementary Table 1, and Supplementary Figure 1). The baseline date was defined as the date of diagnosis of first metachronous metastasis. Metastasectomies were not restricted to a particular site (ie we considered any-type metastasectomy including lung, liver, bone, and others). However, interventional procedures without surgical removal of tumor masses (eg radiofrequency ablation of lung or liver metastases, palliative radiation of bone metastases,…) were not counted as metastasectomies. Primary endpoint of this study was death-from-any-cause within 5 years after the baseline date. Mortality status was obtained by central query of the Austrian Social Security Database. Data collection and analysis was approved by the local institutional review board (Ethics Committee of the Medical University of Graz, Austria; document number No. 31-082 ex 18/19, [email protected]).

      Statistical Methods

      All statistical analyses were performed using Stata 15.0 (Stata Corp., Houston, TX). Continuous variables were reported as medians (25th-75th percentile), and count data as absolute frequencies (%). Differences in means and proportions between patients with and without metastasectomy were quantified using standardized mean differences (SMDs),
      • Austin PC
      • Stuart EA.
      Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies.
      and tested with rank-sum tests, χ2-tests, and Fisher's exact tests, respectively. SMDs > 0.30 were considered to indicate relevant imbalance between patients with and without metastasectomy.
      • Austin PC
      • Stuart EA.
      Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies.
      Median follow-up times were estimated with the reverse Kaplan-Meier method.
      • Schemper M
      • Smith TL.
      A note on quantifying follow-up in studies of failure time.
      Overall survival (OS) was estimated with Kaplan-Meier estimators, and compared between patients with and without metastasectomy using log-rank tests. Moreover, landmark analyses (landmark for metastasectomy set at 6 months after metastasis diagnosis) and Mantel-Byar tests were performed to reduce immortal time bias. Uni- and multivariable modeling of the primary endpoint (5-year OS) was performed with Cox proportional hazards models. Metastasectomy was also treated as a time-dependent variable to fully eliminate immortal time bias. We estimated the propensity score with a multivariable logistic regression model for being in the metastasectomy group using a stepwise backward elimination algorithm from all variables that had either a P-value for difference between the metastasectomy and medical therapy group of ≤ .10 or a corresponding SMD ≥ 0.10 (excluding one-by-one the variables with the smallest strength of association as indicated by the t-statistic). This base model was subsequently reduced by stepwise backward elimination to a final propensity score model with a pre-specified number of 10 predictor variables. The propensity score model development was performed after multiple imputation of missing data using a chained equations algorithm with 20 imputation datasets (imputation models on file with FP).
      • White IR
      • Royston P
      • Wood AM.
      Multiple imputation using chained equations: issues and guidance for practice.
      The propensity score was then defined as the probability of undergoing metastasectomy conditional on the included predictor variables, and the inverse-probability-of-treatment-weight (IPTW) was defined as the inverse of the probability of receiving the treatment that the patient actually received.
      • Austin PC
      • Stuart EA.
      Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies.
      To ascertain whether the IPTW achieved sufficient balance between patients with and without metastasectomy, SMDs were re-estimated after weighing the data with the IPTW.
      • Austin PC
      • Stuart EA.
      Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies.
      Kaplan-Meier estimators and Cox proportional hazards models were then re-fitted after weighing for the IPTW.
      • Austin PC.
      The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments.
      The proportional hazards assumption was assessed with Schoenfeld tests and by fitting interactions between metastasectomy and linear follow-up time. Rates of death according to metastasectomy status in the presence of non–proportional hazards were estimated with a flexible parametric regression model with restricted cubic splines on the log(cumulative hazard) scale (4 degrees of freedom for the time-invariant effect and 3 degrees of freedom for the time-varying effect of metastasectomy, Stata routine stpm2).
      • Moik F
      • Riedl JM
      • Winder T
      • et al.
      Benefit of second-line systemic chemotherapy for advanced biliary tract cancer: a propensity score analysis.
      Progression risks after metastasectomy were estimated with 1-Kaplan-Meier estimators (because no competing mortality event occurred prior progression events). Effect modification between baseline variables and metastasectomy were explored in a hypothesis-generating analysis by fitting interactions between metastasectomy and the respective variable in an IPTW-weighted, immortal-time-bias-adjusted Cox model. The full analysis code is available on reasonable request from the corresponding author.

      Results

      Characteristics of the Study Population

      One-hundred-and-six patients were included in the analysis at the time of diagnosis of metachronous metastasis (Table 1). Median age of the cohort was 70 years (25th-75th percentile: 62-76), and 39 patients (37%) were female. Most patients had clear-cell histology (n = 98, 92%) with a median number of 9 (2-9) organs/sites affected by metastases. Most frequent organs/sites affected by metastasis were the lungs (67%), distant lymph nodes (34%), and bones (26%). After metachronous metastasis diagnosis, median follow-up interval was 6.2 years (25th-75th percentile: 2.5-8.1). During the pre-specified study period of 5 years after metastasis diagnosis, we observed 63 deaths-from-any-cause. One-, 2-, 3-, and 5-year OS estimates after metachronous metastasis diagnosis were 69% (95%CI: 59-77), 52% (41-62), 41% (30-51), and 28% (19-39), respectively (Supplementary Figure 2). The delivered systemic therapies and their timing in relation to metastasectomy are reported in Supplementary Tables 2 & 3.
      Table 1Baseline characteristics of the study population – Distribution overall and by treatment assignment to metastasectomy (n = 106).
      Variablen (% miss.)Overall (n = 106)No metastasectomy (n = 70)Metastasectomy (n = 36)P
      P-values for difference metastasectomy versus no metastasectomy are from Pearson's χ2 tests (categorical variables with expected cell counts ≥ 5), Fisher's exact tests (categorical variables with expected cell counts < 5), or Wilcoxon rank-sum tests (continuous variables).
      S|IPTW|
      Demographics
      Age (y)106 (0%)70 (62-76)72 (64-77)68 (60-74).0260.410.31
      BMI (kg/m2)
      P-values for difference metastasectomy versus no metastasectomy are from Pearson's χ2 tests (categorical variables with expected cell counts ≥ 5), Fisher's exact tests (categorical variables with expected cell counts < 5), or Wilcoxon rank-sum tests (continuous variables).
      90 (15%)28 (24-31927 (24-30)29 (25-32).2240.320.25
      Female Gender106 (0%)39 (37%)22 (31%)17 (47%).1100.320.04
      Charlson Comorbidity Index (points)
      variable from the time of nephrectomy.
      106 (0%)5 (4-7)5 (5-7)5 (4-6).0140.520.23
      Procedural features: Nephrectomy
      Right side106 (0%)58 (55%)38 (54%)20 (56%).9010.030.06
      Laparoscopic nephrectomy104 (2%)19 (18%)12 (18%)7 (19%).8210.050.35
      Partial nephrectomy106 (0%)25 (24%)17 (24%)8 (22%).8130.050.01
      Surgical access: transperitoneal104 (2%)67 (64%)44 (64%)23 (66%).8450.040.11
      Tumor features: Nephrectomy
      Fuhrmann grade: G3-G4106 (0%)47 (44%)32 (46%)15 (42%).6910.080.27
      Non–clear-cell histology106 (0%)8 (8%)5 (7%)3 (8%).9990.040.03
      Sarcomatoid features105 (1%)6 (6%)6 (9%)0 (0%).1750.430.41
      Tumor necrosis105 (1%)50 (48%)34 (49%)16 (46%).7820.060.15
      Macro- or microscopic vascular invasion106 (0%)51 (48%)35 (50%)16 (44%).5880.110.05
      Tumor size (cm)104 (2%)6.0 (4.5-8.5)6.5 (5.0-8.5)6.0 (4.5-8.0).33660.240.19
      TNM pT stage: pT3-4106 (0%)59 (56%)43 (61%)16 (44%).0960.340.16
      TNM pN stage: N0106 (0%)19 (18%)14 (20%)5 (14%).4370.160.14
      TNM M stage: M1106 (0%)0 (0%)0 (0%)0 (0%).99900
      Tumor features: Time of metastasis diagnosis
      Time from nephrectomy to metastasis diagnosis (y)106 (0%)2.0 (0.7-4.3)1.5 (0.6-3.9)3.3 (1.5-6.5).0110.480.25
      Number of organs/sites affected by metastases
      Variable defined as follows: one count for each metastasis, truncated at a maximum value of 9 (=multiple metastases), eg a patient with 3 lung metastases + 1 bone metastasis + 2 liver metastases has a value of 6.
      106 (0%)9 (2-9)9 (6-9)1 (1-2)< .00012.161.04
      Metastasis location//////
      —Lung106 (0%)71 (67%)48 (69%)23 (64%).6270.100.01
      ——Bilateral lung mets71 (0%)45 (64%)37 (77%)8 (35%).0010.930.72
      ——Number of lung mets71 (0%)5 (2-6)6 (3-6)2 (1-2)< .00011.220.41
      —Liver106 (0%)26 (25%)23 (33%)3 (8%).0050.630.54
      ——Number of liver mets24 (8%)3 (1-6)3 (1-6)1 (1-6).4730.340.62
      —Ad renal gland: ipsilateral106 (0%)10 (9%)7 (10%)3 (8%).9990.060.15
      —Adrenal gland: contralateral106 (0%)5 (5%)4 (6%)1 (3%).6600.140.18
      —Contralateral kidney106 (0%)1 (1%)0 (0%)1 (3%).3400.240.21
      —Distant lymph nodes106 (0%)36 (34%)26 (37%)10 (28%).3350.200.21
      —Bone106 (0%)28 (26%)20 (29%)8 (22%).4830.140.23
      —Central Nervous System106 (0%)17 (16%)9 (13%)8 (22%).2130.250.47
      —Soft tissue106 (0%)18 (17%)13 (19%)5 (14%).5430.130.42
      —Others106 (0%)19 (18%)11 (16%)8 (22%).4080.160.46
      —Mediastinal bulk106 (0%)1 (1%)0 (0%)1 (3%).3400.240.18
      Any local recurrence105 (1%)16 (15%)13 (19%)3 (8%).2520.310.52
      Treatments for metastatic disease
      received at any time during follow-up (non–exclusive, ie patients can appear both in the aVEGF TKI group and the mTOR inhibitor group). Abbreviations: n (%miss.) – number of patients with fully observed data (% missing from a total of 80 patients), BMI – Body Mass Index, TNM – Tumor Node Metastasis classification, aVEGF TKI – tyrosine kinase inhibitor targeting the vascular endothelial growth factor receptor pathway, mTOR – mammalian target of rapamycin, RFA – radiofrequency ablation, LDH – lactate dehydrogenase, |ΔS| – Standardized mean difference (SMD), |ΔIPTW| – IPTW-weighted SMD (weighing with the main IPTW based on a 10-variable propensity score model as reported in Supplementary Table 5), p-values <=0.05 are highlighted in bold font.
      Radiotherapy106 (0%)28 (26%)15 (21%)13 (36%).1110.33N/A
      aVEGF TKI106 (0%)56 (53%)37 (53%)19 (53%).9990.00N/A
      mTOR inhibitor106 (0%)16 (15%)12 (17%)4 (11%).5690.17N/A
      Immune checkpoint inhibitor106 (0%)14 (13%)8 (11%)6 (17%).5470.15N/A
      Local ablation (eg RFA)106 (0%)7 (7%)4 (6%)3 (8%).6870.10N/A
      Laboratory parameters: Time of metastasis diagnosis
      Haemoglobin (g/dL)96 (9%)12.8 (11.5-14.1)12.6 (11.0-14.0)13.2 (12.2-14.3).0620.440.20
      Platelet count (G/L)96 (9%)240 (180-291)246 (179-304)237 (186-267).3790.380.24
      Absolute neutrophil count (G/L)89 (16%)5.4 (4.1-6.5)5.7 (4.4-6.6)4.7 (4.0-6.4).2790.140.19
      LDH (U/L)84 (21%)203 (174-249)207 (174-259)194 (173-238).4280.200.44
      Alkalic phosphatase (U/L)68 (36%)84 (63-108)94 (68-135)71 (59-86).0080.640.63
      C-reactive protein (mg/L)92 (13%)7.2 (2.3-30.0)13.7 (3.4-50.8)3.2 (1.2-13.0).0080.480.51
      Albumin (g/dL)49 (54%)4.2 (3.8-4.4)4.1 (3.7-4.4)4.2 (3.9-4.4).5950.100.17
      Comorbidities: Time of metastasis diagnosis
      Charlson Comorbidity Index (points)106 (0%)11 (9-12)11 (10-12)10 (9-11).0330.330.18
      Continuous variables are summarized as medians (25th percentile [Q1] – 75th percentile [Q3]), whereas categorical variables are reported as absolute frequencies and percentages.
      a P-values for difference metastasectomy versus no metastasectomy are from Pearson's χ2 tests (categorical variables with expected cell counts ≥ 5), Fisher's exact tests (categorical variables with expected cell counts < 5), or Wilcoxon rank-sum tests (continuous variables).
      b variable from the time of nephrectomy.
      c Variable defined as follows: one count for each metastasis, truncated at a maximum value of 9 (=multiple metastases), eg a patient with 3 lung metastases + 1 bone metastasis + 2 liver metastases has a value of 6.
      d received at any time during follow-up (non–exclusive, ie patients can appear both in the aVEGF TKI group and the mTOR inhibitor group). Abbreviations: n (%miss.) – number of patients with fully observed data (% missing from a total of 80 patients), BMI – Body Mass Index, TNM – Tumor Node Metastasis classification, aVEGF TKI – tyrosine kinase inhibitor targeting the vascular endothelial growth factor receptor pathway, mTOR – mammalian target of rapamycin, RFA – radiofrequency ablation, LDH – lactate dehydrogenase, |ΔS| – Standardized mean difference (SMD), |ΔIPTW| – IPTW-weighted SMD (weighing with the main IPTW based on a 10-variable propensity score model as reported in Supplementary Table 5), p-values <=0.05 are highlighted in bold font.

      Metastasectomy Procedures

      Thirty-six (34%) patients had at least 1 metastasectomy procedure. The most frequent first metastasectomy procedures included pulmonary metastasectomy (wedge resections (n = 10) lobectomy (n = 2), bilobectomy (n = 1), craniotomies (n = 6), and bone surgery (n = 4), while partial liver resection was only performed in 1 patient (Table 2). Ten patients and 3 patients underwent a second and third metastasectomy, respectively (Supplementary Table 4). Thirty-day mortality of metastasectomy was 0%. Non–metastasectomy treatment procedures, including systemic, and local-ablative therapies are reported in Table 1.
      Table 2Tabulation of metastasectomy procedures (n = 36).
      Metastasectomy proceduren (%)
      Lung: Wedge resection10 (28%)
      Craniotomy / Brain surgery6 (17%)
      Bone surgery4 (11%)
      Adrenalectomy3 (8%)
      Lymphadenectomy3 (8%)
      Lung: Lobectomy2 (6%)
      Skin / Soft tissue surgery2 (6%)
      Thyroidectomy2 (6%)
      Others2 (6%)
      Lung: Bilobectomy1 (3%)
      Liver resection1 (3%)
      These data represent the first metastasectomy used for assigning patients to the metastasectomy group. Some of these patients received a second or even a third metastasectomy after the index metastasectomy (data reported in Supplementary Table 3)

      Crude Analysis of 5-year Overall Survival According to Metastasectomy

      Median and 5-year OS estimates were 4.2 years (95%CI: 2.7-not reached) and 41% (21-60) in patients who were treated with metastasectomy ± medical therapy (“metastasectomy group”), and 1.3 years (0.8-1.9) and 22% (12- 34) in patients who were treated with medical therapy alone (“medical therapy group”), respectively (log-rank P = .0007, Hazard Ratio (HR) = 0.38, 95%CI: 0.21-0.68, Figure 1A). The median time from diagnosis of metachronous metastasis to metastasectomy was 1.5 months (25th-75th percentile: 1.0-2.3). To reduce and eliminate this immortal time bias, landmark analysis (with a landmark date at 6 months) and time-dependent Cox regression were performed. In landmark analysis, 1-, 3-, and 5-year OS estimates were 96%, 58%, and 38% in the 35 patients who underwent metastasectomy within the first 6 months after metachronous metastasis diagnosis, and 71%, 41%, and 30% in the remaining 71 patients who did not undergo metastasectomy within this time window, respectively (Mantel-Byar P = .014, Figure 2). In univariable Cox regression treating metastasectomy as a fully time-dependent variable, metastasectomy was associated with a 0.5-fold lower relative risk of death as compared to medical therapy alone (HR = 0.48, 95%CI: 0.27-0.87, P = .016).
      Figure 1
      Figure 1Unadjusted and propensity-score-weighted Kaplan-Meier curves of 5-year overall survival according to treatment assignment to metastasectomy (n = 106). Panel A – Unadjusted (“crude”) analysis, Panel B – propensity score analysis (Kaplan-Meier estimators and log-rank test weighted by the inverse-probability-of-treatment-weight (IPTW).
      Figure 2
      Figure 2Landmark analysis of 5-year overall survival according to whether patients underwent metastasectomy within the first 6 months after metastasis diagnosis or not. Blue dashed vertical line – Landmark date at 6 months of follow-up.

      Development of the Propensity Score and the IPTW

      As expected, patients who underwent metastasectomy had a significantly higher prevalence of favorable prognostic factors (Table 1). For example, the median number of organs/sites affected by metastasis were 9 (6-9) and 1 (1-2) in the patients who did not and did undergo metastasectomy, respectively (rank-sum P < .0001; standardized mean difference (SMD) = 2.16, with SMDs > 0.30 indicating a potentially important imbalance between study groups). Further, patients in the metastasectomy group had, among others, lower CRP levels (SMD = 0.48), a lower prevalence of bilateral lung metastases (SMD = 0.93) and a longer interval between nephrectomy and metastases diagnosis (SMD = 0.48) than patients in the non–metastasectomy group. Several of these differentially distributed variables were associated with a more favorable overall survival experience (Supplementary Table 5). To control this bias, we constructed a propensity score (PS) using a multivariable logistic regression model with 10 predictor variables (Supplementary Table 6, with the model building process described in the statistical methods section). The PS (Supplementary Figure 3A) was then transformed into the IPTW according to the inverse of the probability of receiving the treatment that the patient actually received (Supplementary Figure 3B). Re-weighing of the data with the IPTW strongly reduced imbalances of baseline covariates between the 2 treatment groups (Table 1). For example, IPTW-weighing reduced the SMDs for the key prognostic variables (1) time from nephrectomy to metastasis diagnosis from 0.48 to 0.25, (2) number of organ/sites affected by metastases from 2.16 to 1.04, and (3) hemoglobin level at metastases diagnosis from 0.44 to 0.20, respectively.

      IPTW-weighted Analysis of 5-year Overall Survival According to Metastasectomy

      After IPTW weighting of the data, median and 5-year OS estimates were 3.5 years and 24% in the metastasectomy group, and 1.0 years and 20% in the medical therapy group, respectively (IPTW-weighted log-rank P = .001, IPTW-weighted HR = 0.45, 95%CI: 0.27-0.73, Figure 1B). In univariable IPTW-weighted Cox regression treating metastasectomy as a fully time-dependent variable in order control immortal time bias, the association between metastasectomy and favorable OS became much weaker (HR = 0.62, 95%CI: 0.39-1.00, P = .050).

      Exploratory Analysis: Time-to-disease Progression After Metastasectomy

      During a median follow-up of 3.7 years after metastasectomy, 28 (78%) of the 36 patients who underwent metastasectomy developed disease progression. This corresponded to a median time-to-disease-progression of 0.7 years, with 75% and 25% of the metastasectomy cohort remaining free from disease progression for at least 0.3 and 2.7 years, respectively (Supplementary Figure 4).

      Exploratory Analysis: “Complete” and “Incomplete” Metastasectomies

      Twenty-seven (75%) of the 36 metastasectomies achieved complete resection of all known metastases within a single metastasectomy procedure (“complete metastasectomy”). Clinical details about the 9 (25%) “incomplete” metastasectomies are tabulated in Supplementary Table 7. Notably, in an exploratory analysis of unweighted data, the potential benefit of metastasectomy appeared to be confined to complete metastasectomies. In detail, median times-to-disease-progression were 0.3 years and 1.5 years in patients with complete and incomplete metastasectomy (log-rank P = .0003, Figure 3A). Crude 5-year OS estimates were 22%, 17%, and 48% in patients with no, incomplete, and complete metastasectomy (log-rank P = .022, Figure 3B). In an IPTW-weighted Cox model treating both incomplete and complete metastasectomy as a time-dependent variable, we observed HRs of 1.55 (95%CI: 0.84-2.86, P = .163) for incomplete metastasectomy versus no metastasectomy, and 0.54 (95%CI: 0.33-0.89, P = .016) for complete metastasectomy versus no metastasectomy, respectively.
      Figure 3
      Figure 3Clinical outcomes according to completeness of metastasectomy. Panel A (left) – Risk of progression (1-Kaplan-Meier estimator due to absence of competing mortality). Panel B (right) Overall survival. A complete metastasectomy was considered as a metastasectomy that removed all known metastatic lesions within 1 surgical procedure. Abbreviations: MED – Medical therapy only group (ie no metastasectomy), Mincomplete+MED – Incomplete metastasectomy group, Mcomplete+MED – Complete metastasectomy group.

      Sensitivity Analysis: Exploring Potential Time-dependencies of Metastasectomy Benefit

      The beneficial “effect” of metastasectomy appeared to become progressively smaller during follow-up (Figure 1). On further investigation, we found (1) strong evidence for a violation of the proportional hazards assumption for metastasectomy (Schoenfeld test of IPTW-weighted univariable Cox model = 0.012), and (2) a 3.7-fold multiplicative decrease of the relative metastasectomy benefit for each year of follow-up time elapsed (interaction HR between metastasectomy and linear follow-up time in the IPTW-weighted univariable Cox model = 3.68, 95%CI: 1.88-7.17, P < .0001). Indeed, non–proportional analysis of mortality hazards showed that rates of death between metastasectomy versus no metastasectomy crossed at around 2.4 years of follow-up (Supplementary Figure 5). In IPTW-weighted Cox models treating metastasectomy as a time-dependent variable, HRs for metastasectomy and overall survival for prediction horizons of 1, 2, 3, 4, and 5 years were 0.05 (P = .004), 0.13 (P < .0001), 0.27 (P = .003), 0.53 (P = .027), and 0.62 (P = .050), respectively.

      Hypothesis-generating Analysis: Predictive Markers for Metastasectomy Benefit

      To explore further potential factors that might aid in the identification of patients who might or might not benefit from metastasectomy, we fitted interactions between metastasectomy and 3 key variables: metachronous interval as an indicator of disease biology (≤ vs. > 1 year), Charleson comorbidity index as an indicator of perioperative risk (≤ vs. > 10 points), and number of metastatic lesions as an indicator of tumor burden and potential resectability (1 vs. 2-6 vs. multiple metastases/affected organs). Here, we found clear signals toward a greater magnitude of benefit of metastasectomy regarding overall survival in patients with only 1 metastatic lesion, while metastasectomy appeared to be without benefit over best medical therapy in patients with multiple metastases (Figure 4). Patients with comorbidity scores ≤ versus > 10 points or patients with metachronous intervals ≤ versus > 1-year experienced similar benefit from metastasectomy.
      Figure 4
      Figure 4Subgroup analysis of potential metastasectomy benefit by 3 key variables (disease-free interval, comorbidity, and metastatic burden). Data were obtained by fitting interactions between metastasectomy and the respective variables within IPTW-weighted, immortal-time-bias-adjusted Cox models. Abbreviations: HR – Hazard ratio, OS – Overall Survival, 95%CI - 95% confidence interval.

      Discussion

      Randomized evidence assessing the potential benefit of metastasectomy in mRCC is currently not available. In this study, we thus performed a propensity-score-based comparative effectiveness analysis of OS outcomes in a large institutional mRCC population among which approximately one third of the patients underwent metastasectomy in addition to standard medical therapy. Crude analysis suggested that patients who underwent metastasectomy had significantly longer OS than patients who were treated with standard medical therapy alone. However, consistent with the non–random assignment to metastasectomy in this cohort, this finding was confounded by selection bias. Additionally, immortal time bias was present. After controlling for these 2 biases with propensity score weighting and time-dependent analysis, the association between metastasectomy and favorable OS prevailed, although at a much smaller magnitude of benefit, and strength of association. In exploratory analyses, we found a weakening “effect” of metastasectomy over time, and no evidence that “incomplete” metastasectomies associate with improved OS. In exploratory analyses, a higher number of metastatic lesions was associated with less metastasectomy benefit, while patients appeared to benefit from metastasectomy irrespective of comorbidity level or the disease-free interval between nephrectomy and metastasis diagnosis. In summary, these data support the hypothesis that metastasectomy improves OS in mRCC, but that the magnitude of this potential OS benefit weakens over time and appears to be confined to patients who achieve excision of all known metastatic lesions within a single metastasectomy procedure. Number of metastatic lesions can further inform indication toward metastasectomy.
      Several additional insights could be gained from exploratory and hypothesis-generating analyses. First, although 25% of patients remained free from disease progression for at least 2.7 years, the median time-to-progression after metastasectomy of 0.7 years was quite short. An interpretation of this finding is that the clinical utility of metastasectomy should be further improved by carefully selecting patients for metastasectomy based on prognostic factors. Several authors have already provided hints in this direction.
      • Zhao Y
      • Li J
      • Li C
      • Fan J
      • Liu L.
      Prognostic factors for overall survival after lung metastasectomy in renal cell cancer patients: a systematic review and meta-analysis.
      ,
      • Hofmann H-S
      • Neef H
      • Krohe K
      • Andreev P
      • Silber R-E.
      Prognostic factors and survival after pulmonary resection of metastatic renal cell carcinoma.
      ,
      • Murthy SC
      • Kim K
      • Rice TW
      • et al.
      Can we predict long-term survival after pulmonary metastasectomy for renal cell carcinoma?.
      Another interpretation of this finding is that although some patients derive clinically significant long-term freedom-from-disease from metastasectomy, true cure is not as frequent as one may expect, and many patients eventually relapse after metastasectomy. Second, we found a highly time-dependent effect of metastasectomy that weakened over time, suggesting that metastasectomy can be considered an intervention that delays death but does not necessarily lead to a higher proportion of long-term survivors. Uro-oncologists should take this into account when discussing metastasectomy with their patients. Third, we found that so-called “incomplete” metastasectomies which did not remove all known metastatic lesions within the first metastasectomy surgery were not associated with any OS benefit. This finding confirms several previous reports who have shown that the most important prognostic factor for OS after metastasectomy is complete resection.
      • Hofmann H-S
      • Neef H
      • Krohe K
      • Andreev P
      • Silber R-E.
      Prognostic factors and survival after pulmonary resection of metastatic renal cell carcinoma.
      ,
      • Kawashima A
      • Nakayama M
      • Oka D
      • et al.
      Pulmonary metastasectomy in patients with renal cell carcinoma: a single-institution experience.
      A clinical interpretation of this finding is that incomplete metastasectomies can still be performed with a reasonable symptomatic treatment goal (see eg the patient vignettes in Supplementary Table 6, such as the patient with pain from femoral bone metastasis who underwent femoral resection for pain relief in the presence of several other bone metastases), but one should not expect that such metastasectomies prolong a patient's survival. Fourth, in subgroup analyses we found signals toward a greater benefit of metastasectomy in patients with no more than 6 metastatic lesions / sites affected by metastasis. Here, the number of metastases may be interpreted as a proxy for tumor burden and technical resectability, and comorbidity scores as a proxy for perioperative risk. Considering the disease-free interval between nephrectomy and metastasis diagnosis as a proxy for disease biology, subgroup analyses suggested that patients appeared to benefit from metastasectomy irrespective of the disease-free interval. Higher comorbidity emerged as a strong predictor of worse OS, but the potential relative OS benefit of metastasectomy appeared to be consistent across comorbidity subgroups. Although we urge readers to consider these subgroup results purely hypothesis-generating due to the small patient numbers, arbitrary cut-offs, and high potential for false-positive and false-negative results, they could provide the basis for improved metastasectomy indication once being confirmed by other studies.
      Otherwise, our study cohort compares well to other study cohorts in the field with regard to treatment outcomes. In detail, our unadjusted 5-year OS of 41% in the metastasectomy group is similar to the corresponding estimates of and 33%, 36%, and 45% in the pulmonary metastasectomy cohorts of Hofmann et al.,
      • Hofmann H-S
      • Neef H
      • Krohe K
      • Andreev P
      • Silber R-E.
      Prognostic factors and survival after pulmonary resection of metastatic renal cell carcinoma.
      Kawashima et al,
      • Kawashima A
      • Nakayama M
      • Oka D
      • et al.
      Pulmonary metastasectomy in patients with renal cell carcinoma: a single-institution experience.
      Procházková et al.,
      • Procházková K
      • Vodička J
      • Fichtl J
      • et al.
      Outcomes for patients after resection of pulmonary metastases from clear cell renal cell carcinoma: 18 years of experience.
      and 43% in the pulmonary metastasectomy meta-analysis by Zhao et al.
      • Zhao Y
      • Li J
      • Li C
      • Fan J
      • Liu L.
      Prognostic factors for overall survival after lung metastasectomy in renal cell cancer patients: a systematic review and meta-analysis.
      Importantly, 30-day mortality of our metastasectomy cohort was 0%, showing that metastasectomy in mRCC is safe when patients are well selected, and treated at an experienced center by experienced surgeons. While most studies in this field focused on pulmonary metastasectomies, it is noteworthy that pulmonary metastasectomies constituted only one third of metastasectomies in our cohort which also included a significant number of craniotomies and several less frequent procedures such as adrenalectomy, skin/soft tissue surgeries, and thyroidectomies. This is consistent with the study by Adashek and colleagues,
      • Adashek JJ
      • Aydin AM
      • Kim P
      • Spiess PE.
      The role of metastasectomy in the treatment of metastatic renal cell carcinoma.
      who report on mRCC metastasectomies of the brain, the liver, the pancreas, the bone, and the lymph nodes.
      Finally, several limitations should be discussed. First, as with all retrospective analyses, we cannot exclude information bias by miscoding of exposures and outcomes. Second, our propensity score model may not have reduced all imbalances between the 2 treatment groups, because the assumption that a propensity score model is difficult-to-test. Although balance diagnostics after IPTW weighting showed removal of most differences, some variables still had SMDs >0.3. This means that the “true” causal effect of metastasectomy on OS in mRCC could be even slightly lower than our estimate. Next, we did not consider interventional procedures without removal of tumor masses such as radiofrequency ablations as metastasectomies. Whether these increasingly popular methods are associated with improved OS should thus be addressed in future studies. Fourth, some metastasectomy procedures such as adrenalectomy of skin/soft tissue surgery had small numbers. Thus, we cannot provide specific effect estimates for each individual metastasectomy procedure, but rather for metastasectomy as a whole. Fifth, a limited number of patients underwent further metastasectomies during follow-up after their first metastasectomy procedure, and with the advent of immunotherapy, the effect of systemic therapies can be expected of having improved over time. Modeling of these 2 factors was considered not possible within our analysis framework for difficult-to-ascertain time-dependent confounding. A further limitation is that quality-of-life data were not available in our retrospective study. As metastasectomies can be highly invasive surgical procedures requiring post-surgical intensive care unit surveillance and in-hospital stays, future studies should not only focus on overall survival but also on quality-of-life aspects to gain a more patient-oriented picture of the overall utility of metastasectomy in this setting. Moreover, we did not have data on performance status available, and could thus not stratify our analysis on the clinically-relevant prognostic groups according to the MSKCC and IMDC risk scores.
      • Ko JJ
      • Xie W
      • Kroeger N
      • et al.
      The international metastatic renal cell carcinoma database consortium model as a prognostic tool in patients with metastatic renal cell carcinoma previously treated with first-line targeted therapy: a population-based study.
      ,
      • Motzer RJ
      • Mazumdar M
      • Bacik J
      • Berg W
      • Amsterdam A
      • Ferrara J.
      Survival and prognostic stratification of 670 patients with advanced renal cell carcinoma.
      Next, we pre-specified to consider only patients with metachronous metastasis after nephrectomy for our study cohort. The reasons for this were that inclusion of patients with synchronous metastases would have led to more immortal time bias and more time-dependent selection (eg patients with synchronous metastases who received metastasectomy only after having achieved stable disease on a systemic therapy, or patients treated with cytoreductive nephrectomy as recently examined in the CARMENA trial).
      • Méjean A
      • Ravaud A
      • Thezenas S
      • et al.
      Sunitinib alone or after nephrectomy in metastatic renal-cell carcinoma.
      Although this stringent selection of metachronous metastasis patients can be considered as a strength, we of course cannot generalize our results to patients with synchronous metastasis, and future studies should thus also address this sub-population. Moreover, we included an “all-comer” population of patients with metachronous metastasis. A restriction of the cohort to patients who are “optimal” candidates for metastasectomy (ie 1 or few metastatic lesions, low comorbidity scores) may have resulted in a more delineated population at the cost of decreased generalizability of results. Finally, our study did not address the potential impact of medical therapy after metastasectomy, as this would have been a propensity score analysis on its own. Here, emerging randomized evidence points toward improved post-metastasectomy outcomes with checkpoint inhibitor monotherapy,
      • Choueiri TK
      • Tomczak P
      • Park SH
      • et al.
      Pembrolizumab versus placebo as post-nephrectomy adjuvant therapy for patients with renal cell carcinoma: randomized, double-blind, phase III KEYNOTE-564 study.
      but not with VEGF-directed TKI therapy.
      • Procopio G
      • Apollonio G
      • Cognetti F
      • et al.
      Sorafenib versus observation following radical metastasectomy for clear-cell renal cell carcinoma: results from the phase 2 randomized open-label RESORT study.
      ,
      • Appleman LJ
      • Puligandla M
      • Pal SK
      • et al.
      Randomized, double-blind phase III study of pazopanib versus placebo in patients with metastatic renal cell carcinoma who have no evidence of disease following metastasectomy: a trial of the ECOG-ACRIN cancer research group (E2810).

      Conclusion

      In conclusion, this comparative effectiveness analysis supports the hypothesis that metastasectomy is associated with an overall survival benefit in patients with mRCC. This benefit slightly weakens over time and appears to be restricted to patients in which complete resection of all known metastatic lesions is achieved within a single metastasectomy procedure. Metastasectomy benefit appears to be greatest in patients with only 1 metastatic lesion. These results can inform cancer specialists and patients when planning comprehensive multi-disciplinary treatment for mRCC.

      Clinical Practice Points

      Metastasectomy is a frequently performed procedure in patients with metastatic renal cell carcinoma (mRCC). Clinical experience and several retrospectives non–comparative studies show favourable survival outcomes in mRCC patients after metastasectomy. However, the magnitude of benefit of metastasectomy as compared to best medical therapy alone has not been established. Given that randomized controlled trials are difficult to perform in this situation, we performed a so-called propensity score analysis of 106 mRCC patients, of whom 36 (34%) were treated with metastasectomy and 70 (66%) with medical therapy alone. After a median follow-up of over 6 years, we found that patients who had undergone metastasectomy had better overall survival (OS) outcomes, but also a significantly higher prevalence of favorable prognostic factors, than patients who had not undergone metastasectomy. After adjusting for this selection bias with propensity score weighting and additionally controlling for immortal time bias, the favorable association between metastasectomy, and OS prevailed but became much weaker. We then found signals that any overall survival benefit of metastasectomy was confined to patients who achieved a macroscopically complete resection of all known metastases. Within the limitations of an observational study, these findings can inform patients with mRCC about the potential magnitude of benefit of metastasectomy, and support their treatment team toward a more refined indication of metastasectomy in this setting.

      CRediT authorship contribution statement

      • Conceived and designed the study: FP.
      • Database setup: FM SM FP.
      • Collected data: FM MAS DAB MS GH MP.
      • Analyzed the data: FM FP.
      • Interpreted the results: All authors.
      • Wrote the first draft of the article: FM FP.
      • Revised the first draft and contributed to the writing of the article: All authors.
      • Agree with the article's results and conclusions: All authors.
      • ICMJE criteria for authorship read and met: All authors.

      Data Accessibility Statement

      The current data cannot be made available online under the current institutional review board approval. However, the data may be shared with investigators on reasonable request to the corresponding author within a research cooperation project.

      Disclosure

      The authors have no conflicting interests to declare.

      Acknowledgements

      We thank Mrs. Andrea Schlemmer and Mrs. Astrid Mandl-Pohl (both Institute of Medical Informatics, Statistics, and Documentation; Medical University of Graz) for helpful technical assistance with data extraction from our hospital's electronic healthcare database.

      Appendix. Supplementary materials

      References

        • Rini BI
        • Plimack ER
        • Stus V
        • et al.
        Pembrolizumab plus Axitinib versus sunitinib for advanced renal-cell carcinoma.
        N Engl J Med. 2019; 380: 1116-1127
        • Choueiri TK
        • Escudier B
        • Powles T
        • et al.
        Cabozantinib versus Everolimus in advanced renal cell carcinoma (METEOR): final results from a randomised, open-label, phase 3 trial.
        Lancet Oncol. 2016; 17: 917-927
        • Motzer RJ
        • Hutson TE
        • Cella D
        • et al.
        Pazopanib versus sunitinib in metastatic renal-cell carcinoma.
        N Engl J Med. 2013; 369: 722-731
        • Motzer RJ
        • Escudier B
        • McDermott DF
        • et al.
        Nivolumab versus Everolimus in advanced renal-cell carcinoma.
        N Engl J Med. 2015; 373: 1803-1813
        • Ko JJ
        • Xie W
        • Kroeger N
        • et al.
        The international metastatic renal cell carcinoma database consortium model as a prognostic tool in patients with metastatic renal cell carcinoma previously treated with first-line targeted therapy: a population-based study.
        Lancet Oncol. 2015; 16: 293-300
        • van der Poel HG
        • Roukema JA
        • Horenblas S
        • van Geel AN
        • Debruyne FM.
        Metastasectomy in renal cell carcinoma: a multicenter retrospective analysis.
        Euro Urol. 1999; 35: 197-203
        • Zhao Y
        • Li J
        • Li C
        • Fan J
        • Liu L.
        Prognostic factors for overall survival after lung metastasectomy in renal cell cancer patients: a systematic review and meta-analysis.
        Int J Surg. 2017; 41: 70-77
        • Ouzaid I
        • Capitanio U
        • Staehler M
        • et al.
        Surgical metastasectomy in renal cell carcinoma: a systematic review.
        Eur Urol Oncol. 2019; 2: 141-149
        • Moik F
        • Riedl JM
        • Winder T
        • et al.
        Benefit of second-line systemic chemotherapy for advanced biliary tract cancer: a propensity score analysis.
        Sci Rep. 2019; 9: 5548
        • Austin PC.
        The use of propensity score methods with survival or time-to-event outcomes: reporting measures of effect similar to those used in randomized experiments.
        Stat Med. 2014; 33: 1242-1258
        • Posch F
        • Leitner L
        • Bergovec M
        • et al.
        Can multistate modeling of local recurrence, distant metastasis, and death improve the prediction of outcome in patients with soft tissue sarcomas?.
        Clin Orthop Relat Res. 2017; 475: 1427-1435
        • Posch F
        • Silina K
        • Leibl S
        • et al.
        Maturation of tertiary lymphoid structures and recurrence of stage II and III colorectal cancer.
        Oncoimmunology. 2018; 7e1378844
        • Seles M
        • Posch F
        • Pichler GP
        • et al.
        Blood platelet volume represents a novel prognostic factor in patients with nonmetastatic renal cell carcinoma and improves the predictive ability of established prognostic scores.
        J Urol. 2017; 198: 1247-1252
        • Austin PC
        • Stuart EA.
        Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies.
        Stat Med. 2015; 34: 3661-3679
        • Schemper M
        • Smith TL.
        A note on quantifying follow-up in studies of failure time.
        Control Clin Trials. 1996; 17: 343-346
        • White IR
        • Royston P
        • Wood AM.
        Multiple imputation using chained equations: issues and guidance for practice.
        Stat Med. 2011; 30: 377-399
        • Hofmann H-S
        • Neef H
        • Krohe K
        • Andreev P
        • Silber R-E.
        Prognostic factors and survival after pulmonary resection of metastatic renal cell carcinoma.
        Euro Urol. 2005; 48: 77-82
        • Murthy SC
        • Kim K
        • Rice TW
        • et al.
        Can we predict long-term survival after pulmonary metastasectomy for renal cell carcinoma?.
        Ann Thorac Surg. 2005; 79: 996-1003
        • Kawashima A
        • Nakayama M
        • Oka D
        • et al.
        Pulmonary metastasectomy in patients with renal cell carcinoma: a single-institution experience.
        Int J Clin Oncol. 2011; 16: 660-665
        • Procházková K
        • Vodička J
        • Fichtl J
        • et al.
        Outcomes for patients after resection of pulmonary metastases from clear cell renal cell carcinoma: 18 years of experience.
        Urologia Internationalis. 2019; 103: 297-302
        • Adashek JJ
        • Aydin AM
        • Kim P
        • Spiess PE.
        The role of metastasectomy in the treatment of metastatic renal cell carcinoma.
        AME Med J. 2019; 4
        • Motzer RJ
        • Mazumdar M
        • Bacik J
        • Berg W
        • Amsterdam A
        • Ferrara J.
        Survival and prognostic stratification of 670 patients with advanced renal cell carcinoma.
        J Clin Oncol. 1999; 17: 2530-2540
        • Méjean A
        • Ravaud A
        • Thezenas S
        • et al.
        Sunitinib alone or after nephrectomy in metastatic renal-cell carcinoma.
        N Engl J Med. 2018; 379: 417-427
        • Choueiri TK
        • Tomczak P
        • Park SH
        • et al.
        Pembrolizumab versus placebo as post-nephrectomy adjuvant therapy for patients with renal cell carcinoma: randomized, double-blind, phase III KEYNOTE-564 study.
        J Clin Oncol. 2021; 39
        • Procopio G
        • Apollonio G
        • Cognetti F
        • et al.
        Sorafenib versus observation following radical metastasectomy for clear-cell renal cell carcinoma: results from the phase 2 randomized open-label RESORT study.
        Eur Urol Oncol. 2019; 2: 699-707
        • Appleman LJ
        • Puligandla M
        • Pal SK
        • et al.
        Randomized, double-blind phase III study of pazopanib versus placebo in patients with metastatic renal cell carcinoma who have no evidence of disease following metastasectomy: a trial of the ECOG-ACRIN cancer research group (E2810).
        J Clin Oncol. 2019; 37: 4502