You are here

  • Home
  • Archive
  • Volume 13, Issue 5
  • Associations between immune checkpoint inhibitor response, immune-related adverse events, and steroid use in RADIOHEAD: a prospective pan-tumor cohort study

Article Text

Article menu

Download PDFPDF

Clinical cancer immunotherapy
Original research
Associations between immune checkpoint inhibitor response, immune-related adverse events, and steroid use in RADIOHEAD: a prospective pan-tumor cohort study
  1. Zoe Quandt1,2,
  2. Anastasia Lucas3,
  3. Samantha I Liang3,
  4. EnJun Yang3,
  5. Samantha Stone4,5,
  6. Muhammad Zaki Hidayatullah Fadlullah5,6,
  7. Nicholas L Bayless3,
  8. Sara Siebel Marr3,
  9. Marshall A Thompson3,
  10. Lacey J Padron3,
  11. Samantha Bucktrout3,
  12. Lisa H Butterfield3,7,
  13. Aik Choon Tan5,6,
  14. Kevan C Herold8,9,
  15. Jeffrey A Bluestone3,
  16. Mark S Anderson1,2,
  17. Christine N Spencer3,
  18. Arabella Young4,5 and
  19. John E Connolly3
  1. 1 Department of Medicine, Division of Endocrinology and Metabolism, UCSF, San Francisco, California, USA
  2. 2 Diabetes Center, UCSF, San Francisco, California, USA
  3. 3 Parker Institute for Cancer Immunotherapy, San Francisco, California, USA
  4. 4 Division of Microbiology and Immunology, Department of Pathology, University of Utah, Salt Lake City, Utah, USA
  5. 5 Huntsman Cancer Institute, University of Utah, Salt Lake City, Utah, USA
  6. 6 Departments of Oncological Sciences and Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA
  7. 7 Microbiology and Immunology, University of California, San Francisco, California, USA
  8. 8 Department of Immunobiology, Yale University, New Haven, Connecticut, USA
  9. 9 Department of Internal Medicine, Division of Endocrinology and Metabolism, Yale University, New Haven, Connecticut, USA
  1. Correspondence to Dr John E Connolly; jconnolly{at}parkerici.org

Abstract

Background Immune checkpoint inhibitors (ICIs) have led to enduring responses in subsets of patients with cancer. However, these responses carry the risk of immune-related adverse events (irAEs), which can diminish the overall benefit of ICI treatment. While associations between irAE development and overall survival have been increasingly documented, there is a need for further understanding of these connections in large prospective real-world cohorts.

Methods The Resistance Drivers for Immuno-Oncology Patients Interrogated by Harmonized Molecular Datasets (RADIOHEAD) study, a pan-tumor, prospective cohort of 1,070 individuals undergoing standard of care first-line ICI treatment, aims to identify factors driving irAEs and clinical response. Clinical data and longitudinal blood samples were collected prospectively at multiple time points from 49 community-based oncology clinics across the USA. Structured, harmonized clinical data underwent unbiased statistical analysis to uncover predictors of real-world overall survival (rwOS) and risk factors for irAEs.

Results Across 1,070 participants’ treatment courses, RADIOHEAD accumulated over 4,500 clinical data points. Patients experiencing any irAE (25.4%, n=272) exhibited significantly improved rwOS in the pan-tumor cohort (n=1,028, HR=0.41, 95% CI=(0.31, 0.55)). This association persisted when adjusting for age and metastatic disease in multivariate time-dependent Cox proportional hazard analysis, and was consistent across major tumor subtypes, including lung cancer and melanoma. Skin and endocrine irAEs of any grade were strongly associated with improved rwOS (Cox proportional hazard analysis, skin, p=2.03e−05; endocrine, p=0.0006). In this real-world cohort, the irAE rate appeared lower than those reported in clinical trials. Patients receiving corticosteroids prior to initiation of ICI treatment had significantly worse survival outcomes than non-users (HR 1.37, p=0.0054), with a stronger association with systemic steroid use (HR 1.75, p=0.0022). The risk of irAE was increased by exposure to combination immunotherapy relative to monotherapy (OR 4.17, p=2.8e−7), zoster vaccine (OR 2.4, p=5.2e−05), and decreased by prior chemotherapy (OR 1.69, p=0.0005).

Conclusion The RADIOHEAD cohort is a well-powered, real-world cohort that clearly demonstrates the association between irAE development with improved response and baseline steroid use with worse response to ICI treatment after adjustment for survival bias.

  • Immune Checkpoint Inhibitors
  • Immune related adverse event - irAE
  • Immunotherapy

Data availability statement

Data are available upon reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/jitc-2025-011545

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Previous studies of immune-related adverse events (irAEs) and survival following immune checkpoint inhibitors (ICIs), have largely been based in clinical trials or of a retrospective nature. These studies suggest a link between irAEs and improved survival with ICIs, but real-world, large-scale data on this association is limited.

    WHAT THIS STUDY ADDS

    • This study, using data from the Resistance Drivers for Immuno-Oncology Patients Interrogated by Harmonized Molecular Datasets (RADIOHEAD) cohort, confirms the link between irAEs and better survival outcomes, shows worse responses to ICIs in patients on baseline steroids, and reports a novel association between the zoster vaccine and irAEs.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • The study establishes the RADIOHEAD cohort as a key resource for understanding irAEs and response to ICI therapy, guiding future research and precision medicine strategies, and potentially influencing clinical practices for managing irAEs.

    Introduction

    Immune checkpoint inhibitors (ICIs) have revolutionized cancer treatment by inducing durable responses in select cancer patient populations. However, this therapeutic advancement is accompanied by the risk of immune-related adverse events (irAEs), which have the potential to cause significant morbidity and mortality, offsetting the overall benefit of ICI therapy.1–3

    While associations between the frequency of irAEs and immunotherapy treatment response have been observed in clinical trial settings and retrospective studies, conflicting evidence exists with initial associations varying by irAE type.4–6 Many studies later found that this link weakened when accounting for survival bias, as patients needed to survive and continue treatment long enough to develop an irAE.7 Our understanding of ICI efficacy, irAE incidence and their relationship with therapeutic response in real-world, standard of care settings remains limited. Additionally, ICI treatment in standard of care cohorts, characterized by broad inclusion criteria and minimal exclusion criteria, has been increasingly met with a disconnect between clinical use and benefit.8 9 This highlights the need for prospective studies in the real-world setting to bridge the gap between research findings and clinical practice.

    To address these knowledge gaps and to elucidate the clinical and molecular drivers of ICI-induced irAEs and treatment response, a real-world cohort of patients with cancer receiving first-line ICI therapy as standard of care was established, paired with prospective, longitudinal sample collection. Named Resistance Drivers for Immuno-Oncology Patients Interrogated by Harmonized Molecular Datasets (RADIOHEAD), this pan-tumor cohort comprises 1,070 individuals undergoing standard of care first-line ICI treatment.

    In this report, we present the demographics and clinical characteristics of the RADIOHEAD cohort. Through comprehensive statistical analyses, we identify associations between clinical risk factors and real-world overall survival (rwOS) and irAEs. Results from this study will provide insights into the clinical outcomes of ICI therapy in real-world settings, ultimately informing future treatment strategies and improving cancer patient care.

    Methods

    Patient identification

    Following informed consent, subjects were enrolled from 49 community oncology clinics (online supplemental table 1) across the USA from May 2018 to May 2022. Patients with solid cancers and who were naïve to ICIs were eligible to participate. A cohort of over 1,000 subjects receiving standard of care therapy was deemed adequate to capture rare irAEs.

    Supplemental material

    Data and sample collection

    To capture a comprehensive profile of each patient, we prospectively collected blood samples at pretreatment, prior to cycle 3 (3–6 weeks after treatment initiation), and 6-month and 12-month time points together with clinical features recorded via case report forms and electronic medical records. If patients experienced irAEs, additional sample and clinical data collections were made if possible at the time of irAE, and then at 4–6 weeks, 6 months, and 12 months after irAE onset. Clinical data collected included basic demographic information, past medical history, medication use, vaccinations and prior cancer therapy alongside longitudinal data specific to their cancer treatment, such as cancer stage, presence of metastases, and treatment changes. Tissue-specific irAEs, such as rash, hypothyroidism, and colitis, were noted with details including treatment and grade at multiple study time points. Data were collected by research coordinators at each site and entered into a study-wide REDCap (Research Electronic Data Capture)10 database.

    Clinical data quality control and harmonization

    Clinical data and annotations were systematically reviewed to identify probable erroneous data points, for example, infeasible dates and inconsistent responses in intentionally redundant fields. Flagged data points were subsequently reviewed and reconciled with sites. Data was de-identified and transacted into an in-house integrated datomic database, part of a suite of tools called the Cancer Data and Evidence Library (CANDEL) which enable storage, harmonization, and query of clinical and molecular data.11 Data was mapped to standard ontologies or CANDEL-specific entities, and study time points were transformed to be relative to study or therapy start. Pre-existing conditions and irAEs, which were recorded as a combination of free and structured responses, were mapped to Medical Dictionary for Regulatory Activities (MedDRA) preferred terms using a combination of manual lookups and Biomedical Ontology’s BioPortal REST API. We employed a multistage quality control procedure for patient inclusion, first excluding individuals who did not have sufficient evidence of being on an immunotherapy treatment regimen or who were not immunotherapy-naive prior to study, and second those who were on study for less than 30 days.

    For statistical analyses of survival, data was truncated at 15 months to reduce potential non-systematic bias from individuals involved in an extension study where select subjects, particularly those with irAEs, were followed for longer periods of time. Calculation of rwOS was performed by using reported ICI treatment start date and either the date of death or the last date of contact with the patient. For survival analyses, patients who exited the study on hospice were grouped with patients who had died on study. Patients included in survival analyses were required to be on a regimen that included programmed death (ligand) 1 (PD-(L)1) and have a reported rwOS of at least 30 days. It was possible for patients to have had previous anticancer treatments that were not ICIs prior to study enrollment.

    For statistical analyses of irAEs, MedDRA12 preferred terms were grouped into MedDRA System Organ Classes which were reviewed by a clinician and recategorized where necessary so that each term contributed to a unique class (online supplemental table 2). Corticosteroids were designated as systemic, non-systemic, or unclear according to their route of administration. Oral medications were assigned to the systemic category, while inhaled and topical corticosteroids comprised the non-systemic category. Medications that can span multiple routes of administration were categorized as unclear if clarification was not included in the patient’s original medication record. Clarifying terms include qualifiers such as “Spray”, “Inhaler”, “Powder”, and “Ointment”.

    Statistical analysis

    Demographics of the cohort including age, sex, treatment type, cancer type and stage, were described using count and percent. Prior to statistical analysis, we confirmed that study center was not significantly associated with irAE status or survival through χ² tests. As appropriate, comparisons between irAE groups used χ² tests unless cell sizes were ever less than 5 (in which case Fisher’s exact tests were used) for testing associations between categorical variables. Student’s t-test was used for testing associations between categorical and continuous variables. Alpha levels were set to 0.05.

    We also ran exploratory multi-phenotype association studies to identify relationships between clinical variables, survival, and irAE features. To classify irAE features, we considered multiple levels of binning including (1) any irAE, (2) irAE grade (all vs severe (grade 3 or 4)), (3) irAEs in a broad organ class, such as endocrine and skin, and (4) specific irAE, such as hypothyroidism and rash. The R survival package (V.3.6–4, Therneau 2024) was used to perform univariate Cox regression using rwOS as the outcome and clinical variables, including irAE status, as predictors. To account for survival bias (that perhaps the association between irAEs and rwOS is caused by patients living long enough to develop an irAE), these same associations were also tested using a time-dependent Cox proportional hazard analysis. Time-dependent Cox analysis allows variables to change over time and is able to address survival bias. Associations between irAE outcomes and clinical variables were assessed using logistic regression models. A minimum of five or more patients was required per category for each clinical variable. Analyses were performed in the pan-tumor cohort as well as the two largest tumor-specific subcohorts: lung cancer and melanoma. Ad hoc analyses were performed to ensure trends were maintained when adjusting for age.

    Results

    Patient demographics

    The final RADIOHEAD cohort consists of 1,070 ICI-naive patients with cancer treated in a standard of care community hospital setting (table 1). Initially, 1,254 subjects were consented, 1,212 had at least one sample collected, and 1,070 were included in the final analysis following exclusions for registration errors (such as enrollment when they had been on prior ICIs), less than 30 days of clinical follow-up (such as death prior to ICI initiation), missing critical treatment or cancer data. The median age was 69 (IQR 62–77) with full range from 25 to 89 years old and 56.3% of patients were male. Consistent with standard of care treatment regimens, PD-(L)1 inhibitors (nivolumab, pembrolizumab, cemiplimab, atezolizumab, durvalumab, and avelumab) were the most used form of ICI, comprising 91.6% of the cohort (with 67.0% receiving ICI alone, 22.4% receiving ICI and chemotherapy, and 2.1% administered ICI alongside other anticancer agents). The remaining 8.4% were patients administered PD-(L)1 and CTLA-4 inhibitor (ipilimumab) in combination. Within this pan-cancer cohort, non-small cell lung cancer (NSCLC) was the most common form of cancer (37.2%), followed by melanoma (12.0%), renal cell cancer (8.8%) and urothelial cancer (8.3%). Most patients were being treated for either stage III (24.0%) or stage IV (68.7%) cancer. Patients were predominantly white (90.7%) and non-Hispanic (90.6%) and were collected broadly across the USA but with more representation from the Northeast (47.3%) and Midwest (36.0%) than the South (14.2%) or West (2.5%).13

    View this table:
    Table 1

    Characteristics of patients included in the RADIOHEAD cohort

    ICI efficacy

    Consistent with previous reports in clinical trial data,14 ICI-treated patients with melanoma displayed significantly improved survival compared to patients with NSCLC or individuals with other cancer types (online supplemental figure 1). At 15 months of follow-up, 61% of patients were alive, 33% were dead, 2% were enrolled in hospice, and 3.9% were lost to follow-up. Therefore, survival analyses were limited to the 1,028 subjects that were known to be alive, dead or entered in hospice.

    ICI toxicity

    We next examined the real-world occurrence of irAEs (table 2). Skin and subcutaneous tissue disorders were the predominant irAE (26.5%, n=111), followed by endocrine disorders (22.9%, n=96), gastrointestinal disorders (16.0%, n=67), and respiratory, thoracic and mediastinal disorders (11.0%, n=46) (figure 1A). The most common organ-specific irAEs were rash (8.9%, n=95), thyroid dysfunction (6.2%, n=66) and diarrhea/colitis (4.8%, n=51). Diarrhea/colitis, hepatitis, and pneumonitis had the highest incidence of severe (grade III–IV) irAEs (figure 1B). Most patients experiencing irAEs reported only one (17.9% of overall cohort, n=191), fewer had two irAEs (5.4%, n=58) and 2% (n=23) of patients had three or more irAEs. Most of the patients who developed irAEs did so in the first 3 months (n=176 of 272 total patients with irAEs) (online supplemental figure 2).

    View this table:
    Table 2

    Observed irAEs in the RADIOHEAD cohort study

    Figure 1
    Figure 1

    Immune-related adverse events and impact on survival outcome. (A) Distribution of observed irAEs; (B) Most frequently observed irAEs; (C) Kaplan-Meier survival analysis of all patients in the RADIOHEAD cohort with confirmed alive or dead (including hospice) status for any irAE incidence. Statistical significance for this association is shown for both a Cox proportional hazards model adjusted for age (p) and time-dependent Cox proportional hazard model adjusting for age and metastatic disease (p(time dependent)); (D) Kaplan-Meier survival analysis of patients with lung cancer (NSCLC and SCLC) with confirmed alive or dead (including hospice) status for any irAE incidence. Again, significance from both analytic approaches is shown; (E) Kaplan-Meier survival analysis of patients with melanoma with confirmed alive or dead (including hospice) status for any irAE incidence. Again, significance from both analytic approaches is shown. AI, adrenal insufficiency; IrAE; immune-related adverse event; NSCLC, non-small cell lung cancer; RADIOHEAD, Resistance Drivers for Immuno-Oncology Patients Interrogated by Harmonized Molecular Datasets; SCLC, small cell lung cancer; SIRS, systemic inflammatory response syndrome.

    Relationship between irAEs and rwOS

    Given previous reports that suggest an association between improved survival outcomes and irAE incidence, we sought to identify whether rwOS varied by irAE development. Improved rwOS was observed in patients who developed any irAE in the pan-cancer cohort (n=1,028, HR=0.41, 95% CI=(0.31, 0.55)) (figure 1C). The association between irAE and survival was maintained when adjusting for age, sex, and metastatic disease even in multivariate time-dependent Cox proportional hazard analysis (p=1.49e-9) that accounts for survival time bias. This was similarly observed in the tumor-specific subcohorts for lung cancers (n=449, HR 0.45, p=0.0001) and melanoma (n=122, HR 0.25, p=0.0071) (figure 1D,E).

    Clinical characteristics associated with rwOS

    To identify additional risk factors between clinical variables and rwOS, a univariate multiple phenotype association study was conducted, with adjustment for age (figure 2A). Development of any skin and subcutaneous tissue irAE (HR 0.24, p=2.41e-6) and endocrine irAE (HR 0.33, p=0.0001), driven by thyroid irAEs (HR 0.38, p=0.001), was associated with improved rwOS (figure 2B,C). The association between these organ-specific irAEs and survival was maintained in a multivariate time-dependent Cox proportional hazard analysis (skin, p=4.26e-6; endocrine, p=8.70e-5).

    Figure 2
    Figure 2

    Univariate risk analysis for survival outcomes. (A) Univariate Cox survival graph for age-adjusted clinical risk factors, directionality of the arrow indicates association with either improved or worse survival. HRs are provided for variables if nominally significant (p<0.05 prior to adjustment for multiple comparisons, gray dashed line) or significant after adjusting for multiple comparisons (Bonferroni adjusted, purple dashed line); (B) Kaplan-Meier survival analysis of all patients for any skin or subcutaneous tissue irAE incidence. As in figure 1, statistical significance for this association is shown for both a Cox proportional hazards model adjusted for age (p) and time-dependent Cox proportional hazard model adjusting for age and metastatic disease (p(time dependent)); (C) Kaplan-Meier survival analysis of all patients for any endocrine irAE incidence. Again, significance from both analytic approaches is shown; (D) Kaplan-Meier survival analysis of all patients for baseline corticosteroid use and route of administration. Time-dependent Cox proportional hazards not done as the use of the steroid preceded ICI initiation. ICI, immune checkpoint inhibitor; irAE, immune-related adverse event; MSK, musculoskeletal; PD-1, programmed cell death protein-1; PD-L1, programmed death-ligand 1.

    Patients who reported taking any form of corticosteroids at baseline (n=298/1028), prior to initiating ICI treatment, had significantly reduced survival than those who did not (HR 1.37, p=0.0054). This association was more pronounced in patients taking systemic steroids (n=68/1028, HR 1.75, p=0.0022), and was not observed in individuals receiving non-systemic corticosteroids only (n=87/1028, HR 1.16, p=0.44) (figure 2D). Of the 272 subjects who developed an irAE, 68 were on any baseline steroids and there was no association observed between steroid use and irAE risk (p=0.13). The association between baseline steroid use and rwOS persisted after adjustment for the development of any irAE (systemic corticosteroids HR 1.66, p=0.0056, any corticosteroids HR 1.32, p=0.014)

    Clinical characteristics associated with risk of irAEs

    Clinical variables were also assessed for potential association with the development of any or severe grade III–IV irAEs (figure 3). Using logistic regression, adjusting for age at time of ICI treatment, a decreased risk of any grade of irAE was observed in patients receiving PD-(L)1 inhibitors alone compared with CTLA-4/PD-(L)1 inhibitors in combination (OR 0.24, p=2.8e-7). Similarly, the risk of severe irAEs was also significantly decreased following exposure to PD-(L)1 inhibitor monotherapy compared with combination CTLA-4/PD-(L)1 inhibitors (OR 0.24, p=6.99e-4). This was consistent with outcomes from clinical trials that compare single-agent and combination treatment.5

    Figure 3
    Figure 3

    Univariate risk analysis for irAE occurrences. Univariate predictors of any irAE incidence (pink arrow) and any severe Grade III–IV irAE (purple arrow); directionality of the arrow indicates association with higher or lower risk of irAE. ORs are provided for variables if nominally significant (p<0.05 prior to adjustment, gray dashed line) or significant after adjusting for multiple comparisons (Bonferroni adjusted, purple dashed line). IrAE, immune-related adverse event; PD-1, programmed cell death protein-1; PD-L1, programmed death-ligand 1.

    Use of chemotherapy prior to or along with ICIs is indicated in only a subset of cancers. Within the RADIOHEAD cohort, lung cancer (NSCLC and SCLC) was the predominant cancer with prior treatment with chemotherapy (124 with prior chemotherapy, 238 without prior chemotherapy) and/or being treated with concurrent chemotherapy and ICI therapy (163 received conventional chemotherapy with anti-PD-(L)1, 258 anti-PD-(L)1 monotherapy and 41 combination anti-CTLA-4/PD-(L)1). Within patients with lung cancer, ICI with concurrent chemotherapy was associated with decreased risk of irAE (OR 0.49, unadjusted p value=0.005). Concurrent chemotherapy is indicated in advanced lung cancer, and once adjusted for stage, there was no association with overall survival (unadjusted p value=0.31). We also identified that prior treatment with chemotherapy across all cancer types reduced the risk of any irAE (OR 0.59, p=0.0005, pB=0.0431), with severe irAEs showing a consistent but non-significant trend (OR 0.49, p=0.17). Prior chemotherapy within the lung cancer-specific subcohort was not associated with irAE risk (p=0.86 for any irAEs, p=0.86 for severe irAEs). There was no definite association between prior chemotherapy and response. Within the overall cohort, after adjustment for stage, prior chemotherapy was associated with rwOS but did not pass multiple comparison correction (HR 1.26, unadjusted p 0.024, pB=1.0). Within the lung cancer cohort, after adjustment for stage, prior chemotherapy was not associated with rwOS (unadjusted p 0.91, pB=1.0).

    Increased risk of any irAE was associated with previous exposure to the zoster vaccine (OR 2.4, p=5.2e-5, pB=0.0044). Despite this association with any irAEs, severe irAE was not associated with zoster vaccination. In those who received a zoster vaccination, 39.7% developed an irAE, compared with 21.7% developing an irAE if not vaccinated (table 1).

    There was no association between the development of any or severe irAEs and sex, region in which clinical care was performed, vaccination against pneumococcus, or influenza within the last year, baseline corticosteroid use, antibiotic exposure within 30 days of ICI initiation, or other reported comorbidities.

    Discussion

    To our knowledge, this is the first large-scale, prospective real-world study of standard of care, first-line ICI treatment with broad collection of clinical features paired with robust longitudinal blood sampling of patients with cancer. In this initial analysis, we aimed to identify the complex connectivity between reported patient demographics and therapeutic response in the RADIOHEAD cohort. As this cohort is drawn from community centers across the USA and the eligibility criteria were broader than for clinical trials, it is generalizable to most Americans. While this study is limited by variably sized groups for each cancer type, lower than expected irAE rates and some missing data around vaccination status, this work will be critical to future applications of the prospectively collected longitudinal samples. Consistent with cancer frequency and FDA-approved clinical indications, NSCLC was the most frequent cancer receiving ICI treatment followed by melanoma. Single-agent PD-(L)1 inhibitors were the most common form of ICI administered. We also see confirmation of the known natural history of disease and ICI efficacy, with survival differing by cancer type, with better survival outcomes in melanoma than patients with NSCLC.14

    The association between improved antitumor immunity and irAE incidence has been increasingly suggested,7 15 16 which we clearly demonstrate in this well-powered RADIOHEAD cohort with appropriate statistical methods to account for survival bias. Additionally, we further identified that organ-specific irAEs of the skin and endocrine organs were particularly associated with improved therapeutic response. For both skin and endocrine irAEs, the need for ICI discontinuation is often limited, and the treatment options rely primarily on topical immunosuppressants and replacement hormones, respectively. This contrasts with irAEs affecting other organs or tissue sites, in which there may be a greater need for administration of systemic corticosteroids. While systemic corticosteroids have been suggested by some to not impact ICI treatment outcomes, this may not adequately account for the survival advantage that the indication for which they are being received provides. This form of confounding by indication is challenging to avoid, but there are examples that show the potential that corticosteroids attenuate the antitumor immunity. In individuals developing ICI-induced hypophysitis, significantly worse survival outcomes were observed for patients receiving high concentrations of corticosteroids compared with individuals receiving them in a physiological range.17

    In the RADIOHEAD cohort, pretreatment exposure to systemic corticosteroids led to worse survival outcomes which has been established in multiple other smaller studies without affecting rates of irAEs.18 19 Further study should be undertaken to address if switching patients to steroid-sparing immunomodulatory agents prior to ICI initiation would improve their response and survival without affecting the underlying disease requiring the steroid and irAEs.

    Aside from corticosteroid exposure, prior studies have also demonstrated associations between baseline antibiotic use and worsened overall survival,18–21 which RADIOHEAD did not replicate. Baseline antibiotic use within the last 30 days was relatively low (11.3%), and we did see a similar trend to prior studies, although it was not significant (HR 1.12, p=0.5).

    Overall, there were lower rates of irAEs in this standard of care patient population than what is generally reported for clinical trials and retrospective studies, particularly from academic centers.5 22 23 This highlights the importance of the RADIOHEAD cohort to help identify challenges with irAE diagnosis in the community hospital setting. While the top five most common irAEs in this cohort, rash, thyroid dysfunction, colitis/diarrhea, pneumonitis and hepatitis align with the top five irAEs in other cohorts, the rates across all types are lower.5 22 23 Differences in rates of severe irAEs were smaller, but the number of subjects with severe irAEs is also lower. This lower rate was apparent even for irAEs where extraction should be well defined, such as for thyroid dysfunction where both laboratory changes and medication initiation are performed for all but grade 1 disease, compared with other irAEs where lower grades may be monitored rather than intervened on which could lead to lower identification by chart extractors. Possible reasons for these lower irAE rates include that these patients have had fewer prior treatments, including tyrosine kinase inhibitor use which has been seen to increase rates of irAEs,24 25 less frequent clinical visits or lab testing, or differing risk patterns and potential exposure to immunosuppressive medications in these standard of care patients than clinical trial patients leading to an increase in under-reporting. It is unclear whether this is due to under-reporting, and potentially inadequate diagnosis through reduced involvement of subspecialties in the community hospital setting, or if in fact there are lower rates in community settings than in trial settings. If the latter is true, ICI safety profiles should be revisited in this context. The lower than anticipated irAE rates did decrease the power of this study, particularly for rare irAEs.

    Despite these lower irAE rates, we were able to identify risk factors for irAEs. As previously well established, rates are higher with combination immunotherapy compared with anti-PD-(L)1 monotherapy.2 26 Interestingly, both prior use of chemotherapy and concurrent use of chemotherapy in patients with lung cancer were associated with decreased risk of irAE. Given the decrease in immune cells from conventional chemotherapy, there is a potential mechanistic rationale that this could be related to a decrease in the presence of autoreactive T cells. Chemotherapy has also been used in some cases to reset the immune system and reduce inflammation in patients with autoimmune disease, including scleroderma, systemic lupus erythematosus, and certain types of arthritis. Most clinical trials have largely assessed standard of care chemotherapy alone to chemoimmunotherapy with the absence of an immunotherapy alone arm, which has led to limited information relating to differences in the irAE profile with and without chemotherapy. Retrospective analyses have identified increased irAEs and treatment discontinuation in lung cancer when ICI is combined with chemotherapy in some27 28 but not all cohorts.29 Additionally, a clinical trial using first-line tremelimumab (anti-CTLA-4), durvalumab (anti-PD-L1), and chemotherapy versus durvalumab and chemotherapy in metastatic NSCLC revealed comparative frequency and severity of treatment-related adverse events and similar rates of treatment discontinuation.30 This may indicate that chemotherapy has the potential to modify the adverse event profile and even restrain the increase in adverse events usually associated with multiple lines of immunotherapy.2 26 Alternatively, the rationale for treating a patient with chemoimmunotherapy, such as tumor type (low tumor mutation burden, low PD-L1 expression) and/or patient status (high disease burden requiring rapid response), may also select for patients that are less likely to respond to ICI treatment, which may also be indicative of reduced risk for irAEs. This suggests the need for greater interrogation of the impact of chemotherapy on both ICI-induced irAE frequency and type to determine whether it impacts certain irAEs, particularly those that are autoimmune in nature, preferentially.

    We found an association between prior zoster vaccine and development of any irAE, but not severe irAEs. There was no association between prior zoster vaccine and overall survival despite this association with irAE (p=0.38). A limitation of this finding is that the rate of zoster vaccination in this population was only 14.1%, and status was unknown for 36.8% of the cohort. The rates of irAEs in this group with unknown vaccination status were between the irAE rate in the unvaccinated and vaccinated cohort, potentially suggesting a mixed group. Prior literature has largely reported a lack of associations between irAEs and inactivated vaccines,31 COVID-19 vaccination,32 33 or seasonal influenza vaccines.34–39 There is a rare association of Guillain-Barré Syndrome, an autoimmune disease, with zoster vaccine and perhaps this link could help explain, in part, the irAE association40 but further study is warranted. In the USA, the commonly used Shingrix vaccine is a recombinant, inactivated vaccine which replaced the prior vaccine, Zostavax, a live-attenuated vaccine, in 2017. In 2021, at the tail end of this study, the FDA expanded the indication for the Shingrix vaccine to include adults undergoing conventional chemotherapy due to immunosuppression. Patients were not asked which vaccine type they had received, which limits the ability to hypothesize on mechanism. That said, live vaccines could increase broader adaptive immunity with more chance for cross-reactivity with self.41 The recombinant vaccine combines a varicella-zoster virus (VZV) antigen glycoprotein with an immune-stimulating adjuvant that increases interferon gamma release by natural killer cells and both CD4+ and CD8+ T cells. There has been no increased risk of autoimmune disease with this adjuvant,42 but there is also evidence of both T cell-mediated and humoral immune responses to the recombinant vaccine for at least 9 years postvaccination. While the timing of the vaccination relative to ICI start is not available, there was no association between zoster vaccination and chemotherapy (p=0.35) or zoster vaccination and cancer type (p=0.42). Intriguingly, there is evidence for an association between seasonal influenza vaccine and improved overall survival in the existing literature, however, we did not find this for the zoster vaccine39 and did not show this association with the seasonal influenza vaccine within the RADIOHEAD cohort.

    In conclusion, results from this large, real-world cohort study of standard of care patients definitively identify previously suggested variables associated with ICI response and novel risk factors which may impact patient care. Clinical risk factors that enhance treatment selectivity by incorporating both treatment response and irAE risk are essential to deepen our understanding of ICI efficacy. Many of the studies to investigate this have employed small cohort or retrospective analyses, that have not been tailored towards capturing broad clinical features alongside opportunities to probe molecular mechanisms for clinical outcomes. RADIOHEAD was designed to be able to answer both questions simultaneously. In the future, the potential for molecular and immune profiling of samples collected in the RADIOHEAD cohort provides an opportunity to elucidate the potential mechanistic link between irAE and clinical response to ICI, as well as identify clinically actionable mechanisms for immunotherapy resistance via integrated analyses of multiomic datasets to be generated from longitudinal patient samples.

    Data availability statement

    Data are available upon reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study involves human participants and was approved by WCG IRB Protocol #20182579. Participants gave informed consent to participate in the study before taking part.

    Acknowledgments

    We thank JDRF and the Helmsley Charitable Trust for their support of this project. We would also like to thank Benjamin Williams, Esther Latres, Silvia Boffo, Lisa Salvador, David Oh, Jane Buckner, Osama Rahma, Cheryl Selinsky, Chris Perry, Lance Leong, Kristy Lagarde, and Parker Institute colleagues for their support and guidance. We are grateful to all patients, their families, and the sites who participated in the study.

    References

    1. ↵
      2. Young A ,
      3. Quandt Z ,
      4. Bluestone JA
      . The Balancing Act between Cancer Immunity and Autoimmunity in Response to Immunotherapy. Cancer Immunol Res 2018;6:1445–52. doi:10.1158/2326-6066.CIR-18-0487
      OpenUrlAbstract/FREE Full Text
    2. ↵
      2. Postow MA ,
      3. Sidlow R ,
      4. Hellmann MD
      . Immune-Related Adverse Events Associated with Immune Checkpoint Blockade. N Engl J Med 2018;378:158–68. doi:10.1056/NEJMra1703481
      OpenUrlCrossRefPubMed
    3. ↵
      2. Johnson DB ,
      3. Nebhan CA ,
      4. Moslehi JJ , et al
      . Immune-checkpoint inhibitors: long-term implications of toxicity. Nat Rev Clin Oncol 2022;19:254–67. doi:10.1038/s41571-022-00600-w
      OpenUrlCrossRefPubMed
    4. ↵
      2. Horvat TZ ,
      3. Adel NG ,
      4. Dang T-O , et al
      . Immune-Related Adverse Events, Need for Systemic Immunosuppression, and Effects on Survival and Time to Treatment Failure in Patients With Melanoma Treated With Ipilimumab at Memorial Sloan Kettering Cancer Center. J Clin Oncol 2015;33:3193–8. doi:10.1200/JCO.2015.60.8448
      OpenUrlAbstract/FREE Full Text
    5. ↵
      2. Larkin J ,
      3. Chiarion-Sileni V ,
      4. Gonzalez R , et al
      . Five-Year Survival with Combined Nivolumab and Ipilimumab in Advanced Melanoma. N Engl J Med 2019;381:1535–46. doi:10.1056/NEJMoa1910836
      OpenUrlCrossRefPubMed
    6. ↵
      2. Tawbi HA ,
      3. Schadendorf D ,
      4. Lipson EJ , et al
      . Relatlimab and Nivolumab versus Nivolumab in Untreated Advanced Melanoma. N Engl J Med 2022;386:24–34. doi:10.1056/NEJMoa2109970
      OpenUrlCrossRefPubMed
    7. ↵
      2. Dall’Olio FG ,
      3. Di Nunno V ,
      4. Massari F
      . Immortal Time Bias Question in the Association Between Toxicity and Outcome of Immune Checkpoint Inhibitors. J Clin Oncol 2020;38:105–6. doi:10.1200/JCO.19.01728
      OpenUrl
    8. ↵
      2. Haslam A ,
      3. Gill J ,
      4. Prasad V
      . Estimation of the Percentage of US Patients With Cancer Who Are Eligible for Immune Checkpoint Inhibitor Drugs. JAMA Netw Open 2020;3:e200423. doi:10.1001/jamanetworkopen.2020.0423
    9. ↵
      2. Parikh RB ,
      3. Min EJ ,
      4. Wileyto EP , et al
      . Uptake and Survival Outcomes Following Immune Checkpoint Inhibitor Therapy Among Trial-Ineligible Patients With Advanced Solid Cancers. JAMA Oncol 2021;7:1843–50. doi:10.1001/jamaoncol.2021.4971
      OpenUrl
    10. ↵
      2. Harris PA ,
      3. Taylor R ,
      4. Minor BL , et al
      . The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019;95:103208. doi:10.1016/j.jbi.2019.103208
      OpenUrlCrossRefPubMed
    11. ↵
      2. Kamphaus B ,
      3. Thompson M
      . Four years of datomic powered etl in anger with Candel. In: ClojureConj. Clojure, Durham, 2023.
    12. ↵
      2. Brown EG ,
      3. Wood L ,
      4. Wood S
      . The Medical Dictionary for Regulatory Activities (MedDRA). Drug Saf 1999;20:109–17. doi:10.2165/00002018-199920020-00002
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. CDC National Center for Health Statistics
      . Geographic division or region. 2023. Available: https://www.cdc.gov/nchs/hus/sources-definitions/geographic-region.htm
    14. ↵
      2. Topalian SL ,
      3. Hodi FS ,
      4. Brahmer JR , et al
      . Five-Year Survival and Correlates Among Patients With Advanced Melanoma, Renal Cell Carcinoma, or Non-Small Cell Lung Cancer Treated With Nivolumab. JAMA Oncol 2019;5:1411–20. doi:10.1001/jamaoncol.2019.2187
      OpenUrl
    15. ↵
      2. Hussaini S ,
      3. Chehade R ,
      4. Boldt RG , et al
      . Association between immune-related side effects and efficacy and benefit of immune checkpoint inhibitors – A systematic review and meta-analysis. Cancer Treat Rev 2021;92:102134. doi:10.1016/j.ctrv.2020.102134
      OpenUrlCrossRefPubMed
    16. ↵
      2. Watson AS ,
      3. Goutam S ,
      4. Stukalin I , et al
      . Association of Immune-Related Adverse Events, Hospitalization, and Therapy Resumption With Survival Among Patients With Metastatic Melanoma Receiving Single-Agent or Combination Immunotherapy. JAMA Netw Open 2022;5:e2245596. doi:10.1001/jamanetworkopen.2022.45596
    17. ↵
      2. Faje AT ,
      3. Lawrence D ,
      4. Flaherty K , et al
      . High-dose glucocorticoids for the treatment of ipilimumab-induced hypophysitis is associated with reduced survival in patients with melanoma. Cancer 2018;124:3706–14. doi:10.1002/cncr.31629
      OpenUrlCrossRefPubMed
    18. ↵
      2. Nara K ,
      3. Taguchi S ,
      4. Buti S , et al
      . Associations of concomitant medications with immune-related adverse events and survival in advanced cancers treated with immune checkpoint inhibitors: a comprehensive pan-cancer analysis. J Immunother Cancer 2024;12:e008806. doi:10.1136/jitc-2024-008806
    19. ↵
      2. Cortellini A ,
      3. Di Maio M ,
      4. Nigro O , et al
      . Differential influence of antibiotic therapy and other medications on oncological outcomes of patients with non-small cell lung cancer treated with first-line pembrolizumab versus cytotoxic chemotherapy. J Immunother Cancer 2021;9:e002421. doi:10.1136/jitc-2021-002421
    20. ↵
      2. Derosa L ,
      3. Hellmann MD ,
      4. Spaziano M , et al
      . Negative association of antibiotics on clinical activity of immune checkpoint inhibitors in patients with advanced renal cell and non-small-cell lung cancer. Ann Oncol 2018;29:1437–44. doi:10.1093/annonc/mdy103
      OpenUrlCrossRefPubMed
    21. ↵
      2. Eng L ,
      3. Sutradhar R ,
      4. Niu Y , et al
      . Impact of Antibiotic Exposure Before Immune Checkpoint Inhibitor Treatment on Overall Survival in Older Adults With Cancer: A Population-Based Study. J Clin Oncol 2023;41:3122–34. doi:10.1200/JCO.22.00074
      OpenUrl
    22. ↵
      2. El Osta B ,
      3. Hu F ,
      4. Sadek R , et al
      . Not all immune-checkpoint inhibitors are created equal: Meta-analysis and systematic review of immune-related adverse events in cancer trials. Crit Rev Oncol Hematol 2017;119:1–12. doi:10.1016/j.critrevonc.2017.09.002
      OpenUrlCrossRefPubMed
    23. ↵
      2. Wolchok JD ,
      3. Chiarion-Sileni V ,
      4. Gonzalez R , et al
      . Overall Survival with Combined Nivolumab and Ipilimumab in Advanced Melanoma. N Engl J Med 2017;377:1345–56. doi:10.1056/NEJMoa1709684
      OpenUrlCrossRefPubMed
    24. ↵
      2. Tsai K ,
      3. Ma H ,
      4. Liang TZ , et al
      . The Combined Effect of Immune Checkpoint Inhibitors and Tyrosine Kinase Inhibitors on Thyroid Function. Thyroid 2024;34:158–66. doi:10.1089/thy.2023.0542
      OpenUrl
    25. ↵
      2. Schoenfeld AJ ,
      3. Arbour KC ,
      4. Rizvi H , et al
      . Severe immune-related adverse events are common with sequential PD-(L)1 blockade and osimertinib. Ann Oncol 2019;30:839–44. doi:10.1093/annonc/mdz077
      OpenUrlCrossRefPubMed
    26. ↵
      2. Larkin J ,
      3. Chiarion-Sileni V ,
      4. Gonzalez R , et al
      . Combined Nivolumab and Ipilimumab or Monotherapy in Untreated Melanoma. N Engl J Med 2015;373:23–34. doi:10.1056/NEJMoa1504030
      OpenUrlCrossRefPubMed
    27. ↵
      2. Shin S ,
      3. Moon J ,
      4. Oum C , et al
      . Discontinuation risk from adverse events: immunotherapy alone vs. combined with chemotherapy: a systematic review and network meta-analysis. BMC Cancer 2024;24:152. doi:10.1186/s12885-024-11897-4
    28. ↵
      2. Kurokawa K ,
      3. Mitsuishi Y ,
      4. Shimada N , et al
      . Association between the efficacy and immune-related adverse events of pembrolizumab and chemotherapy in non-small cell lung cancer patients: a retrospective study. BMC Cancer 2022;22:1047. doi:10.1186/s12885-022-10133-1
    29. ↵
      2. Cook S ,
      3. Samuel V ,
      4. Meyers DE , et al
      . Immune-Related Adverse Events and Survival Among Patients With Metastatic NSCLC Treated With Immune Checkpoint Inhibitors. JAMA Netw Open 2024;7:e2352302. doi:10.1001/jamanetworkopen.2023.52302
    30. ↵
      2. Johnson ML ,
      3. Cho BC ,
      4. Luft A , et al
      . Durvalumab With or Without Tremelimumab in Combination With Chemotherapy as First-Line Therapy for Metastatic Non–Small-Cell Lung Cancer: The Phase III POSEIDON Study. JCO 2023;41:1213–27. doi:10.1200/JCO.22.00975
      OpenUrlCrossRefPubMed
    31. ↵
      2. Kurkowski A
      . Safety of Inactivated Vaccines in Patients Receiving Immune Checkpoint Inhibitors. J Hematol Oncol Pharm 2021;11:186–93.
      OpenUrl
    32. ↵
      2. Ruiz JI ,
      3. Lopez-Olivo MA ,
      4. Geng Y , et al
      . COVID-19 vaccination in patients with cancer receiving immune checkpoint inhibitors: a systematic review and meta-analysis. J Immunother Cancer 2023;11:e006246. doi:10.1136/jitc-2022-006246
    33. ↵
      2. Widman AJ ,
      3. Cohen B ,
      4. Park V , et al
      . Immune-Related Adverse Events Among COVID-19-Vaccinated Patients With Cancer Receiving Immune Checkpoint Blockade. J Natl Compr Canc Netw 2022;20:1134–8. doi:10.6004/jnccn.2022.7048
      OpenUrl
    34. ↵
      2. Chong CR ,
      3. Park V ,
      4. Harding JJ , et al
      . Safety of influenza vaccination in patients undergoing immunotherapy treatment for advanced cancer. JCO 2018;36:e15073. doi:10.1200/JCO.2018.36.15_suppl.e15073
    35. ↵
      2. Lopez-Olivo MA ,
      3. Valerio V ,
      4. Karpes Matusevich AR , et al
      . Safety and Efficacy of Influenza Vaccination in Patients Receiving Immune Checkpoint Inhibitors. Systematic Review with Meta-Analysis. Vaccines (Basel) 2022;10:1195. doi:10.3390/vaccines10081195
    36. ↵
      2. Wijn DH ,
      3. Groeneveld GH ,
      4. Vollaard AM , et al
      . Influenza vaccination in patients with lung cancer receiving anti-programmed death receptor 1 immunotherapy does not induce immune-related adverse events. Eur J Cancer 2018;104:182–7. doi:10.1016/j.ejca.2018.09.012
      OpenUrl
    37. ↵
      2. Grima AA ,
      3. Kwong JC ,
      4. Richard L , et al
      . The safety of seasonal influenza vaccination among adults prescribed immune checkpoint inhibitors: A self-controlled case series study using administrative data. Vaccine (Auckl) 2024;42:1498–505. doi:10.1016/j.vaccine.2024.01.023
      OpenUrl
    38. ↵
      2. Läubli H ,
      3. Balmelli C ,
      4. Kaufmann L , et al
      . Influenza vaccination of cancer patients during PD-1 blockade induces serological protection but may raise the risk for immune-related adverse events. J Immunother Cancer 2018;6:40. doi:10.1186/s40425-018-0353-7
      OpenUrlAbstract/FREE Full Text
    39. ↵
      2. Bersanelli M ,
      3. Verzoni E ,
      4. Cortellini A , et al
      . Impact of influenza vaccination on survival of patients with advanced cancer receiving immune checkpoint inhibitors (INVIDIa-2): final results of the multicentre, prospective, observational study. EClinicalMedicine 2023;61:102044. doi:10.1016/j.eclinm.2023.102044
    40. ↵
      1. Prescribing Instructions
      . SHINGRIX (Zoster Vaccine Recombinant, Adjuvanted), suspension for intramuscular injection [Preprint]. GlaxoSmithKline Biologicals 2024.
    41. ↵
      2. Benn CS ,
      3. Amenyogbe N ,
      4. Björkman A , et al
      . Implications of Non-Specific Effects for Testing, Approving, and Regulating Vaccines. Drug Saf 2023;46:439–48. doi:10.1007/s40264-023-01295-3
      OpenUrlCrossRefPubMed
    42. ↵
      2. Heineman TC ,
      3. Cunningham A ,
      4. Levin M
      . Understanding the immunology of Shingrix, a recombinant glycoprotein E adjuvanted herpes zoster vaccine. Curr Opin Immunol 2019;59:42–8. doi:10.1016/j.coi.2019.02.009
      OpenUrlCrossRefPubMed

    Footnotes

    • X @Parker Institute for Cancer Immunotherapy

    • ZQ, AL and SIL contributed equally.

    • Contributors ZQ, AL, SIL, EY, SS, and MZHF curated the clinical data. ZQ, SIL, and AL conducted statistical analyses, interpreted the results, and contributed to drafting the manuscript. SIL, LJP, SB, and JEC supervised the research team and coordinated project activities, while NLB, SSM, SB, KCH, JAB, and MSA developed the study methodology and oversaw project planning. LHB, JEC, JAB, and MSA provided supervision, interpreted statistical results, and secured financial support for the project. ACCT and CS contributed to data analysis, manuscript revisions, and drafting. AY developed the study methodology and contributed to both statistical interpretation and manuscript preparation. All authors reviewed and edited the manuscript for intellectual content. ZQ is the guarantor.

    • Funding Supported by the Leona M and Harry B Helmsley Charitable Trust (G-2933), JDRF (3-PAR-2019-735-S-N), Bristol Myers Squibb, and the Parker Institute for Cancer Immunotherapy. ZQ was supported by the American Diabetes Association (ADA 1-19-PDF-131), Larry L Hillblom Foundation Start Up Grant, and K12DK133995 (David Maahs, Linda DiMeglio, Multicenter Program Directors) from the National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases, Physician–Scientist Career Development Award. AY was supported by the Huntsman Cancer Foundation and the National Cancer Institute of the National Institutes of Health under Award Number R00CA246061 and P30CA042014. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    • Competing interests ZQ has consulted for Sanofi and Novartis. MSA owns stock in Merck and Medtronic. SIL owns stock in AbbVie. LHB has served on advisory boards for Calidi Biotherapeutics, Western Oncolytics/Kalivir, Torque Therapeutics/Repertoire, Pyxis, CytomX, Roche-Genentech (Biomarkers Roundtable), Khloris, Takeda Oncology, Adaptimmune, RAPT, and DCPrime. LHB holds stocks or shares in Repertoire, Pyxis, RAPT, and CytomX.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

    • ↵Bonferroni-corrected p-value