Report

Ram C. Tiwari Associate Director Office of Biostatistics, CDER, FDA [email protected] Disclaimer This presentation reflects the views of the author and should not be construed to represent FDA’s views or policies. Benefit-risk Assessment 2 Benefit-risk Assessment 3 Outline Introduction Commonly-used Benefit-risk (BR) measures Methodology BR measures based on Global benefit-risk (GBR) scores and a new measure Bayesian approaches Power prior Illustration and simulation study Future work Benefit-risk Assessment 4 Introduction The benefit-risk assessment is the basis of regulatory decisions in the pre- market and post market review process. The evaluation of benefit and risk faces several challenges. Benefit-risk Assessment 5 Commonly used B-R measures Various measures have been proposed to assess benefit and risk simultaneously: Q-TWiST by Gelbert et al. (1989) Ratio of benefit and risk by Payne (1975) The Number Needed to Treat and the Number Needed to Harm by Holden et al. (2003) Global Benefit Risk (GBR) scores by Chuang-Stein et al. (1991) Benefit-risk Assessment 6 BR categories A five-category multinomial random variable to capture the benefit and risk of a drug product on each individual simultaneously: Table 1: Possible outcomes of a clinical trial with binary response data Benefit No benefit No AE Category 1 Category 3 AE Category 2 Category 4 withdrawal Benefit-risk Assessment Category 5 7 Example: Hydromorphone Data was provided by Jonathan Norton. Benefit-risk Assessment 8 GBR scores 2 5 i 1 i 3 BRScore _ Linear wi pi wi pi 2 BRScore _ Ratio ( wi pi ) e i 1 5 w p i 3 i BRScore _ Cmp _ Ratio i w1 p1 w2 p2 ( )f w5 p5 w3 p3 w4 p4 where w1 2, w2 1, w3 0, w4 1, w5 2 Benefit-risk Assessment 9 Methodology: BR measures BR measures based on the global scores proposed by Chuang-Stein et al. 2 5 i 1 i 3 2 5 i 1 i 3 BR _ Linear ( wi pi ,T wi pi ,T ) ( wi pi ,C wi pi ,C ) 2 BR _ Ratio ( wi pi ,T ) e i 1 5 w p i 3 i BR _ Cmp _ Ratio 2 ( wi pi ,C ) e i 1 5 w p i ,T w1 p1,T i 3 ( i i ,C w2 p2,T w5 p5,T w3 p3,T w4 p4,T )f w1 p1,C ( w2 p2,C w5 p5,C w3 p3,C w4 p4,C )f where w1 2, w2 1, w3 0, w4 1, w5 2 BR measures based on the global scores are for each arm (treatment and comparator) separately. BR_Linear can take a continuous value on a scale of -4 to 4 (inclusive). Benefit-risk Assessment 10 Methodology: New BR measure A new indicator based measure is proposed: 2 5 i 1 i 3 BR _ Indicator wi ( pi ,T , pi ,C ) wi ( pi ,T , pi ,C ) 1 if a b where (a, b) 0 if a b 1 if a b BR_Indicator compares two arms simultaneously. It takes a integer value between -6 to 6 (inclusive). Benefit-risk Assessment 11 Methodology: Dirichlet prior Dirichlet distribution is used as the conjugate prior for multinomial distribution, and the posterior distribution of the five-category random variable is derived at each visit using sequentially updated posterior as a prior. Benefit-risk Assessment 12 Methodology: Sequential Updating Sequential updating of the posteriors are given by: The posterior mean (i.e., Bayes estimate) and 95% credible interval for each of the four measures are obtained using a Markov chain Monte Carlo (MCMC) technique. Benefit-risk Assessment 13 Methodology: Decision Rules For a BR measure, If the credible interval include the value zero, the benefit does not outweigh the risk; If the lower bound of the credible interval is greater than zero, the benefit outweighs the risk; If the upper bound of the credible interval is less than zero, the risk outweighs the benefit. Benefit-risk Assessment 14 Methodology: Power Prior Power prior (Ibrahim and Chen, 2000) is used through the likelihood function to discount the information from previous visits, and the posterior distribution of the five-category random variable is obtained using the Dirichlet prior for p and a Beta (1, 1) as a power prior for a0 . Benefit-risk Assessment 15 Methodology: Model Fit The model fit of the two models (with and without power prior) is assessed through the conditional predictive ordinate (CPO) and the logarithm of the pseudo-marginal likelihood (LPML). The larger the value of LPML, the better fit the model is. Here, n(i) is the data with ni removed. Benefit-risk Assessment 16 Back to our example: Hydromorphone Benefit-risk Assessment 17 3 Illustration: Posterior Means and 95% Credible Intervals for BR_Linear Measure Benefit 4 1 0 1 2 3 4 5 6 7 8 -1 -2 without power prior with power prior -3 -4 Risk BR Linear Measure 2 Visit Benefit-risk Assessment 18 Illustration: Posterior Means and 95% Credible Intervals for BR_Indicator Measure Benefit 6 2 0 1 2 3 4 5 6 7 8 -2 -4 without power prior with power prior Risk BR Indicator Measure 4 -6 Visit Benefit-risk Assessment 19 Illustration: Results a. The model without power prior Benefit-risk Assessment b. The model with power prior 20 Illustration: Posterior Means and 95% Credible Intervals for Power Prior Parameter Benefit-risk Assessment 21 Illustration: Model Fit LPML values Model without power prior Model with power prior Benefit-risk Assessment Treatment Control -14.230 -14.209 -6.432 -6.190 22 Simulation study Correlated longitudinal multinomial data are simulated using the R package SimCorMultRes.R, which uses an underlying regression model to draw correlated ordinal response. Two scenarios are simulated: The treatment arm is similar to the control arm in terms of benefit-risk; The treatment arm is better than control arm in the sense that the benefit outweighs risk. Benefit-risk Assessment 23 Simulation study: Scenarios Scenario 1: Treatment benefit does not outweigh risk compared to control Scenario 2: Treatment benefit outweighs risk compared to control Benefit-risk Assessment 24 Simulation study: Scenario 1 Treatment benefit does not outweigh risk compared to control a. The model without power prior Benefit-risk Assessment b. The model with power prior 25 Simulation study: Scenario 2 Treatment benefit outweighs risk compared to control a. The model without power prior Benefit-risk Assessment b. The model with power prior 26 Simulation study: Results Scenario 1: Treatment benefit does not outweigh risk compared to control Scenario 2: Treatment benefit outweighs risk compared to control Benefit-risk Assessment 27 Simulation study: Model Fit LPML values Treatment Control -23.536 -23.354 -8.472 -7.667 -27.099 -21.840 -8.532 -8.393 Scenario 1: Model without power prior Model with power prior Scenario 2: Model without power prior Model with power prior Benefit-risk Assessment 28 Benefit-risk Assessment 29 Future work in BR assessment Frequentist approaches: Bootstrap approach General linear mixed model (GLMM) approach Other Bayesian approaches: Normal priors Dirichlet process Benefit-risk Assessment 30 Bootstrap Approach Approximate underlying distribution using the empirical distribution of the observed data; Resample from the original dataset; Calculate the estimates and confidence intervals (CIs) of the BR measures based on the bootstrap samples; Percentile bootstrap CIs; Basic bootstrap CIs; Studentized bootstrap CIs; Bias-Corrected and Accelerated CIs. Apply the decision rules. Benefit-risk Assessment 31 Bootstrap Approach-Results Benefit-risk Assessment 32 General linear mixed model (GLMM) approach Within each arm (T or C), the ith subject falls in the jth category (vs. the first category) at kth visit can be modeled as, P(Yik j ) log 0 j k ik P(Yik 1) where, α0 is the baseline effect assumed common across all categories, βj is the category effect, and γk is the longitudinal effect at kth visit, with and, ik ~ N (0, 2.) Benefit-risk Assessment 33 GLMM approach Note that different variance-covariance structures can be used for (γ1,γ2,…γ8), to model the longitudinal trend. Compound-symmetry Power covariance structure Unstructured covariance structure The estimates of the confidence intervals of the global measures can be derived from Monte Carlo samples, and the decision rules can be determined based on the confidence intervals. Benefit-risk Assessment 34 General linear mixed model approach-Results Benefit-risk Assessment 35 Bayesian approaches with GLMM log P(Yik j ) 0 j k ik P(Yik 1) (α0 , βj ; j=1,…,5)~ independent Normal with means 0 and large variances; Variance parameters~ IG Dirichlet Process Approach: Let α0 to depend on subjects, that is, assume that α0i |G ~ iid G, with G~ DP(M, G0), M>0 concentration parameter and G0 a baseline distribution such as a normal with mean 0 and large variance. βj ; j=1,…,5 are independent normal with means 0, and large variances. The posterior distributions of the probability and the global measures can be derived, and the decision rules can be determined based on the credible intervals. Benefit-risk Assessment 36 Discussion Quantitative measure of benefit and risk is an important aspect in the drug evaluation process. The Bayesian method is a natural method for longitudinal data by sequentially updating the prior; Power prior can be used to discount information from previous visits. Frequentist approaches such as bootstrapping method and general linear mixed model can be applied for benefit risk assessment. Continuous research in longitudinal assessment of drug benefit-risk is warranted. Benefit-risk Assessment 37 Benefit-risk Assessment 38 Selected References Gelber RD, Gelman RS, Goldhirsch A. A quality-of-life oriented endpoint for comparing treatments. Biometrics. 1989;45:781-795 Payne JT, Loken MK. A survey of the benefits and risks in the practices of radiology. CRC Crit Rev Clin Radiol Nucl Med. 1975; 6:425-475 Holden WL, Juhaeri J, Dai W. “Benefit-Risk Analysis: A Proposal Using Quantitative Methods,” Pharmacoepidemiology and Drug Safety. 2003; 12, 611–616. 154 Chuang-Stein C, Mohberg NR, Sinkula MS. Three measures for simultaneously evaluating benefits and risks using categorical data from clinical trials. Statistics in Medicine. 1991; 10:1349-1359. Norton, JD. A Longitudinal Model and Graphic for Benefit-risk Analysis, with Case Study. Drug Information Journal. 2011; 45: 741-747. Ibrahim, JG, Chen, MH. Power Prior Distributions for Regression Models. Statistical Science. 2000; 15: 46-60. Benefit-risk Assessment 39 Q&A Benefit-risk Assessment 40