Report

Predictive Modelling of Operational Characteristics at the Design and Monitoring Clinical Trial PSI Annual Conference 11-14 May 2014, London Dr. Vladimir Anisimov Sr Strategic Biostatistics Director Predictive Analytics, Innovation [email protected] Copyright © 2013 Quintiles Background Stochastic behaviour and complexity of trial operation require developing predictive analytic techniques accounting for • major uncertainties in input information • stochasticity of basic processes: enrolment, trial start-up, various events (clinical & non-clinical) Most of trial operational characteristics are driven by patient enrolment which is stochastic by nature. Inefficient enrolment forecast is one of the main reasons of trial failure: - more than 60-70% of trials fail to recruit in time The majority of existing tools in pharma companies still use ad-hoc simplified or deterministic models. 2 Outline Discussion of predictive analytic techniques for • Patient enrolment modelling • Predicting trial/site enrolment performance and risk-based monitoring • Forecasting trial operational characteristics - follow-up patients, visits, events, costs • Event predictive modelling in event-driven trials 3 Patient enrolment Analytic data-driven statistical methodology is developed * Stochastic modelling technique: • • • Patients arrive at sites according to Poisson processes Variation in enrolment rates is modelled using a gamma distribution Delays in site initiation also be random (uniform, beta, gamma) In site i, ni(t) = πλi (t – vi) χ(vi ≤t ≤ bi ) λi = γ(α,β) - rate, vi – delay in site initiation, bi - closure time. Start-up predictions are created using expected or historical data. Interim re-forecasting uses real data and Bayesian re-estimation. Closed-form expressions for mean and predictive bounds are derived (no need in Monte Carlo simulation). Methodology is world-wide accepted, validated on tens of real trials. Basic version is implemented in some pharma companies. * Anisimov & Fedorov, 2005, 2007; Anisimov, 2009-2011, presented at DIA, JSM (invited sessions), ISCB, PSI,... 4 Enrolment forecaster tools The developed techniques and software tools (in C# and R) allow: • Compute mean and credibility bounds for the predictive number of patients recruited over time and for time to complete trial at any stage of the trial • Realize a new paradigm: Plan with confidence e.g. “70% chance that trial will complete enrolment in time” • Evaluate the optimal number of sites needed to complete in time • Advise on adaptive adjustment: o Evaluate Probability to recruit in time; o If study is likely to go late, calculate the number of additional sites needed to complete in time with a given confidence 5 Enrolment adjustment Scenario: 400 pts, 70 sites, time = 365 days. Sites initiated in 5-month period, half of sites will be closed in two months before the end of enrolment. Initial design: to complete with 90% confidence. Predictive area: mean and confidence bounds. Interim analysis after 150 days: 88 pts recruited. Enrolment is slower than predicted. Interim adjustment: to complete with 90% confidence: 22 new sites to add. Adjusted prediction is going on time. 6 Site performance and risk-based monitoring Technique/tools* for forecasting site/trial performance: Triggers for detecting unusual behavior • • Low and High-enrolling sites, Late-start, inactive, high number of AE, etc. Predictive triggers (interim time, data-driven): • • Predict performance Create dynamic forecasts in future periods Opportunity for optimal decision-making: choose the optimal # of sites and allocation accounting for study costs, completion time and risks per delays Current triggers usually use assumptions of normality and detect unusual behaviour of sites within cohort using Mean and SD: X > Mean(cohort) + K*SD(cohort), K=1,2,3 However, many of variables describing operation of clinical trials are rather far from the normal distribution. Thus, using triggers based on Mean and SD will lead to biased results. * Predictive Analytics Team, Quintiles 7 Real data analysis Histogram of the enrolment rate (# of patients)/(site enrolment duration) far from normal distribution, Red dot – 0.004: Mean + 3*SD, heavy tailed. Adequate model – Poisson mixed with gamma Histogram - Time from Last Patient Enrolled till current time – red line – fitted Pareto distribution far from normal distribution, heavy tailed. Adequate model – Exponential mixed with gamma 8 Case study Classification of Low-High enrolling sites using Probability Quantile trigger. Different dots – different sites - 332. Each site is defined by (# of patients, exposure time) Two-dimensional site’s characterization. Probability Quantiles: 95% - red line, 90% - magenta, 80% - brown. Adequate model - Poisson process with gamma rate. Thresholds based on normal approximation: Black linear lines: Mean + SD; Mean + 2*SD; Mean + 3*SD – current trigger 9 Case study: predictive probabilities Interim analysis, real study, 330 active sites. Sites: 1006 and 1017 – low enrollers Sites: 1901 and 1906 –good enrollers Predictive probabilities to enrol: | zero | <= one | <= two | <= three | at least one patient | ML estimation and Bayesian re-estimation. In particular, if at time t0 site i recruited zero patients during time interval vi , then Prob (zero patients at time t) = (α,β) – estimated by ML model parameters 10 Predictive enrolment bounds SID 57 58 59 60 61 62 63 64 66 67 68 69 70 71 72 15 days 2 7 8 2 9 2 9 9 26 8 2 7 7 2 5 30 days 2 12 13 2 14 2 14 14 45 13 2 10 10 2 8 45 days 3 15 17 3 19 3 19 19 63 18 3 14 14 3 10 Real trial, interim look, 544 sites active; 1790 patients screened Predictive 90%-upper bounds for the number of recruited patients in the future. Goal: to predict supply in advance, prepare staff in hospitals,.. Site 66 is highly enrolling; sites 57, 60, 68, 71 – low enrolling 11 Dynamics of non-enrolling sites Blue curve: site’s start-up green curve: mean # of empty sites red curves- 90% predictive bounds Artificial case study: number of patients =450; number of clinical sites =175; time to complete enrolment =9 months; all centres initiated uniformly in 3-month period. At the end: mean 55, low bound = 45, upper bound = 67 Proportion of empty sites at study completion ~ 30% 12 Dynamics of highly performing sites Green curve: mean # of sites recruited > 8 pts red curves- 90% predictive bounds At the end of enrolment: mean = 8, low bound = 4, upper bound = 13 Predicted number of sites recruited more than 8 pts each. The same case study: number of patients =450; number of clinical sites =175; time to complete enrolment =9 months; all centres initiated at start up. 13 Modelling trial operation Next stage: Predictive modelling operational processes. A new technique using hierarchic evolving processes is developed. Model: In centre i recruited at time tki patient k generates some follow-up evolving process with random life time ξki(t- tki), tki ≤ t ≤ tki +τki, e.g.: • follow-up visits, related events (clinical or operational, e.g. - AE, CRF, monitoring visits,...) • associated costs (visits, maintenance, supply) Zi(t) = Σk ξki(t – tki) - sum of evolving processes in centre i, Statistical technique for forecasting evolving processes is developed. Closed-form solutions are derived for many practical scenarios. Tools in C#, R and RExcel are created * * Predictive Analytics Team, Quintiles 14 Operation cost modelling Monthly cost of trial operation Input: Costs per patient visits Visit No 1 2 3 4 5 6 7 8 9 10 Mean and 90% credibility intervals. Input – tables: • Site’s initiation data • # of sites • Costs per sites/patient visits Time # of visits 1 600 30 450 58 400 87 350 114 304 143 297 172 290 198 283 282 276 366 269 Cost 100 50 40 40 50 40 30 40 45 40 Input: Operational costs: • Opening site = $10,000 • Maintenance cost per site/day = $100 • Site closing cost = $1,000 • Monitoring cost per patient/day = $20 15 Patients in trial/visits Study design: Sites – 200, patient’s target – 800, enrolment duration – 1 year Predictive number of follow-up patients Predictive number of Visits No. 3 Mean, Low and Upper 90% bounds Visits time-schedule: Region with 100 sites, μ – drop-out rate, 4 visits in total, each after 60 days Follow-up period L=180 days. At some conditions – stable regime: for L < t <T, n(t) ~ nstat – Poisson with rate λ(1 – e- μL) /μ 16 Event Forecaster tool Event-driven trials. Patients are followed-up. Analytic technique* is developed for predicting over time the number of events together with ongoing enrolment and time to stop trial Assumptions: • • • Enrolment follows a Poisson-gamma model Can be different types of events For multiple events Markov evolving processes are used. Predictive process: and - transition probabilities of Markov chain, - global posterior rate (using Bayesian re-estimation in sites). Predictive characteristics are derived in a closed form (no need in Monte Carlo simulation) Opportunity: to combine prediction of the # of events with predictive power. * Anisimov, Pharma Stats, 2011 17 Probability of operational success A Scenario. Enrolment completion target time 21 months, Trial completion target time 70 months Other 150 sites (with 0.4 pts/month/site) are added after interim look PA = 100% (enrolment complete in time) PB = 95% (events complete in time) Hazard ratio – 0.8, test on non-inferiority. Power depends on the # of events - random PC = 97.5% - Prob( Power >0.9) B Trial completion target time C Trial completion target time Target number of events Acknowledgment: Val Fedorov, Xiaoqiang Xue 18 Conclusions The advanced statistical techniques for predictive analytic modelling of trial operation are developed: • • Predict and adaptively adjust patient enrolment Forecast trial operational characteristics trial/site performance, visits, follow-up patients, operational costs... • • Provide risk-based monitoring of the main characteristics Forecast of different event’s We are not aware about similar tools used by other pharma companies and CRO. Supporting software tools are created and in use by Predictive Analytics Team, Innovation, Quintiles. Tools open wide opportunities for improving the efficiency and quality of CT prediction, additional benefits and cost savings. 19 References 1. Anisimov V., Fedorov V., Modelling, prediction and adaptive adjustment of recruitment in multicentre trials, Statistics in Medicine, 26, 2007, 4958– 4975. 2. Anisimov V., Predictive modelling of recruitment and drug supply in multicentre clinical trials, Proc. of JSM, Washington, August, 2009, 12481259. 3. Anisimov V., Predictive event modelling in multicentre clinical trials with waiting time to response, Pharmaceutical Statistics, 10, iss. 6, 2011, 517522. 4. Anisimov V., Statistical modelling of clinical trials (recruitment and randomization), Communications in Statistics – Theory and Methods, 40: 19-20, 2011, 3684-3699. 5. Anisimov V., Predictive hierarchic modelling of operational characteristics in clinical trials, Communications in Statistics - Simulation and Computation (submitted) 20