Report

Applied Bayesian Methods Phil Woodward Phil Woodward 2014 1 Introduction to Bayesian Statistics Phil Woodward 2014 2 Inferences via Sampling Theory • Inferences made via sampling distribution of statistics – – – – A model with unknown parameters is assumed Statistics (functions of the data) are defined These statistics are in some way informative about the parameters For example, they may be unbiased, minimum variance estimators • Probability is the frequency with which recurring events occur – – – – The recurring event is the statistic for fixed parameter values The probabilities arise by considering data other than actually seen Need to decide on most appropriate “reference set” Confidence and p-values are p(data “or more extreme”| θ) calculations • Difficulties when making inferences – Nuisance parameters an issue when no suitable sufficient statistics – Constraints in the parameter space cause difficulties – Confidence intervals and p-values are routinely misinterpreted • They are not p(θ | data) calculations Phil Woodward 2014 3 How does Bayes add value? • Informative Prior – – – – Natural approach for incorporating information already available Smaller, cheaper, quicker and more ethical studies More precise estimates and more reliable decisions Sometimes weakly informative priors can overcome model fitting failure • Probability as a “degree of belief” – Quantifies our uncertainty in any unknown quantity or event – Answers questions of direct scientific interest • P(state of world | data) rather than P(data* | state of world) • Model building and making inferences – – – – – Nuisance parameters no longer a “nuisance” Random effects, non-linear terms, complex models all handled better Functions of parameters estimated with ease Predictions and decision analysis follow naturally Transparency in assumptions • Beauty in its simplicity! – p(θ | x) = p(x | θ) p(θ) / p(x) – Avoids issue of identifying “best” estimators and their sampling properties – More time spent addressing issues of direct scientific relevance Phil Woodward 2014 4 Probability • Most Bayesians treat probability as a measure of belief – Some believe probabilities can be objective (not discussed here) – Probability not restricted to recurring events • E.g. probability it will rain tomorrow is a Bayesian probability – Probabilities lie between 0 (impossible event) and 1 (certain event) – Probabilities between 0 and 1 can be calibrated via the “fair bet” • What is a “fair bet”? – Bookmaker sells a bet by stating the odds for or against an event – Odds are set to encourage a punter to buy the bet • E.g. odds of 2-to-1 against means that for each unit staked two are won, plus the stake – A fair bet is when one is indifferent to being bookmaker or punter • i.e. one doesn’t believe either side has an unfair advantage in the gamble Phil Woodward 2014 5 Probability • Relationship between odds and probability – One-to-one mapping between odds (O) and probability (P) Where O equals the ratio X/Y for odds of X-to-Y in favour and the ratio Y/X for odds of X-to-Y against an event e.g. odds of 2-to-1 against, if fair, imply probability equals ⅓ • Probabilities defined this way are inevitably subjective – – – – – – People with different knowledge may have different probabilities Controversy occurs when using this definition to interpret data Science should be “objective”, so “subjectivity” to some is heresy But where do the models that Frequentists use come from? Are the decisions made when designing studies purely objective? Is judgment needed when generalising from a sample to a population? Phil Woodward 2014 6 Probability • Subjectivity does not mean biased, prejudiced or unscientific – Large body of research into elicitation of personal probabilities – Where frequency interpretation applies, these should support beliefs • E.g. the probability of the next roll of a die coming up a six should be ⅙ for everyone unless you have good reason to doubt the die is fair – An advantage of the Bayesian definition is that it allows all other information to be taken into account • E.g. you may suspect the person offering a bet on the die roll is of dubious character • Bayesians are better equipped to win at poker than Frequentists! • All unknown quantities, including parameters, are considered random variables Epistemic uncertainty – each parameter still has only one true value – our uncertainty in this value is represented by a probability distribution Phil Woodward 2014 7 Exchangeability • Exchangeability is an important Bayesian concept – exchangeable quantities cannot be partitioned into more similar sub-groups – nor can they be ordered in a way that infers we can distinguish between them – exchangeability often used to justify prior distribution for parameters analogous to classical random effects Phil Woodward 2014 8 The Bayesian Paradigm From P r(A | B ) and P r(B | A) P r(A, B) P r(B) A P r(A, B) P r(A) B comes Bayes Theorem P r(A | B) P r(A) P r(B | A) P r(B ) Nothing controversial yet. Phil Woodward 2014 9 The Bayesian Paradigm How is Bayes Theorem (mis)used? Coin tossing study: Is the coin fair? Model ri ~ bern(π) i = 1, 2, ..., n ri = 1 if ith toss a head, = 0 if a tail Let terms in Bayes Theorem be A = π (controversial) B=r then p ( ) p (r | ) p ( | r ) Why? p(r ) Phil Woodward 2014 10 The Bayesian Paradigm What are these terms? p(r|π) is the likelihood = bin(n, Σr| π) (not controversial) p(π) is the prior = ??? (controversial) The prior formally represents our knowledge of π before observing r Phil Woodward 2014 11 The Bayesian Paradigm What are these terms (continued)? MCMC to the rescue! p(r) is the normalising constant = ∫ p(r|π) p(π) dπ (the difficult bit!) In general, not in this particular case p(π|r) is the posterior The posterior formally represents our knowledge of π after observing r Phil Woodward 2014 12 The Bayesian Paradigm A worked example. Coin tossed 5 times giving 4 heads and 1 tail What if data were p(r|π) = bin(n=5, Σr=4| π) 5 dogs in tox study: 4 OK, 1 with an AE? p(π) = beta(a, b), when a=b=1 ≡ U(0, 1) ...but is a stronger Why choose a beta distribution?! prior justifiable? - conjugacy … posterior p(π|r) = beta(a+Σr, b+n-Σr) - can represent vague belief? - can be an objective reference? - Beta family is flexible (could be informative) Phil Woodward 2014 13 The Bayesian Paradigm A worked example (continued). Applying Bayes theorem p(π|r) = beta(5, 2) 95% credible interval π : (0.36 to 0.96) Pr[π ϵ (0.36 to 0.96) | Σr = 4] = 0.95 95% confidence interval π : (0.28 to 0.995) Pr[Σr ≥ 4 | π = 0.28] = 0.025, Pr[Σr ≤ 4 | π = 0.995] = 0.025 Phil Woodward 2014 14 The Bayesian Paradigm Bayesian inference for simple Normal model Clinical study: What’s the mean response to placebo? Model yi ~ N(µ, σ2) i = 1, 2, ..., n (placebo subjects only) assume σ known and for convenience will use precision parameter τ = σ-2 (reciprocal of variance) Terms in Bayes Theorem are p( ) p(y | ) p( | y ) p(y ) Phil Woodward 2014 15 The Bayesian Paradigm Improper prior density Phil Woodward 2014 16 The Bayesian Paradigm Posterior precision equals sum of prior and data precisions Posterior mean equals weighted mean of prior and data Phil Woodward 2014 17 The Bayesian Paradigm Phil Woodward 2014 18 The Bayesian Paradigm A worked example (continued). Applying Bayes theorem p(µ |y) = N(80, 0.5) 95% credible interval µ : (78.6 to 81.4) 95% confidence interval µ : (78.6 to 81.4) Phil Woodward 2014 19 The Bayesian Paradigm Bayesian inference for simple Normal model The case when both mean and variance are unknown Model yi ~ N(µ, σ2) i = 1, 2, ..., n Terms in Bayes Theorem are p( , ) p(y | , ) p( , | y ) p(y ) Phil Woodward 2014 20 The Bayesian Paradigm Phil Woodward 2014 21 The Bayesian Paradigm Phil Woodward 2014 22 The Bayesian Paradigm Bayesian inference for Normal Linear Model Model y = Xθ + ε εi ~ N(0, σ2) i = 1, 2, ..., n y and ε are n x 1 vectors of observations and errors X is a n x k matrix of known constants θ is a k x 1 vector of unknown regression coefficients Terms in Bayes Theorem are p(θ, ) p(y | θ, ) p(θ, | y ) p(y ) Phil Woodward 2014 23 The Bayesian Paradigm Phil Woodward 2014 24 The Bayesian Paradigm In summary, for Normal Linear Model (“fixed effects”) Classical confidence intervals can be interpreted as Bayesian credible intervals But, need to be aware of implicit prior distributions Not generally the case for other error distributions But for “large samples” when likelihood based estimator has approximate Normal distribution, a Bayesian interpretation can again be made “Random effects” models are not so easily compared Don’t assume classical results have Bayesian interpretation Phil Woodward 2014 25 The Bayesian Paradigm Conditional (on µ) distribution for future response Phil Woodward 2014 Posterior distribution for µ 26 The Bayesian Paradigm N(µ, σ2) N(µ1, 1/τ1) yf ~ N(µ1, 1/τ1 + 1/τ) Sum of posterior variance of µ and conditional variance of yf Phil Woodward 2014 27 The Bayesian Paradigm Predictive Distributions When are predictive distributions useful? When designing studies “design priors” must be informative we predict the data using priors to assess the design we may use informative priors to reduce study size, these being predictions from historical studies When undertaking interim analyses we can predict the remaining data using current posterior When checking adequacy of our assumed model model checking involves comparing observations with predictions When making decisions after study has completed we can predict future trial data to assess probability of success, helping to determine best strategy or decide to stop Some argue predictive inferences should be our main focus be interested in observable rather than unobservable quantities e.g. how many patients will do better on this drug? Phil Woodward 2014 28 The Bayesian Paradigm δ is treatment effect Phil Woodward 2014 29 The Bayesian Paradigm Phil Woodward 2014 30 The Bayesian Paradigm Phil Woodward 2014 31 The Bayesian Paradigm Making Decisions A simple Bayesian approach defines criteria of the form Pr(δ ≥ Δ) > π where Δ is an effect size of interest, and π is the probability required to make a positive decision For example, Bayesian analogy to significance could be Pr(δ > 0) > 0.95 But is believing δ > 0 enough for further investment? Phil Woodward 2014 32 END OF PART 1 intro to WinBUGS illustrating fixed effect models Phil Woodward 2014 33 Bayesian Model Checking Phil Woodward 2014 34 Bayesian Model Checking Brief outline of some methods easy to use with MCMC Consider three model checking objectives 1. Examination of individual observations 2. Global tests of goodness-of-fit 3. Comparison between competing models In all cases we compare observed statistics with expectations, i.e. predictions conditional on a model Phil Woodward 2014 35 Bayesian Model Checking yi is the observation Yi is the prediction E(Yi) is the mean of the predictive distribution Bayesian residuals can be examined as we do classical residuals p-value concept Phil Woodward 2014 36 Bayesian Model Checking Ideally we would have a separate evaluation dataset Predictive distribution for Yi is then independent of yi Typically not available for clinical studies Cross-validation next best, but difficult within WinBUGS Following methods use the data twice, so will be conservative, i.e. overstate how good model fits data Will illustrate using WinBUGS code for simplest NLM Phil Woodward 2014 37 Bayesian Model Checking { (Examination of Individual Observations) More typically, each Y[i] ### Priors has different mean, mu[i]. mu ~ dnorm(0, 1.0E-6) prec ~ dgamma(0.001, 0.001) ; sigma <- pow(prec, -0.5) ### Likelihood for (i in 1:N) { Y[i] ~ dnorm(mu, prec) } each residual has a distribution use the mean as the residual Y.rep[i] is a prediction accounting for uncertainty in parameter values, but not in the type of model assumed ### Model checking for (i in 1:N) { ### Residuals and Standardised Residuals resid[i] <- Y[i] – mu st.resid[i] <- resid[i] / sigma } } mean of Pr.big[i] estimates the probability a future observation is this big ### Replicate data set & Prob observation is extreme only need both Y.rep[i] ~ dnorm(mu, prec) Pr.big[i] <- step( Y[i] – Y.rep[i] ) when Y.rep[i] could Pr.small[i] <- step( Y.rep[i] – Y[i] ) exactly equal Y[i] Phil Woodward 2014 38 Bayesian Model Checking (Global tests of goodness-of-fit) Identify a discrepancy measure e.g. a measure of skewness for testing this aspect of Normal assumption typically a function of the data but could be function of both data and parameters Predict (replicate) values of this measure conditional on the type of model assumed but accounting for uncertainty in parameter values Compute “Bayesian p-value” for observed discrepancy similar approach used for individual observations convention for global tests is to quote “p-value” Phil Woodward 2014 39 Bayesian Model Checking { (Global tests of goodness-of-fit) … code as before … ### Model checking for (i in 1:N) { ### Residuals and Standardised Residuals resid[i] <- Y[i] – mu st.resid[i] <- resid[i] / sigma m3[i] <- pow( st.resid[i], 3) ### Replicate data set Y.rep[i] ~ dnorm(mu, prec) resid.rep[i] <- Y.rep[i] – mu[i] st.resid.rep[i] <- resid.rep[i] / sigma m3.rep[i] <- pow( st.resid.rep[i], 3) } skew <- mean( m3[] ) skew.rep <- mean( m3.rep[] ) p.skew.pos <- step( skew.rep – skew ) p.skew.neg <- step( skew – skew.rep ) p.skew interpreted as for classical p-value, i.e. small is evidence of a discrepancy } Phil Woodward 2014 40 Bayesian Model Checking (Comparison between competing models) Bayes factors not easy to implement using MCMC will not be discussed further ratio of marginal likelihoods under competing models Bayesian analogy to classical likelihood ratio test Phil Woodward 2014 41 Bayesian Model Checking (Comparison between competing models) Deviance Information Criterion (DIC) a Bayesian “information criterion” but not the BIC will not discuss theory, focus on practical interpretation WinBUGS & SAS can report this for most models DIC is the sum of two separately interpretable quantities DIC = Dbar + pD Dbar pD Dhat : the posterior mean of the deviance : the effective number of parameters in the model pD = Dbar - Dhat : deviance point estimate using posterior mean of θ Phil Woodward 2014 42 Bayesian Model Checking (Comparison between competing models) Deviance Information Criterion (DIC) DIC = Dbar + pD pD will differ from the total number of parameters when posterior distributions are correlated typically the case for “random effect parameters” non-orthogonal designs, correlated covariates common for non-linear models pD will be smaller because some parameters’ effects “overlap” Phil Woodward 2014 43 Bayesian Model Checking (Comparison between competing models) Deviance Information Criterion (DIC) DIC = Dbar + pD Measures model’s ability to make short-term predictions Smaller values of DIC indicate a better model Rules of thumb for comparing models fitted to the same data DIC difference > 10 is clear evidence of being better DIC difference > 5 (< 10) is still strong evidence There are still some unresolved issues with DIC relatively early days in it use, so use other methods as well Phil Woodward 2014 44 Bayesian Model Checking (practical advice) “All models are wrong, but some are useful” if we keep looking, or have lots of data, we will find lack-of-fit need to assess whether model’s deficiencies matter depends upon the inferences and decisions of interest judge the model on whether it is fit for purpose Sensitivity analyses are useful when uncertain should assess sensitivity to both the likelihood and the prior e.g. replace Normal with t distribution Model expansion may be necessary Bayesian approach particularly good here Informative priors and MCMC allow greater flexibility Phil Woodward 2014 45 Introduction to BugsXLA Parallel Group Clinical Study (Analysis of Covariance) Phil Woodward 2014 46 BugsXLA (case study 3.1) Switch to Excel and demonstrate how BugsXLA facilitates rapid Bayesian model specification and analysis via WinBUGS. Phil Woodward 2014 47 BugsXLA (case study 3.1) Phil Woodward 2014 48 BugsXLA (case study 3.1) Settings used by WinBUGS Posterior distributions to be summarised Posterior samples to be imported Save WinBUGS files, create R scripts Suggested settings Phil Woodward 2014 49 BugsXLA (case study 3.1) Fixed factor effects parameterised as contrasts from a zero constrained level Priors for other parameter types Default priors chosen to be “vague” (no guarantees!) Bayesian model checking options Phil Woodward 2014 50 BugsXLA (case study 3.1) BugsXLA uses generic names for parameters in WinBUGS code (deciphered on input!) Recommend adding MC Error to input list The Excel sheet used to display the results Phil Woodward 2014 51 BugsXLA Generic names … deciphered (case study 3.1) Posterior means, st.devs. & credible int.s Could compute ratio MC Error / St.Dev. using cell formula Reminder of model, prior and WinBUGS settings Phil Woodward 2014 52 BugsXLA (case study 3.1) Phil Woodward 2014 53 BugsXLA (case study 3.1) BugsXLA interprets contents of cells to define predictions & contrasts to be estimated In this case, predicted means for each level of factor TRT are defined Phil Woodward 2014 54 BugsXLA (case study 3.1) Phil Woodward 2014 55 Other default settings can be personalised, e.g. default priors Recommend turn this off once understand how parameterised Can set own alerts to be used with model checking functions BugsXLA (Default Settings) Phil Woodward 2014 56 Fixed factor effects parameterisation can be changed to SAS (last level) BugsXLA (Default Settings) Phil Woodward 2014 57 Obtaining Prior Distribution Phil Woodward 2014 58 Obtaining Prior Distributions • Brief overview of main approaches* • Further issues in the use of Priors* * based on chapter 5 of Spiegelhalter et al (2004) Phil Woodward 2014 59 Obtaining Prior Distributions • Misconceptions: They are not necessarily – – – – Prespecified Unique Known Influential • Bayesian analysis – – – – But prespecification strongly recommended, data must not influence the prior distribution Transforms prior into posterior beliefs Doesn’t produce the posterior distribution Context and audience important Sensitivity to alternative assumptions vital • Prior could differ at design & analysis stage – May want less controversial vague priors in analysis – Design priors usually have to be informative Phil Woodward 2014 60 Obtaining Prior Distributions • Five broad approaches – Elicitation of subjective opinion – Summarising past evidence – Default priors – Robust priors – Estimation using hierarchical models Phil Woodward 2014 61 Obtaining Prior Distributions • Elicitation of subjective opinion – – – – Most useful when little ‘objective’ evidence Less controversial at the design stage Elicitation should be kept simple & interactive O’Hagan is a strong advocate • Spiegelhalter et al do not recommend – Prefer archetypal views; see Default Priors Phil Woodward 2014 62 Summarising Past Evidence. Typically (b) adequate, maybe with more complexity. Meta-analytic-predictive Typically, yh ~ N(θh, σh2) Exchangeable θ,θh ~ N(μ, τ2) (a) τ = ∞, μ = K (b) τ ~ dist. (f) τ = 0, μ = θ (c) θh = θ + δh δh ~ N(0, σδh2) θh ~ N(θ, σδh2) Phil Woodward 2014 63 Obtaining Prior Distributions • Default Priors Parameter “big” can be derived via eliciting inferred quantities, e.g. credible differences between study means. – Vague a.k.a. non-informative or reference • WinBUGS (general advice for ‘simple’ models): – Location parms ~ Normal with huge variance – Lowest level error variance ~ inv-gamma(small, small) – Hierarchical error variances … controverisal sd ~ Uniform(0, big) or ~ Half-Normal(big); big < huge! – Sceptical & Enthusiastic Priors • • • • Sceptical used to determine when success achieved Enthusiastic used to determine when to stop Sceptical prior centred on 0 with small prob. effect > Δ Enthusiastic prior centred on Δ with small prob. effect < 0 – ‘Lump-and-smear’ Priors • Point mass at the null hypothesis Phil Woodward 2014 Might be appropriate for unprecedented mechanisms in ED stage. 64 Obtaining Prior Distributions • Robust Priors – We always assess model assumptions – Bayesians assess prior assumptions also – Use a ‘community of priors’ • Discrete set • Parametric family • Non-parametric family Perhaps develop a range of priors appropriate in typical case. – Interpretation section recommended in report • Show how data affect a range of prior beliefs Phil Woodward 2014 65 Example of a parametric family of priors α is the discounting factor discussed previously (variant d: “equal but discounted”) Not Recommended as no operational Interpretation & no means of assessing suitable values for alpha Phil Woodward 2014 66 Obtaining Prior Distributions • Hierarchical priors – In simplest case, the same as (b) Exchangeable – ‘Borrow strength’ between studies • counter view: ‘share weakness’ – Three essential ingredients • Exchangeable parameters • Form for random-effects dist. – Typically Normal, although t is perhaps more realistic • Hyperprior for parms of random-effects dist. – sd ~ Uniform(0, Max Credible) or Half-Normal(big) or Half-Cauchy(large) Phil Woodward 2014 67 Obtaining Prior Distributions • Case Study – Dental Pain Studies – Informative prior for placebo mean • Used in the formal analysis – Meta-analytic-predictive approach Phil Woodward 2014 68 Obtaining Prior Distributions (part of table of prior studies considered relevant) TOTPAR[6] Title Characterization of rofecoxib as a cyclooxygenase-2 isoform inhibitor and demonstration of analgesia in the dental pain model Valdecoxib Is More Efficacious Than Rofecoxib in Relieving Pain Associated With Oral Surgery Rofecoxib versus codeine/acetaminophen in postoperative dental pain: a double-blind, randomized, placebo- and active comparator-controlled clinical trial Analgesic Efficacy of Celecoxib in Postoperative Oral Surgery Pain: A Single-Dose, Two-Center, Randomized, Double-Blind, Active- and PlaceboControlled Study Combination Oxycodone 5 mg/Ibuprofen 400 mg for the Treatment of Postoperative Pain: A Double-Blind, Placebo and Active-Controlled ParallelGroup Study Mean Authors Treatment 3.01 0.51 Elliot W. Ehrich et. al Rofecoxib 50 and 500 mg Ibuprofen 400 mg Placebo 3.01 0.76 Fricke J. et al. Valdecoxib 40 mg Rofecoxib 50 mg Placebo 3.4 1.22 Chang DJ; et al. Rofecoxib 50 mg Codeine/Acetaminophen 60/600 mg Placebo 3.7 0.75 Raymond Cheung, et al Celecoxib 400 mg Ibuprofen 400 mg Placebo 4.2 0.83 Thomas Van Dyke, et al Oxycodone/Ibuprofen 5 mg/400 mg Ibuprofen 400 mg Oxycodone 5 mg Placebo Phil Woodward 2014 (Placebo Data) SE 69 Obtaining Prior Distributions • Meta-analysis of historical data – Published summary data • Normal Linear Mixed Model Yi = θi + ei θi ~ N(µθ, ω2) ei ~ N(0, SEi2) Yi are the observed placebo means from each study SEi are their associated standard errors Phil Woodward 2014 70 Obtaining Prior Distributions • WinBUGS used to determine prior • assumes study means exchangeable • but not responses from different studies { ‘newtrial’ provides prior for a future study. for (i in 1:N) { Y[i] ~ dnorm(theta[i], prec[i]) theta[i] ~ dnorm(mu.theta, tau.theta) prec[i] <- pow(se[i], -2) } newtrial ~ dnorm(mu.theta, tau.theta) mu.theta ~ dnorm(0, 1.0E-6) tau.theta <- pow(omega, -2) omega ~ dunif(0, 100) } If studies smaller, model should account for fact that each se is estimated. If few studies, might need slightly more informative prior. Phil Woodward 2014 71 Obtaining Prior Distributions Gamma Distribution (or Inv-Gamma Dist.) Particularly useful in Bayesian statistics Conjugate for Poisson mean Marginal distribution for σ-2 in NLMs NOTE: more than one parameterisation of Gamma Dist. Chi-Sqr is a special case of the Gamma ChiSqr(v) ≡ Gamma(v/2, 0.5) If s2 (v d.f.) is ML estimate of Normal variance (and conventional vague prior: p(σ2) α σ-2) Posterior p(σ2 | s2) = v s2 Inv-ChiSqr(v) = Inv-Gamma(v/2, v s2/2) Phil Woodward 2014 72 Obtaining Prior Distributions • Empirical criticism of priors e.g. observed placebo mean response – George Box suggested a Bayesian p-value • • • • Prior predictive distribution for future observation Compare actual observation with predictive dist. Calculate prob. of observing more extreme see model checking Measure of conflict between prior and data section – But what should you do if conflict occurs? • At least report this fact • Greater emphasis on analysis with a vaguer prior – Robust prior approach or heavy tailed e.g. t4 distribution • Formally model doubt using a mixture prior Phil Woodward 2014 73 Obtaining Prior Distributions • Key Points – – – – – – – Subjectivity cannot be completely avoided Range of priors should be considered Elicited priors tend to be overly enthusiastic Historical data is best basis for priors Archetypal priors provide a range of beliefs Default priors are not always ‘weak’ Exchangeability is a strong assumption • but with hierarchical model plus covariates, best option? – Sensitivity analysis is very important Phil Woodward 2014 74 BugsXLA Deriving and using informative prior distributions Phil Woodward 2014 75 BugsXLA (case study 5.3, details not covered in this course) Predict placebo mean response in a future study BugsXLA can model study level summary statistics (d.f. optional) Typically, model is much simpler than this, e.g. placebo data only, no study level covariates, so only random STUDY factor in model Phil Woodward 2014 76 BugsXLA (using informative prior distributions) Back to Case Study 3.1 Will assume have derived informative priors for: Placebo mean response Normal with mean 0 and standard deviation 0.04 Residual variance Scaled Chi-Square with s2 = 0.026 and df = 44 Switch back to Excel and show how to use this in BugsXLA Phil Woodward 2014 77 BugsXLA (case study 3.1, informative prior) Import samples so prior and posterior can be compared. Ignore this, unless you have R loaded and wish to explore in own time. Phil Woodward 2014 78 Informative priors for placebo mean and residual variance Phil Woodward 2014 79 BugsXLA (case study 3.1, informative prior) Click ‘sigma’ then ‘Post Plots’ icon Update Graph Can edit histogram (‘user specified’) Repeat for ‘Beta0’ (placebo) & ‘X.Eff[1,3]’ (TRT C) Phil Woodward 2014 80 BugsXLA (case study 3.1, informative prior) Phil Woodward 2014 81 BugsXLA (case study 3.1, informative prior) CAUTION Although prior for TRT:C is flat, posterior is influenced by other priors Can obtain other posterior summaries Phil Woodward 2014 82 Bayesian Study Design Phil Woodward 2014 83 Bayesian Study Design Consider a generic decision criterion of the form GO decision if Pr(δ ≥ Δ) > π δ is the treatment effect Δ is an effect size of interest π is the probability required to make a positive decision As previously discussed, a Bayesian analogy to significance could be Pr(δ > 0) > 0.95 Phil Woodward 2014 84 Bayesian Study Design Phil Woodward 2014 85 Bayesian Study Design Number of subjects Phil Woodward 2014 86 Bayesian Study Design Operating Characteristics (OC) Simple to calculate in any statistical software e.g. for 2 group PG or AB/BA XO design R code non-central t cdf 1 - pt( qt(pi, df= df), df= df, ncp= (delta – DELTA)/(sigma*sqrt(2/N)) ) normal cdf 1 - pnorm( qnorm(pi), mean= (delta – DELTA)/(sigma*sqrt(2/N)) ) Phil Woodward 2014 87 Bayesian Study Design Operating Characteristics (OC) If wanted to account for uncertainty in σ Determine Bayesian distribution for σ e.g. σ ~ U(12, 18) Use simulation to calculate the OC 1. Simulate σ value from its distribution 2. For each value of δ and this σ value compute Pr(GO) 3. Repeat 1 & 2 10,000 times, say, and mean for each δ value Warning This unconditional Pr(GO) averages high and low probabilities Is under powered more concerning than over powered? Phil Woodward 2014 88 Bayesian Study Design Phil Woodward 2014 89 Bayesian Study Design (Bayesian NLM inference reminder) Prior: p(δ) = N(δ0, ω2) vague if ω ≈ ∞ Known σ Likelihood (sufficient statistic): p(d | δ) = N(δ, Vd) e.g. 2 arm PG or AB/BA XO, Vd = 2σ2/N Posterior: p(δ | d) = N(Mδ, Vδ) Mδ = Vδ(δ0/ω2 + d/Vd) weighted average 1/Vδ = 1/ω2 + 1/Vd precisions are additive Vague prior implies p(δ|d) = N(d, Vd) = “confidence dist.” Phil Woodward 2014 90 Bayesian Study Design (Bayesian NLM predictions reminder) Posterior Distribution: As yet unobserved p(δ | d) = N(Mδ, Vδ) Conditional Distribution for d* from future study: p(d* | δ) = N(δ, Vd*) assuming studies are “exchangeable” Vd* determined by future study design Predictive Distribution for d*: p(d* | d) = ∫ p(d* | δ) p(δ | d) dδ = N(M*, V*) M* = Mδ V* = Vδ + Vd* sum of posterior and conditional variances Phil Woodward 2014 91 Bayesian Study Design (Prior Predictive Distribution) Can make predictions based on “prior beliefs” Prior Distribution: Design Prior p(δ) = N(δ0, ω2) Conditional Distribution for d from planned study: p(d | δ) = N(δ, Vd) Unobserved at design stage Predictive Distribution for d: p(d) = ∫ p(d | δ) p(δ) dδ = N(M0, V0) M0 = δ0 V0 = ω2 + Vd sum of prior and conditional variances Phil Woodward 2014 92 Bayesian Study Design (Assurance) expected/marginal power or predictive probability Classical power: Let C denote the event “reject null hypothesis” Power = Pr[C | δ] i.e. a conditional probability More generally, C can be any decision criteria Refer to the earlier OC calculations Bayesian predictive probability: Pr[C] = ∫ Pr[C | δ] p(δ) dδ The “unconditional” probability of C occurring (although it is conditional on our prior beliefs) “The probability, given our prior knowledge, that we will meet the decision criteria at the end of the study.” Phil Woodward 2014 93 Bayesian Study Design (Assurance) Consider original GO decision Pr[C | d] = Pr[d – tπ se(d) > Δ] p(d) = N(δ0, ω2 + Vd) with vague Analysis Prior prior predictive distribution with informative Design Prior Pr[C] = Pr[ N(δ0, ω2 + Vd) > tπ se(d) + Δ] Pr[C] = Φ[δ0 – tπ se(d) – Δ) / (ω2 + Vd)½] where Φ[.] is the standard Normal cdf What if vague Design Prior, i.e. ω very large? Phil Woodward 2014 94 Bayesian Study Design (Assurance) Plot comparing classical (‘conditional power’) OC and assurance ω δ0 Phil Woodward 2014 95 Bayesian Study Design (Assurance) For superiority, Δ = 0, and noting z = t for large d.f. Pr[C] = Φ[θ – zπ se(d)) / (ω2 + Vd)½] same as Eq.3 in O’Hagan et al (2005) For non-inferiority, Δ is negative same as Eq.6 in O’Hagan et al (2005) Phil Woodward 2014 96 Bayesian Study Design (Interim Analysis) Can make predictions after an interim analysis Let estimate at interim (n subjects) be d’ Let estimate from part 2 (m subjects) be d* p(d* | d’) = N(Mδ, Vδ + Vd*) unobserved at interim stage predictive distribution If vague prior at study start (Design Prior) Mδ = d’ Vδ = Vd’ For a 2 arm PG or AB/BA XO Vd* = 2σ2/m and Vd’ = 2σ2/n Phil Woodward 2014 97 Bayesian Study Design (Interim Analysis with vague Design & Analysis Priors) Consider original GO decision (with Vd = 2σ2/N) Pr[C | d] = Pr[d – tπ se(d) > Δ] This criterion, C, can be expressed in terms of d’ and d* (md* + nd’)/N – tπ σ(2/N)½ > Δ At the interim stage d* is the only unknown And so it is convenient to express C as but Bayesians can d* > (N½ tπ σ2½ + NΔ - nd’) / m do better than this! Classical “conditional power”, Pr[C | δ, d’] Pr[ N(δ, 2σ2/m) > (N½ tπ σ2½ + NΔ - nd’) / m ] 1 - Φ[ (N/m)½ tπ + (NΔ - nd’ - mδ) / (σ(2m)½) ] which, for Δ = 0 & tπ = zπ, is same as Eq.2 in Grieve (1991) Phil Woodward 2014 98 Bayesian Study Design (Interim Analysis with vague Design & Analysis Priors) Bayesian predictive probability, Pr[C | d’] Pr[ N(d’, 2σ2(n-1+m-1)) > (N½ tπ σ2½ + NΔ - nd’) / m ] 1 - Φ[ (n/m)½ {tπ - N½(d’ – Δ) / (σ2½)} ] which, for Δ = 0 & tπ = zπ, is same as Eq.3 in Grieve (1991) “The probability, given our knowledge at the interim, that we will meet the decision criteria at the end of the study.” Interim futility / success criteria could be based on this probability e.g. futile if Pr[C | d’] < 0.2 success if Pr[C | d’] > 0.8 Phil Woodward 2014 99 Bayesian Study Design (Interim analysis predictive probability) Plot comparing ‘conditional power’ and predictive probability following interim analysis (25/grp), vague prior distribution Only differences to analysis done prior to study start are: 1)OC curve conditional on both delta and interim data 2)‘Belief distribution’ for delta updated using interim data Prior or Posterior depends on one’s perspective (‘Belief Distribution’) could use informative design prior, updated using interim data … Vd’ d' Phil Woodward 2014 100 Bayesian Study Design (Interim Analysis with informative Design Prior) If we allow an informative design prior at study start refer back to p(δ) = N(δ0, ω2) Bayesian NLM reminders p(d* | d’) = N(Mδ, Vδ + 2σ2/m) Mδ = Vδ(δ0/ω2 + d’n/(2σ2)) 1/Vδ = 1/ω2 + n/(2σ2) Bayesian predictive probability, Pr[C | d’] Pr[ N(Mδ, Vδ + 2σ2/m) > (N½ tπ σ2½ + NΔ - nd’) / m ] 1 - Φ[(N½ tπ σ2½ - nd’ - mMδ + NΔ) / (m(Vδ + 2σ2/m)½) still with vague analysis prior Phil Woodward 2014 101 Bayesian Study Design (using informative prior to reduce sample size) Typically, only have informative prior for placebo response Notation (in addition to that used previously) γ is the placebo true mean response p(γ) = N(γ0, ψ2) , the informative prior for γ nA, nP are the number of subjects receiving active and placebo The effective number of subjects this prior contributes is … but is our nγ = σ2 / ψ2 intuition correct? which may be intuitive by re-expressing as ψ2 = σ2 / nγ Phil Woodward 2014 102 Bayesian Study Design (using informative prior to reduce sample size) If prior for treatment effect, δ, is vague then posterior with … left as an exercise to prove It can be seen that the informative prior is equivalent to nγ additional placebo subjects with a sample mean of γ0 Phil Woodward 2014 103 Bayesian Study Design (using informative prior to reduce sample size) Worked example Suppose predictive distribution (placebo prior) p(γ) ~ N(18, 122) Forecast residual standard deviation (obtained in usual way, not shown here) Design study in usual way, σ = 70 ignoring informative prior. Effective N of placebo prior Eff.N = (70 / 12)2 = 34 Then reduce placebo arm by 34 and have same power / precision. Phil Woodward 2014 104 Bayesian Study Design (using informative prior to reduce sample size) Unless no doubts at all, use Robust Prior i.e. a mixture of informative and vague prior distributions p(placebo mean) ~ 0.9 x N(18, 122) + 0.1 x N(18, 1202) Represents 10% chance meta-data not exchangeable in which case, will effectively revert to vague prior (can also be thought of as heavy tailed distribution) Also compute Bayesian p-value of data-prior compatibility Pr( “> observed mean” | prior ~ N(18, 122) ) Note: predictive dist. for obs. mean ~ N(18, 122 + σ2 /nP) Phil Woodward 2014 105 Bayesian Emax Model dose/concentration response model Phil Woodward 2014 106 Bayesian Emax Model Emax model is often used for dose response data even more common for concentration response data in biological (non-clinical) context known as the logistic or sigmoidal curve More generally, could be used to model a monotonic relationship between response and covariate initially the response changes very slowly with the covariate then the response changes much more rapidly finally the response slows again as a plateau is reached Phil Woodward 2014 107 Bayesian Emax Model λ sometimes referred to as ‘Hill slope’ when λ = 1 need ~80 fold range to cover ED10 to ED90 approximately linear on log-scale between ED20 and ED80 Phil Woodward 2014 108 Bayesian Emax Model Convergence issues are common with MLE of Emax models Hill coefficient not restrained Fitted Curve no data on upper asymptote 7 7 6 6 5 5 Response Response Fitted Curve Hill coefficient = 1 4 4 3 3 2 2 1 1 1 10 100 1000 Concentration (ug/m L) 1 10 100 1000 Concentration (ug/mL) Most clinical data more variable than this and smaller dose range Classical fitting algorithms can fail to provide any solution Phil Woodward 2014 109 Bayesian Emax Model Prior distributions required for all parameters E0 : placebo (negative control) response utilise historical data as discussed earlier Emax : maximum possible effect relative to E0 typically vague, similar approach to treatment effect prior ED50: dose that gives 50% of Emax effect e.g. log-normal centred on mid/low dose 90% CI (0.1, 10)GM could be weakly informative based on same information used to choose dose range λ determines gradient of dose response e.g. log-normal centred on 1 90% CI (0.5, 2) typically needs to be very informative clinical data rarely provides much information regards λ Phil Woodward 2014 110 Bayesian Emax Model (WinBUGS code) { priors should be checked for appropriateness in each particular case ### Priors prec ~ dgamma( 0.001, 0.001 ) ; sigma <- pow(prec, -0.5) E0 ~ dnorm( 0, 1.0E-6 ) #... but could be informative for placebo mean response Emax ~ dnorm( 0, 1.0E-6 ) #... typically vague log.ED50 ~ dnorm( ???, 1.4 ) ED50 <- exp( log.ED50 ) #... Gives 90% CI of (0.1, 10) x exp(???) log.Hill ~ dnorm( 0, 0.42 ) Hill <- exp( log.Hill ) #... Gives 90% CI of (0.5, 2) ### Likelihood for (i in 1:N) { Y[i] ~ dnorm(mu[i], prec) mu[i] <- E0 + (Emax * pow( X[i], Hill) ) / ( pow( X[i], Hill ) + pow( ED50, Hill ) ) } } Bayesian Emax Model (WinBUGS code) { … code as before … ### Quantities of potential interest for (i in 1:N.doses) { ### effect over placebo for pre-specified doses (values entered as data in node DOSE) effect[i] <- (Emax * pow( DOSE[i], Hill) ) / ( pow( DOSE[i], Hill ) + pow( ED50, Hill ) ) ### predicted mean response for pre-specified doses DOSE.mean[i] <- effect[i] + E0 ### probability of exceeding pre-specified effect of size ??? (mean of node DOSE.PrEffBig) DOSE.PrEffBig[i] <- step( effect[i] - ??? ) } ### estimate dose giving effect of size ??? DOSE.BigEff.0 <- ED50 * ( pow( ???, 1/Hill ) ) / ( pow( Emax - ???, 1/Hill ) ) # set to LARGE dose (LARGE pre-specified) if ??? > Emax DOSE.BigEff <- DOSE.BigEff.0 * step( Emax - ???) + LARGE * step( ??? – Emax ) } BugsXLA Emax models Pharmacology Biomarker Experiment Phil Woodward 2014 113 BugsXLA (case study 7.1) Phil Woodward 2014 114 BugsXLA (case study 7.1) Fixed & random effects Emax models can be fitted using BugsXLA. Details not covered in this course. Phil Woodward 2014 115 References Bolstad, W.M. (2007). Introduction to Bayesian Statistics. 2nd Edition. John Wiley & Sons, New York. Gelman, A., Carlin, J.B., Stern, H.S. and Rubin, D.B. (2004). Bayesian Data Analysis. 2nd Edition. Chapman & Hall/CRC. (3rd Edition now available). Grieve, A. (1991). Predictive probability in clinical trials. Biometrics, 47, 323-330 Lee, P.M. (2004). Bayesian Statistics: An Introduction. 3rd Edition. Hodder Arnold, London, U.K. Neuenschwander, B., Capkun-Niggli, G., Branson, M. and Spiegelhalter, D.J. (2010). Summarizing historical information on controls in clinical trials. Clinical Trials; 7: 5-18 Ntzoufras, I. (2009). Bayesian Modeling Using WinBUGS. John Wiley & Sons, Hoboken, NJ. O’Hagan,A., Stevens,J. and Campbell,M. (2005). Assurance in clinical trial design. Pharmaceut. Statist. 4, 187201 Spiegelhalter, D., Abrams, K. and Myles,J. (2004). Bayesian Approaches to Clinical Trials and Health-Care Evaluation. John Wiley & Sons, New York. Woodward, P. (2012). Bayesian Analysis Made Simple. An Excel GUI for WinBUGS. Chapman & Hall/CRC. Phil Woodward 2014 116