Report

Markov versus Medical Markov Modeling – Contrasts and Refinements Gordon Hazen February 2012 Medical Markov Modeling • We think of Markov chain models as the province of operations research analysts • However … • The number of publications in medical journals – using Markov models – to address medical cost-effectiveness – approaches 300 per year! 2 Medical Markov Modeling • Why the large buy-in from the medical community? – Easy-to-use software that combines decision trees and Markov models (Data, TreeAge) – Simplicity of models • Discrete time • Transient 3 Overview of this talk 1. Background on medical Markov modeling 2. Population modeling versus individual-level modeling 3. Product structure in medical Markov models Overview 1. Background on medical Markov modeling 2. Population modeling versus individual-level modeling 3. Product structure in medical Markov models. Medical Markov Modeling • The kind of modeling that is typical IHD = Ischemic heart disease MI = Myocardial infarction (heart attack) A simplification of: Palmer S, Sculpher M, Phillips Z, Robinson M, Ginnelly L, Bakhai A et al. Management of non-ST elevation acute coronary syndrome: how cost-effective are glycoprotein IIb/IIIa antagonists in the U.K. National Health Service?. International J Cardiology 100 (2005) 229-40. 6 The kind of modeling that is typical • Cohort analysis 0.9 IHD 0.7 0 Post MI Dead 1000 1 0 0.9867 986.67 0.01 10 0 Alive QALY 1000 1 900 0.9 0.0033 996.67 3.3333 0.9967 0.895 895 0.9735 19.533 973.51 0.0195 6.9556 0.007 993.04 0.993 889.83 0.8898 … … … … 7 … Our preference: Continuous-time pMI = lDt p0 = m0Dt p1 = m1Dt • Cohort analysis in continuous time = 8 Our preference – continuous time • Discounted expected quality-adjusted life years: 9 Cohort analysis in continuous time • Intervention: Post-MI mortality rate m1 = 0.1/yr is decreased by 75% and the MI incidence rate l = 0.12/yr is decreased by 50%. p IHD( t ) 1 1 0.8 0.8 p IHD( t ) 0.6 0.6 p P ostMI ( t )0.4 p P ostMI ( t )0.4 0.2 0.2 0 0 0 5 10 15 20 0 5 10 15 t t (yr) (yr) 6.62 QALY/patient 10.28 QALY/patient 10 20 Continuous-time version of cohort analysis + = + − () • Let dt 0 to obtain … the Kolmogorov differential equations. • The cohort analysis procedure is merely the Euler method for solving the Kolmogorov equations. 11 Overview 1. Background on medical Markov modeling 2. Population modeling versus individual-level modeling 3. Product structure in medical Markov models. Question: How to incorporate population issues? • An intuitive approach: Restart following death Open routing process Closed routing process • Then compute steady-state probabilities in the resulting irreducible chain. 13 Question: How to incorporate population issues? • Balance equations for steady-state probabilities: • Intervention assumptions: – Post-MI mortality rate m1 = 0.1/yr is decreased by 75% – MI incidence rate l = 0.12/yr is decreased by 50%. • Results: PostMI increases from 23.0% to 38.5% • The population is less healthy! • So what is wrong here? 14 Population issues: A more rigorous approach • Observation: A population of non-interacting individuals is equivalent to a Jackson network of infinite-server queues. 15 Equilibrium results • Jackson network balance equations = ℎℎ • Theorem (e.g. Serfozo 1999): The counts nj of individuals in health state j are, at equilibrium, independent Poisson variables with means j given by the solution to the balance equations. 16 Equilibrium results • Solve balance equations with entrance rate n = 1000/yr More survivors under intervention! mean (sd) IHD Status Quo 6250 (79) Intervention 10000 (100) Post MI 1875 (43) 6000 (77) 8125 (90) 16000 (126) The closed routing process again • Convert the open routing process to a closed one in the following way Open Closed ′ = + = 18 . Open versus closed routing Equilibrium means j Steady-state probabilities j • Theorem (Hazen and Huang 2011): One may obtain equilibrium means from steady state probabilities, and vice versa: j n j j m j j j • where = is the total arrival rate, and = the equilibrium departure rate . 19 is . Open versus closed results Equilibrium means j IHD Status Quo 6250 Intervention 10000 MI 1875 6000 Total 8125 16000 Steady-state probabilities j IHD MI Status Quo 77.0% 23.0% Intervention 62.5% 37.5% 20 Example: Re-analysis of the preventive use of tamoxifen • Original analysis: Col et al 2002 • Tamoxifen – an estrogen agonist/antagonist – an effective therapy against established breast cancer • Evidence that it can reduce breast cancer incidence • But life-threatening side effects – endometrial cancer – vascular events. • Would the benefit of its prophylactic use in healthy women be worth the associated risks? 21 Preventive use of tamoxifen: Our model • Cure rate models for breast and endometrial cancer treatment – Mortality decreases in time survived after cancer diagnosis. – This cannot be directly modeled as a Markov model – it is semi-Markov. – Cure rate model with unobserved states Cured/ Not Cured allows implicit mortality to decrease over time survived. No Disease No Disease lb pb Cured Breast Ca 1-pb le mb Death pe Endometrial Ca Not Cured Breast cancer incidence and treatment Cured me Not Cured 1-pe Endometrial cancer incidence and treatment 22 Death Preventive use of tamoxifen: Our model • Overall model is the Cartesian product of the two factors below and a third Background Mortality factor. • More on this later … No Disease No Disease lb pb Cured Breast Ca 1-pb le mb Death pe Endometrial Ca Not Cured Breast cancer incidence and treatment Cured me Not Cured 1-pe Endometrial cancer incidence and treatment 23 Death Preventive use of tamoxifen: Our model • Estimated parameters (max likelihood estimates) No Disease No Disease lb pb Cured Breast Ca 1-pb mb pb μb Death pe Cured Endometrial Ca Not Cured me Death Not Cured 1-pe Variable Description incidence rate of br ca without λb0 tamoxifen (high risk) RRb le Value 0.0086/yr risk ratio of br ca with tamoxifen probability of cure for br ca 0.4936 mortality rate of br ca if not cured 0.0996/yr 0.5631 Variable Description Value incidence rate of endo ca 0.00152/yr λe0 without tamoxifen risk ratio of endo ca with 4.0132 RRe tamoxifen probability of cure for endo ca 0.9019 pe mortality rate of endo ca if not 0.3159/yr μe cured Variable Description Background mortality (age 50) μ0 Value 0.03118/yr 24 Preventive use of tamoxifen: Our model • Product structure for quality of life – Q jk = Qbj Qek – more on this later Endo Ca Qjk Br Ca No Ca Cured Not Cured No Ca 1 0.81 0.39 Cured 0.81 0.656 0.316 Not Cured 0.39 0.316 0.152 • Model entry rate n0 = 110,000/yr – 2.3 M women reaching age 50 each year x 4.8% at high risk for breast cancer 25 Preventive use of tamoxifen: Results Incremental QALYs/woman Incremental equilibrium probabilities Incremental Endo Ca DQALYjk No Ca Cured Not Cured Endo Ca None Cured Not Cured No Ca 0.405 1.07 0.004 Djk Cured -0.791 0 0 None -0.02 0.10 0.00 Not Cured -0.032 0 0 Br Ca Cured -0.08 0.01 0.00 Not Cured -0.01 0.00 0.00 -0.11 0.11 0.00 Br Ca 0.08 -0.07 -0.01 Incremental equilibrium means D jk Incremental Endo Ca (1000s) No Ca No Ca -15.84 Br Ca Not Cured 322.74 2.824 309.7 Cured Cured -262.79 35.31 0.131 -227.3 Not -34.87 3.52 0.023 -31.3 Cured -313.5 361.57 2.98 51.0 • A more nuanced picture of the effects of this intervention than just incremental QALYs . 26 Overview 1. Background on medical Markov modeling 2. Population modeling versus individual-level modeling 3. Product structure in medical Markov models. Markov models with product structure • Product structure is relatively common in medical Markov models Roach PJ, Fleming C, Hagen MD, Pauker SG. Prostatic cancer in a patient with asymptomatic HIV infection: are some lives more equal than others? Med Decis Making. 1988 AprJun;8(2):132-44. 28 Markov models with product structure • Much simpler depiction of model structure: Independent factors 29 Product structure is relatively common Schousboe JT, Nyman JA, Kane RL, Ensrud KE. Cost-effectiveness of alendronate therapy for osteopenic postmenopausal women. Ann Intern Med. 2005 May 3;142(9):734-41. • Schousboe et al. considered 5 different types of fractures: – – – – – hip fracture clinical vertebral (Cv) fracture radiographic vertebral (Rv) fracture distal forearm (Df) fracture other fracture • In principle this should allow 25 = 32 state combinations corresponding to 5 factors each at 2 possible levels. • What their model actually did: 6 states – 5 states corresponding to a single fracture type – 1 other state corresponding to the combination of the worst two possible fracture types – Disadvantage: Such a model “forgets” past fractures when a new fracture occurs, which the 32-state model would not do. 30 Advantages of explicitly accounting for product structure • Model formulation: Simpler to merely consider one factor at a time • Model presentation: Simple factors easier to understand and critique. – Model is less likely to be perceived as a “black box” • Computational advantages as well when factors are independent. 31 Computation of QALYs under product structure • Product structure – Health state = 1 , … , – Quality coefficient = ( ) • The latter assumption on is equivalent to standard gamble independence (Hazen 2003) • Standard gamble independence: 1-p p (yi*,zi,t) ~ (yi*,zi,0) (yi,zi,t) True for one implies true for all - the indifference does not depend on what is. 32 Computation of QALYs under product structure • Product structure – Health state = 1 , … , – Quality coefficient = ( ) • Theorem (Hazen and Li 2010): One may calculate mean QALYs as follows. ∞ E[QALY/person] = 0 − E[ ] E = E = E[ ] ( ) () = P(ℎ ) (cohort analysis in factor i) 33 Computation of QALYs under product structure Cancer Health quality Year 0 1 2 3 4 5 6 7 8 9 10 38 39 40 1 0.6 0 1 E[QALY No Rate]/ Cancer Cancer Death person 10000 0 0 1.0000 9680 320 0 0.9872 9371 545 85 0.9697 9071 700 229 0.9491 8781 804 415 0.9264 8500 872 628 0.9023 8228 912 859 0.8776 7965 934 1101 0.8525 7711 941 1349 0.8275 7464 938 1598 0.8027 7225 928 1847 0.7782 2908 2815 2725 Bkgd Mortality AIDS 399 386 374 6693 6799 6901 No AIDS 10000 9048 8187 7408 6703 6065 5488 4966 4493 4066 3679 0.3148 0.3047 0.2950 224 202 183 0.5 0 E[QALY Rate]/ AIDS Death person 0 0 1.0000 952 0 0.9524 1212 601 0.8793 1226 1366 0.8021 1157 2140 0.7282 1064 2870 0.6598 970 3542 0.5973 880 4154 0.5406 797 4710 0.4892 721 5213 0.4426 653 5668 0.4005 40 36 33 9737 9762 9784 0.0244 0.0220 0.0199 Mortality Rate Survival /10,000 Prob. 1.0000 99 0.9901 107 0.9796 116 0.9682 126 0.9561 136 0.9431 148 0.9293 160 0.9146 173 0.8988 188 0.8821 203 0.8644 1894 2051 2221 0.0952 0.0775 0.0621 Total QALY Rate/ person 0.5 0.9038 0.7873 0.6745 0.5730 0.4843 0.4080 0.3427 0.2872 0.2402 0.2005 0.0002 0.0002 0.0001 6.3331 = P(ℎ ) E = E = ( ) () E[ ] ∞ E[QALY/person] = 0 − E[ ] 34 Computation of expected cost under product structure • Assumptions – Costs accrue in each factor as long as survival is maintained in other factors – = P(Survival through time t in factor i) – Costs add across factors • Theorem (Hazen and Li 2010): Computational formulas are E[Cost/person for factor i] = E = ∞ − E[ 0 ] ≠ ( ) () = P(ℎ ) (cohort analysis in factor i) 35 Advantages of factored computation • Computational work in cohort analysis is proportional to the number of state transitions • Suppose the number of transitions in a factor with s nondeath states is roughly s also. • Then assuming s states in each factor and f factors, – sf transitions in the overall model under naïve cohort analysis – sf transitions in cohort decomposition – Big advantage for large f • Caveat: s and f are not usually large. 36 Cohort decomposition issues • How often are factors independent? – Ans: More often not probabilistically independent. – But one factor is almost always probabilistically independent: Background mortality. • How reasonable is the product form for the quality coefficient v(x)? – Empirical support for product form in HUI literature – additive decomposition is not supported. – Often only one factor carries quality adjustments, in which case product form holds by default. 37 Summary • These are just the basics – Population modeling • Population model Jackson network • One can get at equilibrium population issues by solving the usual balance equations for steady-state probabilities and scaling them up appropriately. – Product structure • Common feature of medical Markov models • Recognizing it can assist in model formulation and presentation, as well as computation. – Drawbacks for continuous-time models • Medical researchers don’t “get” the models • Software not widely available • There is more to do here … Questions?