Technology-based education and intervention tools in medical education: Lessons and methods for the University community Scott M. Strayer, MD, MPH Associate Professor Department of Family Medicine University of Virginia Health System My Colleagues and Co-Authors Sandra Pelletier, PhD Crista Warniment, MD Karen Ingersoll, PhD Steve Heim, MD John Schorling, MD, MPH Preventable Causes of Death The most common causes of disease, disability, premature death, and health care burden in the US can be directly attributed to 4 health risk behaviors: Smoking tobacco Risky use of alcohol Unhealthy diet Physical inactivity 1. McGinnis JM, Foege W. Actual causes of death in the United States. JAMA. 1993;270:2207-2212. 2. Mokdad AH, Marks JS, Stroup DF, Gerberding JL. Actual causes of death in the United States, 2000. JAMA. 2004;291:1238-1245. Health Behavior Counseling Obesity In surveys of patients who had a routine checkup with a physician in the past 12 months, only 14% of respondents reported receiving advice from their physician to lose weight, even though approximately 34% of the sample was classified as overweight and 19% as obese (Sciamanna et al., 2000). Moreover, only 16% of overweight individuals and 40% of obese individuals reported receiving advice from their physician to lose weight during their checkup within the past twelve months (Loureiro, 2006). Health Behavior Counseling Alcohol Use Only 23% of binge drinkers are counseled about alcohol use (Am J Prev Med 2003;24(1):71–74) A survey of family physicians and internists found that only 64.9% of respondents screened 80-100% of their patients for alcohol abuse or dependence during the initial visit and a mere 34.4% screened that many patients during an annual visit Only 20% said treatment resources for early problem drinkers was adequate 72% preferred not to counsel patients themselves (Spandorfer & Turner 1999) Health Behavior Counseling Smoking Cessation Smoking cessation counseling only occurs at 23 to 46% of primary care visits Only 35% of physicians assist with smoking cessation attempts Less than 10% arrange follow-up for smoking patients •Thorndike AN, Rigotti NA, Stafford RS, Singer DE. National patterns in the treatment of smokers by physicians. JAMA 1998; 279:604-608 •Lewis CE, Clancy C, Leake B, Schwartz JS. The counseling practices of internists. Ann Intern Med 1991; 114:54-58. •Goldstein MG, DePue JD, Monroe AD, et al. A population-based survey of physician smoking cessation counseling practices. Prev Med 1998; 27:720-729. Barriers Physicians Competing demands Conflicting recommendations Lack of training Patients Lack of knowledge Fear of discomfort Cost Office Poor reimbursement Lack of systems Opportunities for Intervention Most people visit a primary care doctor about three times per year. Even 2-3 minute interventions are effective, especially when followed up with telephone, e-mail, nurse calls, referrals, 1-800 numbers, etc. Many primary care providers provide 2-3 minute health promotion/behavior interventions at every outpatient visit. Stange, KC, Woolf, SH, Gjeltema K. One minute for prevention: The power of leveraging to fulfill the promise of health behavior counseling. Am J Prev Med, 2002; 22:320-323. Integrating the Behavioral Theories Strayer SM, et al. Development and evaluation of an instrument for assessing brief behavioral change interventions. Patient Educ Couns (2010), doi:10.1016/j.pec.2010.04.012 Research Design RCT with 3rd year family medicine clerkship students All students received smoking cessation workshop based on 5 A’s, Stages of Change and MI Randomization by paired rotation blocks to receive either a reminder card or a handheld computer tool - the Educational Smoking Mobile Intervention Tool (E-SMOK-I.T.) Students assessed pre and post study for smoking cessation counseling knowledge and self-reported behavior and comfort using a previously validated survey instrument Students assessed for appropriate smoking cessation counseling using video-taped standardized patient interview using an instrument that was developed to assess combined smoking cessation counseling interventions 2 independent observers assessed key smoking cessation counseling activities using videotapes and rating form Focus groups held with E-SMOK-I.T. tool users post clerkship Study Flow Diagram Educational Smoking Intervention Tool (E-SMOK-I.T.) Main Screen Assessment Screen E-SMOK-I.T. (cont). Assist Screen Arrange screen Analysis Frequency and descriptive analyses Inter-rater reliability for videotape assessment Pre/post survey analysis using mixed ANOVA Smoking cessation counseling skills using Independent t-tests Results 116 students completed the study and had complete video observations (N=63, control, N=53 intervention). Mean age for participants was 26 and 52% were male. 62% reported handheld computer literacy as intermediate on a 3 point scale. Average Inter-rater reliability was 0.82 (Intraclass correlation) Results Overall smoking cessation counseling behaviors, knowledge and comfort increased among all participants (p<.001) Paper-based group performed better than the handheld tool group (69% vs 62%; t=2.318, p=.022) at end of clerkship No difference between groups at end of academic year (61% vs. 59%; t=.621, p=.538) Focus groups of each rotation block revealed several possible reasons for results. Pre-post Survey Results Mean Summary Score Survey Subcategories 18.97 20.0 12.89 15.0 10.0 11.98 9.71 9.00 6.95 Pre Post 5.0 0.0 Knowledge Behavior Comfort Knowledge pre/post increase 0.71 (F=15.54, p<0.001) Behavior pre/post increase 7.026 (F=194.45, p<0.001) Comfort pre/post increase 5.93 (F=163.620, p<0.001) 69 Mean Proportion Correct Counseling Skills 62 61 60 70 59 51 60 50 40 Paper-Based 30 ESMOK-I.T 20 10 0 Counseling MI Spirit Retention Counseling** p<.022, MI Spirit** p=.004, Retention p=.538 Smoking Cessation Counseling Skills Focus Group Themes Discomfort using at point of care or patient not receptive: “I think I might have used it more if I were more comfortable with all the other aspects of being in that room.” “… but the patients were either like I don’t want to quit or they knew what they needed.” Forgot or Gossip about tool: “I was afraid of it … I heard that there were problems with it with info-retriever I was afraid about that” Focus Group Themes Would have used if more review / training: “It might be useful if I spent more time learning how to use it.” Not enough time, smoking not priority: “The times that it really gets brought up it is a long appointment you don’t have enough time to spend on smoking” Tool used as educational reference “I did like how you could go to related things inside the program. But mostly I went through it to review it.” “Looked at it once or twice but not with patient” Conclusions Smoking cessation counseling by 3rd year medical students was improved using a combination of workshop and a supplementary reference tool. Our hypothesis that the point of care tool would enhance skills significantly did not occur (unlike with our studies in practicing physicians) Barriers specific to the 3rd year training period need to be addressed in order for a point of care tool to be effective in this setting Using the Tool With a Patient SBIR Phase 2 Contract Phase II Research Aims Conduct user-centered design evaluations to validate design and usability of new functionality Assess the ability of QuitAdvisorMD to influence smoking cessation as measured by The increase in clinician initiated smoking cessation counseling The increase in ability of clinicians to provide appropriate smoking cessation counseling based on the PHS guidelines (and in particular, physician performance of the 5 A’s of: asking patients about smoking status, advising patients to quit smoking, assessing patient readiness to quit smoking, assisting patients with their quit attempts or in giving motivational interventions if they are not ready to quit smoking, and arranging follow-up for all patients who are smokers) The increase in patient quit attempts. Cross-study Themes Content is a valued reference/resource (AMIT, QA, ACS) Users liked the software as an educational tool (AMIT, ACS) Need more practice and training (ACS, AMIT, QA) Discomfort using computers with patients (AMIT, ACS) Lack of time (ACS, QA, AMIT) Conclusions The content for computer-assisted counseling tools was appealing to all levels of learners Many users see this software as an educational tool All levels of learners requested more training and practice Discomfort using computers with patients affects learners and even some practicing physicians Lack of time for counseling is a persistent barrier Questions?