“Modules of Workforce Development: Pragmatic Approaches to Knowledge Acquisition, Application, and Information Dissemination” Enterprise Access to Evidence-Based Literature: A Dynamic Digital Library for Public Health Enterprise Access to Evidence-Based Literature: A Dynamic Digital Library for Public Health Karen Dahlen, Project Consultant Elaine Martin, Director New England Region NN/LM Lamar Soutter Library University of Massachusetts Medical School This project is currently funded by the NLM via contract through the NN/LM, New England Region Enterprise Library Approach to Knowledge Acquisition, Application & Information Dissemination Measure Importance of Trusted Information Access Ensure Direct Access from Intranet-Based Site Access (Without Name/Password Required) Support Core Competencies/Training Use Survey Instruments Adapted to the Location Relate Cost Efficiencies to Enterprise Licensing/Creative Partnerships Document Institutional Change in a Dynamic Environment Evaluate Over Time (Illustrate Efficiencies) Shifting the Culture to Evidence-Based Practice • • • • • Culture of evidence-based practice was affected when funding for public health libraries was eliminated. State Libraries (in many states) were mandated to support state agencies, but had no collections to support public health. Streamlined access to resources through the Digital Library Platform-along with trust in new relationships--prompted resurgence in use of resources and has facilitated this cultural shift. Training stimulated access to resources reviving evidence-based interest and relationship to practice. Project has the ability to leverage government resources (NLM, CDC, NAL) to improve evidence-based practice. Background (Why this Project) Project Rationale Partnerships ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH Council on Linkages: Academia & Public Health Practice Public Health Competencies Project Tied to Competencies Competencies: Council on Linkages • Analytic/Assessment Skills – – – • • Policy Development/Program Planning Skills – • References sources of public health data and information. Information technology to collect, store, retrieve data. Utilizes data to address scientific, political, ethical, and social public health issues. Analyzes information relevant to specific public health policy issues Public Health Sciences Skills – – Conducts a comprehensive review of the scientific evidence related to : a public health issue, concern, or intervention. Retrieves scientific evidence from a variety of text and electronic sources. • Knowledge Informatics – Access trusted resources directly from the desktop. – Describe relationships and functionality of eresources to core public health interests. – Introduce specific types of resources: coding and classification tools. Informatics Skills – Train on interoperable features that support public health work, including connectivity to trusted sites; advanced search filters, use of topical alerts and information mgt software. Obstacles to Getting Information Gathering and Analyzing EB Information Rapid Detection of Epidemics Detecting an anthrax epidemic one day earlier would save $1-7 billion CDC Presentation by C. Safran citing: http://www.cdc.gov/ncidod/eid/vol3no2/kaufman.htm Kaufmann AF, EID, V3, N2 Library Without Signature Building Goal of the Project Who Participates Objectives & Data Sets Development of Digital Library ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH PHIA: Innovative Business Model Not Tied to Traditional Building Madison, WI Atlanta, GA Goal of PHIA Public Health Information Access To determine what resources are core, useful, evidence-based to advance public health practice. Full text directly available through DL Core to public health Full text available through LP Unique Aspects of Digital Library • Resides on PHD Intranet (outside a signature building) without need for id and password. • Relies on collaboration, interoperability & funding. • Move toward analysis; less time on access • Skills & training support personalized, interactive styles tied to continuum of learning and institutionalspecific priorities and projects. • Collection of evidence occurs in many aspects of Brownson wheel not just at the literature review stage. • Fundamental change in the way resources are distributed and utilized. Public Health Departments & Library Partners State Public Heath Department Library Partners AK AR CO Alaska Division of Public Health (AKDH) Univ of Alaska (Anchorage) Medical Library Arkansas Department of Health (ADH) Univ of Alabama Medical Sciences Library Colorado Dept of Public Health & Environment Poudre Valley Health System Library CT HI IN KY ME MA MA MD NH RI VT WI WV WA Connecticut Public Health Department Hawaii Department of Health Indiana State Department of Health Kentucky Department for Health Maine CDC Boston Public Health Commission* Massachusetts Public Health Department Maryland Dept of Health & Mental Hygiene New Hampshire DHHS Rhode Island Department of Health (RIDH) Vermont Department of Health (VDH) Wisconsin Department of Health Services West Virginia Bureau of Public Health Washington State Department of Health University of Conn Maynard Stowe Library University of Hawaii Medical Library Ruth Lily Medical Library, Indiana University University of Kentucky Medical Library Maine Medical Center Library University of Massachusetts Soutter Library Lemuel Shattuck Hospital Library University of Maryland HS Library New Hampshire DHHS Library Rhode Island Hospital/Lifespan UVM Dana Medical Library Ebling Library of the Health Sciences West Virginia University Libraries University of Washington HS Library PHIA Evaluation Process Tied to Data Model To advance access to evidence-based resources in support of improved public health practice. What resources are high use? What technical issues arise? What resources are missing? How can training be improved? What project components lead to improved efficiencies? What cost efficiencies are involved with an enterprise approach to digital library implementation, and training. • Logic Model – • Resources, activities, outputs, outcomes (minutes, visits, trainings, people lists, IP management, levels of troubleshooting, RML reports. Data Tools – – – – – Overarching questions Journal SurveyMonkey Pre-Post Training Surveys Just-in-time Information Checks Formal Evaluation (interviews & focus group sessions Data Sets & Sources • • • • • • • • Resource Metrics Collected Journal Survey Monkey Identifies Perceived Need & Library Use. Document IT technical issues. Licensing Issues Related to Journals and Databases. Training Surveys Collected Preliminary Meetings/Updates Held. Trainings Held Year End Evaluation --Interviews & Focus Groups View of PHIA Digital Libraries Connecticut Digital Library Arkansas Digital Library Resources on Digital Library Use of Resources Article Delivery Use & Costs ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH Databases & Major Publishers of Resources PubMed , Other NLM Databases NAL, CDC & Other Gov’t Resources CLSI (Clinical Laboratory Standards) Global Health Cochrane Library – Systematic Reviews Stat!Ref—E Books (35) -- Coding & Classification ASABE –Health Linked to Agriculture Identifying Core, Useful, & Evidence-Based Journals Publisher Total Use Journal Title Use Mass Medical Society (1) 14,136 New England Journal of Medicine 14,136 AAP(1) 9,888 Pediatrics 9888 Oxford University Press (20) 23,369 Clinical Infectious Diseases 3147 Springer (18) 2, 485 Maternal & Child Health 474 ASM Journals (12) 3,518 Journal of Clinical Microbiology 1729 Annual Reviews (9) 697 Annual Review Public Health 394 Sage (11) 939 The Diabetes Educator 135 BMJ (9) 2932 British Medical Journal 1832 Mary Ann Liebert (11) 1798 Breast Feeding Medicine 355 University of Chicago Press (6) 955 Infection Control & Hospital Epidemiology 749 Lippincott/Ovid (6) 122 J Public Health Mgt Practice 55 NEJM Use 2010-2013 2010=2087 2011=11,194 2012=14,022 2013=14,136 7000 6000 5000 4864 4000 3000 2000 1000 0 1772 982 554 314 527 337 311 890 1183 1178 308 494 422 ASM: Journal of Clinical Microbiology Total Use 2013=1729 298 287 233 175 135 94 90 89 82 78 71 42 32 18 5 Oxford University Press Use Jan-Dec 2013 Total Use by Each PHD N=13,827 PHD Total Alaska DPH Arkansas DH BPHC CDPHE CT DPH HI SDH ISDH KYDPH Maine CDC MDHMH MDPH NH DHHS RIDH 1245 1483 182 608 1234 638 700 1134 757 1427 1817 157 620 VDH WI DHS 709 1116 High Use Journal Titles Journal Title Total Use Clinical Infectious Disease American Journal of Epidemiology Schizophrenia Bulletin The Journal of Infectious Diseases 3147 941 771 627 ICES: Journal of Marine Science JNCI: Journal of the National Cancer Institute 474 Journal of Antimicrobial Chemotherapy 297 International Journal of Epidemiology Alcohol and Alcoholism Journal of Analytical Toxicology Health Promotion International 288 274 262 252 Rheumatology Age and Ageing 246 240 434 STAT!Ref E-Books/Tools Provide Basic Understanding of Topic Filters & Tools – Codes • • CPT with RVUs Data File, INGENIX® (2011) ICD-9+CM - Volumes 1, 2 & 3 (2012) – Titles By Discipline Searchable Alerts (by topic) – – – – Functionality with NLM Chapters link to PubMed TOXNET linked to Search Related Concepts/Meta… Functionality with CDC – MMWR, Community Guide; STAT!Ref E- Books for Public Health Titles Containing PH Content E-Book Use Oxford Textbook of Public Health - 5th Ed. (2009) 482 407 Manual of Clinical Microbiology - 10th Ed. (2011) 403 AHFS Drug Information (2013) 305 ACP PIER, Journal Club & AHFS DI Essentials (2013) Coding & Classification Books E-Book Use ICD-9-CM - Volumes 1, 2 & 3 (2013) 881 CPT with RVUs Data File, INGENIX (2013) 474 ICD-10-PCS: Procedure Coding System (2014) 12 Red Book: 2012 Report of the Committee on Infectious Diseases - 29th Ed. (2012) 241 Emerging Infections Series (2008 - 2010) 216 ICD-10-CM: Clinical Modification (2014) 15 209 Diagnostic and Statistical Manual of Mental Disorders 4th Ed. (DSM-IV-TR, 2000) 107 Textbook of Modern Toxicology, A - 4th Ed. (2010) CLSI (Clinical & Laboratory Standards) 20 Documents Used 790 Times April 21, 2014 Doc # Document Name # M100 M100-S24: Performance Standards for -S24 Antimicrobial Susceptibility Testing; TwentyFourth Informational Supplement 248 EP09- Measurement Procedure Comparison and A3 Bias Estimation Using Patient Samples; Approved Guideline—Third Ed. 102 EP12- User Protocol for Evaluation of Qualitative A2 Test Performance 55 MM0 Nucleic Acid Sequencing Methods in 9-A2 Diagnostic Laboratory Medicine; Approved Guideline—Second Edition 54 Calculations Related to Enterprise Licensing State PHD Enterprise Licensing is based on 10% of the Total FTE level to capture baseline information and measure use/interest of resources. FTE 10% Rule ADH AKDPH BPHC* CDPHE CT DPH Hawaii DPH 2781/1390 525 1100/550 1200/600 780 3000/1500 ISDH Kentucky BPH MDHMH Maine CDC MDPH NH DHHS RIDH VDH WDPHS VW DHS Washington 780 400 8000/4000 395 3000/1500 279 400 300 400 780 150 78 40 400 39 150 30 40 35 40 78 1600/800 10,999 80 1546 Total PHIA FTE 139 52 55 60 78 Article Delivery Costs Via Library Partnerships May 2011 – April 2012; Cost=$18,384 STATE Connecticut ILL DD Total 0 83 83 Maine Massachusetts (BPHC) Massachusetts (MDPH) New Hampshire 560 Rhode Island 364 924 8 3 11 1 1 2 0 0 0 47 36 83 Vermont 21 191 212 Colorado 323 156 479 Total 960 834 1794 May 2012 --- April 2013; Cost=$12,241 STATE ILL DD Total 2 20 22 Maine Massachusetts (BPHC) Massachusetts (MDPH) New Hampshire 154 90 244 9 10 19 7 6 13 0 0 0 Rhode Island 82 79 161 Vermont 54 243 297 Colorado 112 40 152 Arkansas 32 73 105 Kentucky 8 3 11 Wisconsin 0 0 0 462 593 1055 Connecticut Total Training Topics Embrace National and State Interests Maryland DHMH Health Initiatives • • • • • • Health Disparities(sexual minorities) Preventive Services ACA* HIV Screening Immigrant Health Undocumented individuals under ACA Drug Resistant Disease which impacts Impact on TB, STI (plus others) Health Care Reform and Interpersonal Violence/Domestic Violence Infectious Diseases; Hepatitis C Virus HIV Infections --HIV and HCV Co-Infection Number of Trainings/Diversity of Workforce STATE PHD Trainings Attendance Maryland Indiana Hawaii Nutrition Specialist (2) State Epidemiologist Education Coordinator (2) Research Statistician Supervisor – Labs + (3) Asthma Coordinators(2 Policy Analyst (2) Injury Prevention Epidemiologist Legislative Liaison RI RIDH 2 30 VT 3 46 ME VDH MAINE CDC* MA BPHC* 4 58 MA MPHD* 3 32 Program Mgr (2) Staff Attorney Epidemiologist NH NH DHHS 2 24 Epidemiologist Microbiologist ( 2) Research Analyst (2) CO CDPHE* 5 100 CT CT DPH* 5 101 Medicare Specialist/Advisor (2) Regional Program Director Program Coordinator, SAPB KY AR KDPH ADH 2 2 46 Policy Advisor (2) Director of Prog Evaluation Epidemiologist (3) WI WI DPH 2 33 Psychiatric VT Director of Prog Development Genetic Counselor AK AK DPH 2 50 Executive Ass’t Field Epi Director Tobacco Prevent Mgr HI HI DPH 2 68 IN ISDH 2 60 Deputy Secretary Access Services Mgr Informatics Analyst MD MDHMH 2 38 Director VSA IT Supervisor Nutritionist 15 41 769 TOTAL 3 52 31 Enterprise Efficiencies Unique Aspects of the Business Model Evaluation Processes ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH Cost Efficiencies: Article Analysis Direct Access to Articles Contract Year Total # Amount $ May 2013 –April 2014 17+ 285,409 May 2012-Apr 2013 15 May 2011-Apr 2012 May 2010-Apr 2011 Alternative Article Delivery Contract Year Total # Amount $ May 2013-March 2014 1038 10,102 200,352 May 2012-Apr 2013 1055 12,241 11 156,280 May 2011-Apr 2012 1794 18,385 9 96,252 May 2010-Apr 2011 672 8,263 Enterprise Licensing Efficiencies: Single Titles Contract Year May 2012-April 2013 Journal Title # Uses Cost Per Use Replacement Cost Cost of License Am J Tropical Med & Hygiene 410 $1.58 $4,510 $ 650 Am J Respiratory & Critical Care Med 189 $14.28 $2,079 $2700 Health Affairs Infection Control & Hospital Epidemiology Public Health Reports 2038 1.81 $22,418 $3690 749 1.50 $8,239 $1125 658 4.69 $7,238 $3085 Intern’l J of Tuberculosis & Lung Diseases 192 5.83 $2112 $1120 Annual Review Series 595 12.60 $6545 $7500 Pediatrics 7846 .68 $86,306 $5300 PHIA: Innovative Business Model • • • • • • Enterprise licensing saves time, money, and effort. Central management of IP addresses ensures quick turn-around. Ability to understand change (System, personnel, environment). Resources are identified via benchmarking,metrics of direct use through Digital Library, and requests from PHDs. Journals are identified via “Article Delivery” on Library Partner side. Alternative delivery of resources strengthens state relationships with immediate access (within 4-24 hours). EVALUATION PROCESSES Levels of Data Collection • • • • • • • • • • Journal SurveyMonkey (baseline data for PHIA and PHD) Enterprise Licensing (cost effectiveness over time) Vendor statistical reports (validates “use of resources”) Monthly reports from “library partners” (measures use and identify resources to develop Digital Libraries). Ongoing capture of suggested e-books, journals, and databases to enhance collection Pre-Post training links – (data related to knowledge of resources) Feedback from hands-on training Interviews with leaders and workforce Focus group sessions (after one year; subsequently) Quarterly/Annual Reports (submitted to NLM) Results • Digital libraries will be in place in 17 PHDs by July. • Full-text access to more than 150 e-journals, 5 databases, reports and more. • Introductory sessions have introduced PHD leadership to the project and explained obligations. • More resources have been added to “digital library.” • Structured training has been held in 15 PHDs. • More than 600 people have been trained with backup models currently in test stage. NACCHO Award Brings Recognition to Project • March 2012 and 2013, the PHIA PHD Digital Library Project received a “promising practice award” from NACCHO. • Presentations have been held at national and local meetings, e.g., APHA in Washington, DC in 2011 and Boston in November 2013. • Article was published in AJPH in January 2014. Project was cited in August 2012 EID (CDPHE). • Presentations have been made at national public health meetings, national and regional library meetings. Testimonial “Great recognition for this very valuable initiative. Kudos and thank you.” Jewel Mullen, MD, MPH, MPA, Commissioner, Connecticut Department of Public Health Discussion & Outcome How Do We? • Expand PHIA to all 50 states. • Enhance partnerships & relationships. • Continuously train PH Workforce to improve competencies (knowledge and skills) given staff turnover. • Build a sustainable model including cost structure and project management.