Association of Research Libraries Assessment Conference, Baltimore, Maryland: October 27, 2010 PAPER BY: Rachel A. Fleming-May, Assistant Professor, School of Information Sciences, The University of Tennessee, and Crystal Sherline, M.L.I.S., Doctoral Student College of Communication and Information, The University of Tennessee (presenter) Current Academic Library Assessment Climate (in a nutshell): Darkening financial horizon Increasing costs of resources and services Increased emphasis on institutional-level evaluation for Foundations, so far… Significant contributions to understanding library’s contribution to research: Study of library return on investment (ROI) in library resources in development of successful grant applications: Phase I (U.S.) and Phase II (International) Ongoing efforts to shift academic library assessment conversation from input and output measurement to a focus on outcomes… ARL New Measures Initiative …and to improve understanding of the “state of the art” of academic library assessment Megan Oakleaf and ACRL’s Value of Academic Libraries: A Comprehensive Research Review and Report Significant progress has been made in establishing models for demonstrating Return on Investment for Research. …but what of the Academic Library’s Contribution to Teaching and Learning ? More difficult to assess and demonstrate CAUSATION. Most studies demonstrate either Correlation between student success and library use, or Short-term effects of information literacy instruction. We’re also not always entirely sure what we should be trying to demonstrate. Our Project Part of IMLS-Funded Project: Carol Tenopir, The University of Tennessee: Lead Principal Investigator Paula Kaufman, The University of Illinois: Co-Principal Investigator http://libvalue.cci.utk.edu/ Our Goals and Approach: Helping to create structure for future assessment efforts: Shift from initial conceptualization presented in abstract for the LAC: Focus on Six Regional Accrediting Agencies’ Standards Several LIS-related projects have looked at Regional Standards, but most have either: • Focused on one agency’s standards, or • Focused on standards specifically related to THE LIBRARY or LIBRARY-RELATED ENDEAVORS, such as INFORMATION LITERACY SO WHAT! IL is important! , IT IS… But we believe (and we’re not alone) academic libraries need to expand evidence of their influence to reflect institution-level standards in order to maintain relevance. QUICK REVIEW: Institution-Level Accreditation in the U.S. No common standards at the national level; Six regional agencies Similar themes in standards, but lack of uniformity: Language Structure Frequency of updates All Six Agencies’ Standards address Library Collections with varying degrees of specificity; Information Literacy in direct or “Equivalent Language” Other Standards for Academic Libraries: Program-level accreditation; varies by institution and accredited degree Association of College and Research Libraries (ACRL) Standards for Libraries in Higher Education (2004) Written to “provide a comprehensive outline to methodically examine and analyze all library operations, services, and outcomes in the context of accreditation.” Reviewed the most recent Standards documents of the Six Regional Agencies Collected statements related to Teaching and/or Learning …and library’s support of same Identified common themes Reviewed ACRL’s Standards and matched statements to themes from the Agencies’ Standards …then, We reviewed the LIS literature for existing models appropriate for assessing library services re: the themes we identified. What we did… Common Themes: While the six Agencies’ Standards do differ significantly, it was possible for us to identify several shared themes and objectives: I. The Institution is Focused on Student Achievement Subtheme a. Institution supports learners Subtheme b. Institution maintains uniform educational offerings for all students Subtheme c. Institution promotes a life of learning/development of lifelong learners Subtheme d. Faculty are qualified to facilitate and accountable for student success II. Teaching and Learning is a Clear Institutional Priority Subtheme a. Institution provides adequate support and resources for teaching and learning Subtheme b. Institution Recognizes and Promotes the Scholarship of Teaching and Learning (SoTL)* III. The Institution Promotes a “Culture of Assessment” Subtheme a. Assessment findings are employed to improve student success/evidence-informed pedagogy Subtheme b. Assessment efforts are consistent, thorough, and well-documented; standards, tools, and findings are shared Common Themes: While the six Agencies’ Standards do differ significantly, it was possible for us to identify several shared themes and objectives: I. The Institution is Focused on Student Achievement II. Teaching and Learning is a Clear Institutional Priority III. The Institution Promotes a “Culture of Assessment” I. The Institution is Focused on Student Achievement Institution supports learners Accreditation Standards: e.g. “The organization provides an environment that supports all learners and respects the diversity they bring.” ACRL Standard? Instruction: “As an academic or instructional unit within the institution, the library should facilitate student success” Institution maintains uniform educational offerings for all students Accreditation Standards: e.g. “The institution provides programs, wherever offered and however delivered, with appropriate content and rigor that are consistent with its mission; culminate in achievement of clearly identified student learning outcomes; and lead to collegiate-level degrees or certificates with designators consistent with program content in recognized fields of study.” ACRL Standard? Resources: “Furthermore, distance learning programs should be supported by equivalent means such as remote electronic access to collections, the provision of reliable network connections, and electronic transmission or courier delivery of library materials to remote users. I. The Institution is Focused on Student Achievement (cont.) Institution promotes a life of learning/development of lifelong learners Accreditation Standards: e.g. “The organization promotes a life of learning for its faculty, administration, staff, and students by fostering and supporting inquiry, creativity, practice, and social responsibility in ways consistent with its mission.” ACRL Standard? Instruction: “As an academic or instructional unit within the institution, the library should facilitate student success, as well as encourage lifelong learning.” Faculty are qualified to facilitate and accountable for student success Accreditation Standards: e.g. “Responsibilities of teaching faculty include instruction and the systematic understanding of effective teaching/learning processes and outcomes in courses and programs for which they share responsibility” ACRL Standard? Instruction: “In addition, librarians should collaborate frequently with classroom faculty; they should participate in curriculum planning and information literacy instruction as well as educational outcomes assessment.” Staff: “Librarians should have a graduate degree from an ALA-accredited program. All library professionals should be responsible for and participate in professional activities.” Resources: “The library should provide varied, authoritative and up-to-date resources that support its mission and the needs of its users.” II. Teaching and Learning is a Clear Institutional Priority Institution provides adequate support and resources for teaching and learning Accreditation Standards: e.g. “Budgeting priorities reflect that improvement in teaching and learning is a core value of the organization.” ACRL Standard? Services: “Hours of access to the library should be reasonable and convenient for its users.” Staff: “The staff should be sufficient in size and quality to meet the programmatic and service needs of its primary users. Facilities: “The library facility and its branches should be well planned; it should provide secure and adequate space, conducive to study and research with suitable environmental conditions for its services, personnel, resources and collections. The library's equipment should be adequate and functional.” Budget: “The budget should meet the reasonable expectations of library users when balanced against other institutional needs.” II. Teaching and Learning is a Clear Institutional Priority (cont.) Institution Recognizes and Promotes the Scholarship of Teaching and Learning (SoTL)* Accreditation Standards: e.g. “The institution, with significant faculty involvement, engages in ongoing inquiry into the processes of teaching and learning, as well as the conditions and practices that promote the kinds and levels of learning intended by the institution.” ACRL Standard? Instruction: “By combining new techniques and technologies with the best of traditional sources, librarians should assist primary users and others…” Resources (indirect): “The library should provide varied, authoritative and up-to-date resources that support its mission and the needs of its users.” III.The Institution Promotes a “Culture of Assessment” Assessment findings are employed to improve student success/evidenceinformed pedagogy Accreditation Standards: e.g. “The faculty takes responsibility for evaluating the effectiveness of the teaching and learning process and uses the results for improvement.” ACRL Standard? Outcomes Assessment: “…can be an active mechanism for improving current library practices.” Accrediting Commission for Senior Colleges and Universities, Western Association of Schools and Colleges , Standard 4.6 Creating an Organization Committed to Learning and Improvement. III.The Institution Promotes a “Culture of Assessment” (cont.) Assessment efforts are consistent, thorough, and well-documented; standards, tools, and findings are shared Accreditatio n Standards: e.g. “Leadership at all levels is committed to improvement based on the results of the inquiry, evaluation and assessment that is used throughout the institution.” “An accredited institution is expected to possess or demonstrate the following attributes or activities: …a documented, organized, and sustained assessment process to evaluate and improve student learning… evidence that student learning assessment information is shared and discussed with appropriate constituents” ACRL Standard? Outcomes Assessment: “Outcomes assessment will increasingly measure and affect how library goals and objectives are achieved. It will address the accountability of institutions of higher education for student achievement and cost effectiveness.” Instruction: “Librarians should participate in… educational outcomes assessment” Resources: “Policies regarding access should be appropriately disseminated to library users.” Outcomes Assessment (indirect): “The instrument should be valid, and the way it is used should be appropriate for the task. Colleagues at peer institutions may render invaluable assistance by suggesting assessment questions and sample sizes, by sharing lessons learned, and suggesting alternative methods for measuring outcomes.” *Focus: Library Support for Teaching and the Scholarship of Teaching and Learning (SoTL) Concept introduced by Ernest Boyer in Scholarship Reconsidered: Priorities of the Professoriate (1990) Boyer pointed out that faculty are rarely afforded the same respect for excellence in teaching as they are for successful research. Boyer called for abandoning “the old `teaching versus research’ debate” in favor of acknowledging the scholarly aspects of teaching by giving “the familiar and honourable term `scholarship’ a broader, more capacious meaning, one that brings legitimacy to the full scope of academic work.” Lee Shulman credited with expanding and clarifying the dimensions of the concept. Tenets of SoTL: Application of principles and methods of scholarship to the questions, problems and challenges of teaching Serious study of teaching within disciplinary context Empirical investigation Dissemination of findings in same venues appropriate for other discipline-specific research (i.e., not confined to teaching-related publications and conferences) Reflective teaching practice Peer review of teaching effectiveness Continuous assessment and improvement Application of findings from original research, peer assessment, and other modes of study to improving teaching “The SoTL movement provides excellent opportunities for librarians, both in respect to developing their own projects and for supporting, and developing relationships with, faculty working on SoTL projects. Despite this potential, there has been very little written about SoTL in the professional library literature.” SoTL presents a natural fit for academic libraries… …so why have librarians been so slow to pick up on it? Specifically… Application of principles and methods of scholarship to the questions, problems and challenges of teaching Creation of collections to support scholarly inquiry into teaching and learning Hosting events related to SoTL, including workshops and conferences Engaging in SoTL-related research into librarians’ own instruction Reflective teaching practice Providing support and facilities for faculty and other instructors to engage in observation and review of teaching Librarians engaging in peer review of own instruction Continuous assessment and improvement Application of findings from original research, peer assessment, and other modes of study to improving teaching (in classroom and library instructional settings) Next Steps? For Libraries (our suggestions): Conduct in-depth review of relevant regional accreditation standards Identify standards (beyond those that address the library specifically) related to teaching and learning Develop an action plan for efforts to address and support those priorities Communicate efforts to stakeholders in the institution and beyond. For us? Themes discusses in this paper will provide a framework for a series of studies at UTK related to library support for teaching and learning Instructor surveys Instructor interviews/focus groups Experimental study of impact of library instruction; multiple sections of undergraduate required classes Thank you for your time! Questions? [email protected] Note: full references for works cited in presentation are available in our paper.