Toward Developing Measures of the Impact of Library and Information Services

Abstract

A convergence of factors, both within and outside of librarianship, has created an environment conducive to development of what has in past seemed too difficult--measures that will be able to determine impact of library services. These factors include: advances in research that improve evaluation approaches; demands for public sector accountability; and governmental activities aimed at determining service outcomes. They will influence development of a new generation of evaluation tools for librarians and other professionals. This article examines these factors within framework of today's key evaluation questions, What differences do public services make? Using data from a recently completed IMLS-funded study, authors identify and discuss impacts of library community information services as well as implications for development of context-centered evaluation tools. Factors Influencing Changes in Approaches to Public Sector Evaluation Advances in Evaluation Scholarship Researchers are making contributions to a paradigm shift by turning evaluation lens away from institutions and toward people's activities. In process they are beginning to provide public sector professionals with knowledge and skills they need to turn evaluation lens away from institution and toward users of services. Patton's definitive book on evaluation is a landmark contribution to understanding utilization-focused approaches to evaluation. It not only traces development of these approaches, it also synthesizes many social service evaluations and presents approaches to focusing a particular evaluation, evaluation questions, methods, data analysis, and presentation of findings. (1) Patton puts measuring impact, or end results, at top of his hierarchy of evaluation. Table 1 is based on this hierarchy. The Aspen Institute's Roundtable on Comprehensive Community Initiatives produced a rich array of evaluation research, including two major reports on evaluation research and a database of measures for community research (www.aspenroundtable. org). These reports, developed by some of evaluation field's finest researchers, provide a rich store of relevant theoretical and methodological approaches as well as a cogent discussion of challenges and complexities associated with fundamental questions about to ascertain ways in which an investment of resources has paid off. (2) This work clearly details incredible difficulties faced by evaluators who seek to determine outcomes of community services. These researchers stress need for evaluators to ground their work in theory, in particular theory of change, and to incorporate contextual factors. Finally, they recognize that the practice of evaluation is itself a profoundly political and value-laden process, involving judgments about validity of program objectives and choices about progress can be measured. (3) The work of these researchers is vitally important to developing an understanding of complexities of this type of evaluation. In Local Places, Global Connections: Libraries in Digital Age, Schement also articulated importance of understanding context in which citizens seek information and adopt new technologies. He warned that libraries lag in [their] understanding of evolving social context--a context in which libraries will have to justify themselves, and suggested that libraries consider how Americans [will] live their lives as citizens, as economic actors, and as social beings. (4) Digital library researchers have begun to examine social aspects of design, use, and impact of information systems. (5) Bishop and her colleagues argue that combining these approaches with participatory action research ... focuses digital library design and evaluation directly on digital divide. Participatory action research demands relevant outcomes for marginalized members of society. …

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 22 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.