Helpful Information for Meeting Minimum Public Library Standards

Evaluating Effectiveness

Each library...periodically evaluates the effectiveness of the library's programs, services and collections to address community needs, as outlined in the library’s long-range plan of service.

WHAT IS EVALUATION?

Evaluation is the systematic and ongoing assessment of an organization's progress and success in fulfilling its mission, using a variety of qualitative and quantitative measurement techniques. It measures what has already occurred to guide the planning process for the future.

To meet this standard, a library should be able to demonstrate that it queried its community, developed service objectives based on community need, and evaluated the results of those objectives. The library also needs to demonstrate that action was taken to incorporate the results in the library's planning process. It is not enough to set a questionnaire on the circulation desk for the occasional curious library user to pick up. There should be an organized effort to determine community needs, and then to evaluate how well the library is meeting those needs through its collection and services.

WHY IS EVALUATION ESSENTIAL?

In order to meet community needs effectively, ongoing evaluation is necessary. Evaluation provides a concrete measurement of the quality and impact of what the library offers. Some evaluative efforts will be more comprehensive than others, depending on what the library needs to know. Evaluation includes asking current library users how the library is doing, reaching out to the community at large to see what needs are not being met, and exploring why some potential users are not yet involved in the library. Therefore, the data generated can help the library flourish by providing information regarding how to best meet community needs, fulfill the mission, support advocacy efforts, justify budgets/staffing, etc.

WHAT CAN BE EVALUATED?

Most evaluations focus on one or two areas at a time. Some will be more relevant than others, depending on what the library wishes to learn. Although the area traditionally identified as most important to library users is the library's service offerings, other areas may need to be the focus during any particular evaluation cycle. It is helpful to begin by conducting a community needs assessment and clarifying the library’s goals and objectives (refer to the library’s strategic plan). This process can help determine what questions need to be answered.
Possible categories and areas of evaluation include:

  1. Inputs -- staff, materials
  2. Outputs -- circulation, in-house use of materials, visits, programs, internet use, database searches, engagement on the library’s social media sites
  3. Outcomes -- the “why” behind a program or service
  4. Internal processes -- efficiency, staff helpfulness
  5. Community fit -- public opinion
  6. Access to materials -- the speed of delivery, hours, charges, fees
  7. Physical facility -- building appeal, parking, location
  8. Management elements -- both board and staff activities
  9. Service offerings -- range, variety
  10. Service to special groups -- youth, homebound, aged, people with disabilities, unemployed or underemployed, etc.

WHAT IS THE PROCESS FOR EVALUATING EFFECTIVENESS?

After a library has decided that evaluation is an important part of meeting its mission, goals and objectives, the process of choosing the most effective evaluation tool begins. Refer to Library Research Service and WebJunction courses on assessment, evaluation and planning for helpful information.
Questions a library might ask to facilitate the assessment process include:

  • What does the library wish to evaluate, and why?
  • What level of performance does the library want to achieve?
  • Which investigative technique will work best?
  • How will the library actually measure the performance level?
  • Did the library do what was promised; to what extent did the library meet the objectives?
  • What data does the library already collect or have access to?
  • What is the timeline for the evaluation?

There are three categories of evaluation tools:

  • Quantitative measures - measurements that produce numerical results. Statistical packages are available to assist libraries. Refer to Librarian and Researcher Knowledge Space for more information.
  • Qualitative measures - the data are not in the form of numbers.
    • Focus group/Individual interviews: Interviews and focus groups are conducted with evaluation and program/initiative stakeholders. These include, but are not limited to, staff, administrators, participants and their parents or families, funders, and community members. Interviews and focus groups can be conducted in person or over the phone. Questions posed in interviews and focus groups are generally open-ended and responses are documented in full, through detailed note-taking or transcription. The purpose of interviews and focus groups is to gather detailed descriptions, from a purposeful sample of stakeholders, of the program processes and the stakeholders' opinions of those processes. (www.imls.gov)
    • Observation: Observation is an unobtrusive method for gathering information about how the program/initiative operates. Observations can be highly structured, with protocols for recording specific behaviors at specific times, or unstructured, taking a more casual, "look-and-see" approach to understanding the day-to-day operation of the program. Data from observations are used to supplement interviews and surveys in order to complete the description of the program/initiative and to verify information gathered through other methods. (www.imls.gov)
    • Evaluations: For example, evaluation of a meeting or workshop may involve a brief discussion at the end of the meeting or be a more formal written evaluation by participants immediately following the event. Meaningful long-term change may involve follow-up interviews at a later date.
  • Use Surveys of current and potential library users
    Surveys and questionnaires are also conducted with evaluation and program/initiative stakeholders. These are usually administered on paper, through the mail, in a highly structured interview process in which respondents are asked to choose answers from those predetermined on the survey, or more recently, through email and on the Web. The purpose of surveys/questionnaires is to gather specific information—often regarding opinions or levels of satisfaction, in addition to demographic information—from a large, representative sample. (www.imls.gov)
    Helpful resources include:

WHAT SHOULD BE DONE WITH THE FINDINGS?

The findings need to be analyzed and discussed by the full board and recommendations should be made to achieve a higher level of effectiveness. As with other parts of any planning process, the evaluation results should be widely disseminated and used as a chance to gain publicity and generate support for any proposals. The following tools are very useful when distributing information regarding the value of library services:

  1. Valuing Library Services Calculator – determine the value of resources and services
  2. CBA/ROI Calculator - benefit for every dollar spent calculation

Responsibilities for board-approved changes should be clearly laid out, timelines set, and tasks completed. Evaluating effectiveness does not end here, as it needs to be an ongoing process.

WHERE CAN THE LIBRARY GO FOR HELP?

Consult the library system for further assistance. In addition to staff expertise, there are a number of manuals and other materials which the system can share with the library.

<Written Budget | Back to Contents | Hours>

Last Updated: May 24, 2019