State Library Administrative Agency: New York State Library
Evaluator: Lauren Polvere, Ph.D., Policy Strategy Consultants, LLC
Evaluation Commissioned by: New York State Library, New York State Education Department, The University of the State of New York
Date: 3/29/22
Betty A. Rosa
Sharon Cates-Williams
Mark Schaming
Lauren Moore
The New York State Education Department does not discriminate on the basis of age, color, religion, creed, disability, marital status, veteran status, national origin, race, gender, genetic predisposition or carrier status, or sexual orientation in its educational programs, services, and activities. Portions of this publication can be made available in a variety of formats, including braille, large print, or audio tape, upon request. Inquiries concerning this policy of nondiscrimination should be directed to the Department’s Office for Diversity, Ethics and Access, Room 530, Education Building, Albany, New York 12234.
The New York State Library (NYSL) works in partnership with public libraries and public library systems, reference and research library resources systems, and school library systems to carry out its mission, which is “to provide leadership and guidance for the planning and coordinated development of library services and to serve as a reference and research library for the people of the State.”[1] New York’s 2017–2022 Library Services and Technology Act (LSTA) Plan was created to support this mission through a range of activities implemented through sixteen key projects.
The NYSL commissioned an independent evaluation of its 2017–2022 LSTA Five-Year Plan; the evaluation was conducted by the evaluation firm Policy Strategy Consultants, LLC. The firm conducted a mixed methods evaluation that was designed to highlight effective past practices; assess the efficacy of activity implementation used to advance the state goals; and develop key findings and recommendations to inform the next Five-Year Plan.
A-1. To what extent did your Five-Year Plan activities make progress toward each goal? Where progress was not achieved as anticipated, discuss what factors contributed.
Goal 1 was partially achieved.
There is evidence of notable achievements and commendable progress regarding Goal 1. For example, through the NOVELny project, New Yorkers are provided with access to high-quality information through a suite of databases. Library stakeholders provided positive feedback about the databases, and those trained in their use indicated feelings of confidence regarding their ability to implement what they learned. Further evidence is provided through the Making New York History Materials Accessible project, where targets were met or exceeded regarding making finding aids and collection guides discoverable online. There is additional evidence of strong Documentary Heritage and Preservation Services for New York (DHPSNY) program participation in the most recent years assessed, indicating that the NYSL is participating in collaborative resource-sharing projects.
This goal was determined to have been partially achieved, as some project targets were unmet. For example, target training numbers, metadata for uncataloged collections, and the percentage of libraries with disaster recovery goals fell short of the stated targets.
Through key informant interviews, State Library project managers and staff indicated barriers to implementation of various projects. The impact of the COVID-19 pandemic was noted as a significant barrier related to almost all LSTA-funded projects. Staffing reductions (along with the time-consuming nature of the work) were a barrier to cataloging government documents and upgrading records. Outreach to the public regarding NOVELny was another reported challenge, as the NYSL is reliant on individual libraries to reach patrons.
Goal 2 was achieved.
The NYSL had notable achievements regarding Goal 2. For example, the Data for Decision Making project acquired Bibliostat CollectConnect, and over 1,000 libraries and library systems used this product. It includes a data portal that allows public libraries to conduct peer comparisons with other public libraries on important metrics, such as library expenses and director salaries. Users indicated positive feedback that the data collection resource is meeting library needs and improving services to the public. Through the Building Strong Library Leaders project, there is evidence of strong participation by the library workforce. Attendees report benefitting from webinars and self-paced courses, with a high percentage of attendees indicating that they are learning new information, are increasing their confidence, and are likely to apply what they have learned. Goal 2 was determined to have been achieved, as the stated targets were all met.
Key informant interview data suggested that factors that challenged project implementation included difficulty “getting the word out” to the field regarding requirements and professional development opportunities, including public librarian certification renewals and free WebJunction courses.
Goal 3 was partially achieved.
There is evidence of progress and achievements regarding Goal 3. Public libraries are collaborating with schools and school library systems to promote the Summer Reading project, and there was evidence of strong participation by children and teens prior to the pandemic. Library stakeholders provided positive feedback about the materials provided by the NYSL to promote the program. In addition, library stakeholders who received training through the Ready to Read early literacy program indicated through surveys that they were more confident in what they learned and that they are likely to apply it. It should be noted that all of these programs were affected by the pandemic.
Goal 3 was determined to have been partially achieved, as several project targets, including the number of public libraries offering teen-led activities, the number of public classes and programs held, and the number of public libraries involved in local collaborations to enhance early childhood school readiness were not met.
Data from key informant interviews highlighted factors that hindered project progress, including impacts from the COVID-19 pandemic. For example, as a result of the pandemic public library systems and public libraries could not deliver adult literacy and ESL programs in person, which created implementation challenges for the Targeting Library Services for Outreach project. State Library staff associated with the Ready to Read and Transforming Teen Services projects reported a need for more trainers, as the current demand cannot be met.
Goal 4 was partially achieved.
The NYSL made progress toward Goal 4, as evidenced by the Research Library’s Training and Outreach project. Through this project, State Library staff are providing public classes and programs, exceeding the target numbers set. These programs have been well-received by attendees, based on survey data. Through the Expanding Library Networking and Collaboration project, the NYSL is providing support to school and public library systems through conference calls, face-to-face meetings, virtual meetings, site visits, presentations, orientations, written materials, webinars, and instruction.
Goal 4 was determined to have been partially achieved, as some progress fell short of stated targets. For instance, the NYSL website experienced an increase in users, sessions, and page views each year assessed but did not meet the specified target. There has been an increase in public libraries with access to minimum broadband speeds of 100 mbps, though targets were not quite met. As discussed above, inadequate staffing was reported to be a significant factor that challenged project implementation.
Question A-2. To what extent did your Five-Year Plan activities achieve results that address national priorities associated with Measuring Success focal areas and corresponding intents?
The Five-Year Plan activities achieved results that address several of the focal areas and their intents, particularly lifelong learning, information access, and institutional capacity.
Question A-3. Did any of the following groups[2] represent a substantial focus for your Five-Year Plan activities?
Most LSTA-funded projects serve a general audience. Two projects target more specific audiences[3] but do not meet the “substantial focus” threshold.
Question B-1. How have you used any data from the State Program Reports (SPR) and elsewhere (e.g., Public Libraries Survey) to guide activities included in the Five-Year Plan?
SPR data are discussed by State Library staff internally, typically in an informal manner. State Library staff reported that the data collection process is time consuming, resulting in limited time for dissemination and discussion. Some expressed interest in collecting data that assess project impacts and provide additional context.
Question B-2. Specify any modifications you made to the Five-Year Plan / reason for this change?
There were no modifications to the Five-Year Plan.
Question B-3. How/with whom have you shared data from the SPR and from other evaluation resources? How have you used the last Five-Year Evaluation to inform data collected for the new Five-Year Evaluation? How have you used this information throughout this five-year cycle?
In addition to the SPR data, some projects were evaluated externally to assess impact and to improve implementation. Findings were shared with the leadership and with staff involved in the relevant projects. The previous Five-Year Plan and evaluation served as a starting point for developing the 2017–2022 LSTA Five-Year Plan. A process of stakeholder engagement then followed to complete the plan.
Questions C-1 and C-2: Identify how you implemented an independent Five-Year Evaluation using the criteria described in the section, “Selection of an Independent Evaluator.” Describe the types of statistical and qualitative methods used in conducting the Five-Year Evaluation. Assess their validity and reliability.
Policy Strategy Consultants (PSC)[4] led an objective external evaluation with methodological rigor. PSC used a mixed methods approach to the evaluation, which included:
Two evaluators collaboratively coded and analyzed the qualitative data to ensure inter-rater reliability. Validity was strengthened through the mixed methods approach to the evaluation, in which findings were triangulated (compared, contrasted, combined, and synthesized) across the above-mentioned data sources.
Question C-3. Describe the stakeholders involved in the various stages of the Five-Year Evaluation. How did you engage them?
Using a participatory approach, the evaluator met with an advisory committee of NYSL leaders/staff to obtain contextual information needed for the evaluation, to develop methods, and to refine the data collection instruments. Administrators and staff associated with key projects were engaged through key informant interviews. A broad array of library stakeholders were engaged through the library stakeholder survey, in which they had the opportunity to provide feedback to inform the next Five-Year Plan.
Question C-4. Discuss how you will share the key findings and recommendations with others.
This report will be submitted to IMLS and will also be disseminated to the Regents Advisory Council on Libraries (RAC), State Library staff members, library systems, and other stakeholders. This report will be posted on the State Library’s website.
The library stakeholder survey asked respondents to provide feedback on areas of greatest priority, drawing from the findings of a recent Regents Advisory Council on Libraries (RAC) needs assessment and the current Five-Year Plan. The majority of survey respondents indicated that the following areas from the needs assessment are a high priority: promoting diversity, equity, and inclusion; supporting/promoting libraries and their services; planning for the future of libraries and the profession; building a diverse, well-trained, and sustainable workforce; addressing social and economic issues related to libraries and their communities; and providing training and support regarding information and media literacy. The respondents endorsed the following areas as high priority, organized by the goals of the current Five-Year Plan:
Based on the evaluation findings, the following recommendations are offered:
Question A-1. To what extent did your Five-Year Plan activities make progress towards each goal? Where progress was not achieved as anticipated, discuss what factors (e.g., staffing, budget, over-ambitious goals, partners) contributed?
Goal 1 was partially achieved.
The New York State Library’s LSTA Five-Year Plan (2017–2022) describes activities aligned with Goal 1, which include supporting statewide access to e-resources, as well as training to support e-resource use; expanding public access to digital holdings; enhancing resource sharing and statewide delivery of materials; improving and sustaining high-speed broadband; developing opportunities for libraries to provide virtual trainings and education; expanding access to public libraries to reduce the number of unserved New Yorkers; and strengthening partnerships to ensure that New York State students have access to school libraries and certified school librarians (see Appendix E for a full list of Goal 1 activities).
To support the goal of improving access to library resources for all New Yorkers, the New York State Library implements five key projects: NOVELny; Making New York History Materials Accessible; Discovery Services; Partnerships in Resource Sharing and Access; and Digitization and Preservation Program. This section provides an overview for each of these projects, followed by an assessment of progress toward the output or outcome targets set by the New York State Library. Finally, interview findings regarding implementation barriers are detailed.
NOVELny. Project targets were partially achieved.
Project Overview. NOVELny provides New York State residents with free access to electronic information and resources through licensing and subscriptions. NOVELny is designed to support the goal of improving access to library resources for all New Yorkers in a manner that promotes equity. Findings from a recent external evaluation report on NOVELny[5] indicated that respondents of library organizations valued NOVELny as “a source of high quality, comprehensive, and trusted information.” Respondents further indicated that New Yorkers are benefitting from improved access to information because of the project.
State Library project managers who oversee NOVELny were interviewed for the evaluation. They described creating and promoting webinars to help staff and customers use the tools as efficiently as possible. Surveys are collected from attendees of the webinars to evaluate the effectiveness of these trainings. State Library project managers also reported that outreach through social media (i.e., NOVELny Facebook page) has been a strong focus. Attention has been focused on revamping practices and procedures to strengthen the project’s social media presence. During the period assessed, staff performed an extensive update of the website, promoting accessibility and making it more user-friendly for library patrons. State Library staff operate a help desk to answer questions from library and library system staff regarding use of the databases.
Progress Toward Project Targets. The first output target for NOVELny stated that at least 2,500 library staff and end-users would participate in NOVELny e-resource trainings between 2017 and 2022. State Program Report (SPR) data indicate that in FFY[6] 2017, 276 staff attended vendor webinars and 213 staff attended in-person workshops. In FFY 2018, 306 library staff attended webinars. In FFY 2019, 192 librarians attended vendor webinars. And finally, in FFY 2020, 509 staff attended vendor webinars and customized trainings. In total, 1,496 staff attended these trainings, falling short of the target.
A second target indicated that, by 2022, New Yorkers would conduct forty million searches using databases delivered to New Yorkers through statewide licenses. The reporting period for this target is ongoing, but to date, this target was met in one of the years evaluated (FFY 2018). The number of searches ranged from 27.9 million searches in FFY 2020 to 44.7 million in FFY 2018. Per the NYSL, it should be noted that the State Library discovered that school librarians are very important to promoting use of the databases. During the pandemic when students were home, they were not exposed to school librarians and their teachers were focused on teaching virtually, which may have led usage to decrease.
The final target for NOVELny indicated that at least 50% of library and library system staff who attended a training session on products of the statewide database program would indicate feeling confident about using what they learned. This target was exceeded in each year evaluated, with between 83% of respondents (FFY 2017) and 100% of respondents[7] (FFY 2018) indicating agreement or strong agreement with the survey item “I feel more confident about what I just learned.”
Implementation Barriers. State Library staff affiliated with NOVELny indicated that it is challenging to reach New Yorkers. Most recently, they have focused particularly on outreach through social media. In terms of direct outreach, the State Library staff are reliant on libraries and library systems to reach their patrons, which is a challenge.
Additional implementation barriers were noted in the NOVELny external evaluation report. The evaluation findings suggested that public library staff use NOVELny databases less frequently than do school and academic libraries. Public library staff also promoted these databases to library patrons less frequently. Public library staff expressed the view that public library users lack awareness of NOVELny resources, and perceived that database content does not align to the needs of their users.
Making New York History Materials Accessible. Project targets were achieved.
Project Overview. The Research Library’s Making New York History Materials Accessible Project[8] is designed to strengthen access to New York State historical and genealogical materials for New Yorkers by improving access to these materials both on site and remotely. Key project activities include creating finding aids, cataloging uncataloged items, acquiring and processing significant collections, improving existing catalog records, and rehousing materials, as needed.
The project focuses on cataloging rare books, manuscripts, maps, prints, broadsides, scores, and more, and creating or updating bibliographic information to improve accessibility of these resources. In addition, according to the State Library staff interviewed, special exhibits are developed regularly to highlight collections, both in displays on the public floor and in virtual displays on the New York State Library website. Most virtual displays are archived and thus remain available to the public through the website.
Progress Toward Project Targets. A target for this project indicated that twenty new or revised finding aids or collection guides would be made available online annually during 2017 through 2022. For the years in which data were available, the project met or exceeded this target. Specifically, there were 29 finding aids made available in FFY2017, 20 in FFY2018, 31 in FFY2019, and 68 in FFY2020. The greatest number of collections were made discoverable during FFY 2017 (33 collections), and the greatest number of items were made discoverable during FFY 2018 (392 items).
Discovery Services. Project targets were not achieved.
Project Overview. The Research Library’s Discovery Services project is designed to provide access to New York State Library collections through several activities, including creating metadata records, creating special exhibits, partnering with the LC/NACO (Name Authority Cooperative) program, and cataloging New York State government documents.
Progress Toward Project Targets. The target for Discovery Services indicated that the State Library will provide electronic metadata for three uncataloged collections by 2021. This target was not met, as zero metadata plans or frameworks were produced or updated each year. However, of note, the project resulted in a substantial number of items being made discoverable to the public, ranging from 3,224 items in FFY 2017 to 5,538 items in FFY 2020.
Implementation Barriers. State Library project managers interviewed about the Discovery Services project noted that cataloging takes a long time due to the nature of the items worked on. This project includes a significant number of New York State government documents, which are cataloged at the highest national cataloging standards to make the records useful to other libraries.
The State Library staff also described the substantial time and effort required to add documents to OCLC. They explained that the New York State Library catalog is among the first online catalogs created. Not all records in this catalog are in OCLC. Staff enhance these old, minimal records and add the Library’s holdings to OCLC, which is time consuming, but important to enhance access.
Cataloging work is time-consuming and is further challenged by a lack of staffing due to financial constraints, according to State Library project managers. Priorities for cataloging include state government documents and new purchases.
Partnerships in Resource Sharing and Access. Project targets were partially achieved.
Project Overview. The Partnerships in Resource Sharing and Access project is designed to offer leadership, planning, coordination, consultation, and support services to enable library systems and libraries to improve and expand upon resource sharing and partnerships. Through the project, State Library staff work with the nine regional library councils and also provide state funding to the ten largest research libraries in the state, in addition to competitive discretionary grants for smaller libraries and cultural institutions for the conservation and preservation of unique library research materials. The goal of the state preservation competitive grant funding, according to a State Library project manager, is to advance archiving and preservation, particularly for smaller institutions. A State Library staff member interviewed explained that this is key to accessibility, noting, “If a collection isn’t preserved, it isn’t accessible.” In addition, the project supports digitization projects to promote access for New Yorkers, and offers Documentary Heritage and Preservation Services for New York (DHPSNY) webinars and learning opportunities.
Progress Toward Project Targets. The first target defined for Partnerships in Resource Sharing and Access stated that DHPSNY program participation will increase annually by five percent, over FFY 2017 levels. In FFY 2017, there were 869 attendees. This target was partially met. FFY 2018 showed a decrease to 824 attendees. However, the target was exceeded in FFY 2019 and FFY 2020. In FFY 2019, there were 982 attendees, which is a 13% increase over the 2017 level. In FFY 2020, there was a substantial increase, with 1,248 attendees, representing a 43% increase over the 2017 level. Thus, this project shows impressive progress in the most recent years assessed.
The second target defined for this project indicated that by 2022 the percentage of libraries with disaster recovery plans would increase from 47% (based on 2009 survey results) to 60%. The reporting period for this target is ongoing, and data were limited. Survey data provided by the New York State Library indicated that 39% of respondents (n = 44) had a disaster recovery plan in 2018, falling short of the target. However, of note, there was an increase in disaster recovery plans, per public and association library annual report data provided by the NYSL. In 2017, 586 public libraries had disaster plans; in 2020, 641 public libraries had disaster plans.
The final target associated with this project indicated that the State Library will participate in at least three collaborative resource-sharing projects by 2022. Based on data provided by the New York State Library, the Division of Library Development supported 49 partnerships for collaborative collection development, which included school library systems and 3R systems (Reference and Research Library Resources Councils). The Research Library has three partnerships related to interlibrary loan, which include the Empire State Library Network, the Information Delivery Services Project, and the National Library of Medicine/DOCLINE. Thus, this target has been exceeded.
Implementation Barriers. The State Library staff interviewed indicated that state processes for competitive grants can create barriers for applicants, particularly for small institutions.
Digitization and Preservation Program. Project targets were partially achieved.
Project Overview. The Research Library’s Digitization and Preservation Program is designed to preserve documents related to New York history and governance, and to make these documents accessible. A significant focus is directed toward acquisition and preservation of digital items to promote access in the long term. According to State Library staff affiliated with the project, staff build relationships with state agencies through the acquisition process. Specific focus is placed on harvesting, which entails electronically collecting the items to preserve.
The State Library staff interviewed indicated that the COVID-19 pandemic increased the need for remote services for users and created opportunities for staff to respond to that need. When New Yorkers including Library staff were directed to work from home, the type of work and the manner in which work was being done shifted. Library staff saw this as an opportunity and worked with other units within the library to identify collections of high-value items, such as legislative bill jackets. Staff then worked to upload these items to the digital collections for remote access.
Progress Toward Project Targets. The target defined for this project stated that by 2021 the State Library would increase the number of state agencies contributing to the New York State Library’s digital collections by 20%. A baseline number of agencies contributing to digital collections was not available, and thus this target could not be assessed. State Library staff note that approximately five state agencies are currently directly contributing to digital collections. State Library staff also harvest electronic items from the websites of other agencies. SPR data provide evidence of a substantial number of items being acquired and digitized in some of the years assessed. The number of items acquired ranged from 4,966 in FFY 2019 to 15,034 in FFY 2017. The number of items digitized ranged from 59 in FFY 2019[9] to 62,849 in FFY 2017.
Implementation Barriers. State Library staff interviewed from the Digitization and Preservation Program suggested a need to strengthen engagement with state agencies, noting that state agency staff may be unaware of the value that can be provided to them.
The State Library staff also described how the strict IT infrastructure can be a barrier to the implementation of key activities. Due to firewalls and other IT security mechanisms, staff are sometimes unable to share documents. At present, the State Library staff do not have access to a file transfer system.
Other Progress Related to Goal 1. An activity aligned with Goal 1 is to support increasing the number of certified school librarians. There were modest changes in the number of certified school librarians employed in public and charter schools. Per data provided by the NYSL, there were 2,011 certified school librarians in 2017, and the number increased to 2,087 certified school librarians in 2020.
Another target related to reaching unserved populations. This target stated that by 2021, an additional 75,000 formerly unserved New Yorkers will benefit from having a local library in their community. This target was not reached, but progress is evident. Compared to 2017, as of July 2021, approximately 41,800 formerly unserved New Yorkers now have a local library in their community.
Library Stakeholder Survey Findings Related to Goal 1
According to the findings of the library stakeholder survey (n = 494), the components of Goal 1 endorsed most strongly by the respondents included every New Yorker having access to a public library and every student having access to a school librarian. Ninety-one percent (91%) of survey respondents indicated that every New Yorker having access to a public library is a high priority, and 81% of respondents indicated that supporting the goal of every student having access to a school librarian is a high priority.
Goal 2 was achieved.
Activities associated with Goal 2 include educating library stakeholders about innovative models of public library governance, as well as exemplary library programs and best practices; expanding partnerships with state agencies; assisting library staff in measuring progress and impact to foster improvement; strengthening partnerships to support services of New York’s school, academic, and special libraries; strengthening partnerships to support workforce development, lifelong learning, and literacy programs and services of New York’s public libraries; expanding conservation and preservation; and supporting the development of disaster recovery plans (see Appendix E for a full list of Goal 2 activities).
The New York State Library supports two key projects that align with the goal of delivering and improving library programs: Data for Decision Making and Building Strong Library Leaders.
Data for Decision Making. Project targets were achieved.
Project Overview. The Data for Decision Making project is focused on the provision of technical resources and staffing capacity related to the use of data. Libraries and library systems provide data, which is collected and synthesized by the New York State Library. The goal of the project is to ensure that libraries and library systems in New York State and other key stakeholders have data and information necessary for making sound decisions about library services.
The New York State Library, in partnership with library systems, assisted 756 public libraries with over 1,000 outlets in collecting and reporting a wide range of data elements. According to State Library staff associated with the project, significant time and energy is focused on developing an annual report for the Public and Association Libraries, which includes developing items for data collection and training library systems regarding the new items, including through webinars and answers to ongoing question submissions. Once the report is prepared, a contractor, Baker & Taylor, makes the report available through a portal, Bibliostat CollectConnect. The State Library staff work with public library systems to ensure that the data is accurate before making results public.
During the assessment period, the project continued to use Bibliostat CollectConnect, and over 1,000 libraries used the product for data and reporting. According to State Library staff associated with the project, the Bibliostat CollectConnect portal feature is a particularly useful tool that is enabling public libraries to do peer comparisons with libraries from across the state on important issues, such as library budgets and director salaries, as well as to review data over the years (i.e., changes in circulation or attendance).
Progress Toward Project Targets. The first output target was focused on user satisfaction, including satisfaction with the Bibliostat CollectConnect software.[10] The project administered a survey to assess library stakeholders’ satisfaction.[11] Most respondents indicated satisfaction with the data collection piece of Bibliostat CollectConnect, with between 54% (FFY 2017) and 68% (FFY 2019) of respondents agreeing or strongly agreeing with the statement “I am satisfied that the resource is meeting library needs.” In response to a survey item that stated, “Applying the resource will help improve library services to the public,” between 48% (FFY 2017) and 62% (FFY 2020) of participants indicated agreement or strong agreement.
Implementation Barriers. The State Library staff interviewed indicated that the COVID-19 pandemic was a barrier to project progress, which resulted in a need to “shift gears quickly and dynamically.” However, they described making changes to practices and policies that enabled them to more quickly add federally required survey items that reflected the impact of COVID-19, such as library closings and impacts to services.
Building Strong Library Leaders. Project targets were achieved.
Project Overview. To support New York libraries and library systems, the purpose of the Building Strong Library Leaders project is to enhance the skills and knowledge of the library workforce and library trustees to support leadership development. This project provides access to free training and webinars (i.e., WebJunction’s self-paced courses) for continuing education of library leaders in partnership with OCLC, and also provides webinars hosted or sponsored by the State Library. WebJunction topics include technology, serving older adults, rural libraries, customer service, library funding, and more. The State Library staff interviewed described developing training topics for New York State Library webinars based on input and recommendations from partners in the field, and building relationships with experts in the library field who can provide webinars.
According to the State Library staff, librarians who attend courses are sharing learnings with colleagues and are encouraging others to attend, allowing for word-of-mouth referrals that extend the reach of the project.
In addition, the project provides instruction and assistance to library stakeholders regarding professional development requirements, including New York State Public Librarian Certification, and the project provides support for library system continuing education offerings to library staff and trustees.
Progress Toward Project Targets. The first target set indicated that there would be a 15% increase in participation in State Library and library system supported training and professional development activities compared with 2015 levels. According to data provided by the NYSL, in 2015–2016, 7,320 sessions were held, with 99,297 total attendees. In the years that followed, the number of sessions ranged from 7,030 in 2016–2017 to 7,891 in 2019–2020. The number of attendees ranged from 96,972 in 2017 to a high of 224,358 in 2019–2020. Thus, the increase in attendees well exceeded the 15% increase set.
A second target stated that 60% of library staff, library system staff, and library trustees participating in training hosted or sponsored by the State Library would indicate through survey responses that they learned something by participating in the training activities, that they are confident about using what they learned, that they are likely to apply what they have learned to improve library services to the public, and that they are better able to anticipate and meet changing customer needs and to measure progress toward achieving service excellence.[12] This target was exceeded, with more than 60% of survey respondents indicating agreement or strong agreement with each of these survey items. For example, between 70% (FFY 2018) and 96% of respondents (FFY 2020) agreed or strongly agreed with the statement, “I learned something by participating in this library activity.”
Implementation Barriers. State Library staff associated with the Building Strong Library Leaders project described factors that challenged project progress, particularly regarding outreach and communication. The State Library staff reported that effective outreach regarding WebJunction trainings and the certification process has been a challenge.
The State Library project staff suggested that relationships with partners might improve outreach, information sharing, and communication moving forward. Since the start of the pandemic, NYSL staff reported that there have been more opportunities to share information via training calendars and on social media.
Library Stakeholder Survey Findings Related to Goal 2
According to the findings of the library stakeholder survey, the Goal 2 activity endorsed most strongly by the respondents centered on expanding and enhancing workforce development, lifelong learning, and literacy programs in New York’s libraries. Fifty-four percent (54%) of respondents reported that this activity is a high priority.
Goal 3 was partially achieved.
Activities associated with Goal 3 include assessing needs for building construction, expansion, and renovation; supporting access to high-quality digital literacy training programs; ensuring that library staff and trustees are highly skilled in new technologies; supporting libraries to offer community learning opportunities; increasing awareness and participation in Summer Reading at New York Libraries; providing access to research-based early literacy skills training for library staff; building capacity to enhance early childhood outcomes through Ready to Read at New York Libraries and by strengthening partnerships to support families in building early literacy skills; supporting the development and expansion of teen-led activities; increasing awareness and use of the digital talking book program and the Talking Book and Braille Library; and by expanding programming through partnerships that will provide libraries with timely and free access to exhibit materials and online resources. See Appendix E for a full list of Goal 3 activities.
To support the goal of influencing New Yorkers to view libraries as community learning spaces that offer educational opportunities in several domains, the following key projects are implemented by the New York State Library: Summer Reading at New York Libraries; Targeting Library Services to Meet Changing Needs for Literacy, Education, and Outreach; and Ready to Read at New York Libraries.
Summer Reading at New York Libraries. Project targets were partially achieved.
Project Overview. The Summer Reading at New York Libraries program is a statewide initiative to support public libraries by promoting summer learning and reading for children. As stated on the project website:[13]
This free, annual program is supported by the State Library in conjunction with partners like the State Assembly and Senate, Hunger Solutions of New York, the New York State Reading Association, 4-H, New York Council for the Humanities, the New York State Alliance of Boys and Girls Club Inc., and the School Library Systems Association of New York State. The program gives children the opportunity to access the vast resources of New York’s public libraries to support their summer reading.
The project also supports the 756 public libraries by providing manuals and other materials, and by maintaining a website that provides flyers and promotional items that libraries can use. The project provides online software to libraries to help with registration and promotion, and offers a summer reading website for parents and educators. The project staff consult with public library system youth services staff, who advise and promote the projects. The library system consultants work with the libraries and come up with creative ways to encourage children and families to come to the library to learn new skills and to attend programs.
Progress Toward Project Targets. The first target associated with the Summer Reading project stated that by 2021, 80% of the public libraries in New York State will have collaborated with one or more P–12 schools and/or school library systems in promoting student participation through the project. Based on data provided by the New York State Library, between 2017 and 2020, between 300 and just over 500 public libraries partnered with public schools/BOCES (492 public libraries in 2017, 501 in 2018, 506 in 2019, and 345 in 2020). Approximately 100 to 200 public libraries partnered with non-public schools (197 public libraries in 2017, 198 in 2018, 203 in 2019, and 109 in 2020). This target was not met, as fewer than 80% of public libraries had these partnerships and collaborations.[14]
A second target reflected project participation, stating that by 2021, the project would report an annual participation level of 2.5 million children and teens. Overall, this target was met. It was met in FFY 2018, when 2.5 million children and teens participated, marking an increase of 134,000 over the previous year. In the remaining years assessed, the target was not met (FFY 2017, 2.4 million; FFY 2019, 1.3 million; FFY 2020, 1.1 million). The participation levels in FFY 2019 and FFY 2020 were impacted by COVID, which is discussed in more depth below.
A third target noted that by 2021, the number of public libraries offering teen-led activities would increase by 20%. Baseline data were not available prior to FFY 2017. Using FFY 2017 as the baseline, 212 public libraries had teen-led activities. The target was met in FFY 2018, as there was a 28% increase in the number of public libraries offering teen-led activities (from 212 to 271 public libraries). However, the participation level decreased to 201 public libraries in FFY 2019. New York State Library staff indicated that this decrease may have been due to COVID, as teens were unable to come into the library for programming.
The final target for this project stated that by 2020, 75% of public library and library system staff will indicate through surveys that they are better equipped to provide strong summer reading programs for their communities, and that they use materials provided by the New York State Library for this purpose. Available data did not address the first component of the target (staff indicating that they are better equipped to provide strong summer reading programs). However, survey data [[15]] were available to evaluate use of materials provided by the New York State Library, including manuals and posters, as well as flyers and informational materials. Regarding manuals and posters, between 82% (FFY 2017) and 90% of respondents (FFY 2019) strongly agreed or agreed that these materials are meeting library needs, exceeding the target. Data regarding manuals and posters were not available for FFY 2020. Regarding flyers and informational materials, between 64% (FFY 2018) and 79% of respondents strongly agreed or agreed that these materials are meeting library needs. This target was met in each fiscal year in which data were available, with the exception of FFY 2018.
Implementation Barriers. State Library staff interviewed from the Summer Reading at New York Libraries project reported that as a result of the pandemic, libraries reported a decrease in the number of families participating in the Summer Reading program. However, during the pandemic, the New York State Library increased Summer Reading program collaboration with school libraries, resulting in more outreach and deepened partnerships. In addition, local library staff demonstrated creativity by moving story times outdoors, by developing outdoor “story walks” for children, and by offering a wide variety of virtual offerings.
Information about the Summer Reading program (as well as the Ready to Read program, discussed below) is primarily disseminated through the public library system youth services consultants. State Library staff reported that the New York’s Libraries Information Network (NYLINE) list is too general to reach the programs’ target audience of public library staff. Since the programs do not communicate directly with individual libraries, it is difficult to assess whether information is reaching the library staff.
Targeting Library Services to Meet Changing Needs for Literacy, Education, and Outreach. Project targets were not met or could not be assessed due to insufficient data.
Project Overview. The Targeting Library Services Outreach project provides leadership, planning coordination, consultation, and support services to enable library systems and libraries to reach out to individuals from diverse backgrounds, as well as those who experience difficulty using libraries and library services. According to the SPRs, the State Library supports access for Native Americans, incarcerated people, people with disabilities, the elderly, those who speak languages other than English, the unemployed and underemployed, adults who are illiterate, those who are geographically isolated, and others who experience difficulties using a library.
The project consists of several sub-projects that are implemented by the public library systems, including adult literacy[16] and outreach projects. The adult literacy program works with adults (age 16 and older) who are no longer in school, focusing on English literacy and workforce skills. The outreach services programs provide library programming, services, and resources to incarcerated individuals and those in congregate settings, including nursing homes and group homes.
Progress Toward Project Targets. There were no specific targets set for this project, and thus progress cannot be evaluated. However, data is available on metrics associated with the project. Between 1 and 4 presentations were given each year between FFY 2017 and 2020, with 23-53 participants in attendance. In FFY 2020, 305 consultations/reference transactions were provided.
Each year, the State Library supported outreach and adult literacy services through 23 coordinated outreach programs, 23 adult literacy/workforce development programs, 22 library services to county jail programs, and 17 library services to State correctional facilities programs. Also, the State Library supported outreach and adult literacy services through state-funded programs, and provided funding for two public libraries supporting Native Americans to improve access.
Implementation Barriers. State Library staff associated with the Targeting Library Services Outreach project were interviewed and described barriers to project implementation. The State Library staff reported that teaching adult literacy, English as a Second Language (ESL) and interview preparation courses remotely through the State Library–funded Adult Literacy program has been a challenge for public library systems during the pandemic.
The State Library staff noted that much has been done to reach underserved groups, but expressed interest in reaching more subgroups. The State Library staff would like to assist libraries and library systems in reaching more individuals for whom English is a second language, in addition to those who are geographically isolated and those who need digital literacy training.
Ready to Read at New York Libraries. Project targets were partially met.
Project Overview. Ready to Read is a coordinated statewide early literacy program. According to the program website, the program is designed to “improve and expand the availability of high-quality public library early learning services in local communities across the State.”[17] The program promotes early literacy programs and professional development for library staff.
According to State Library staff interviewed, the program coordinates at least three webinars per year on various topics, including STEM (science, technology, engineering, mathematics) and working with young children, which are provided by experts in the field or staff with expertise on a given topic. The Supercharged Storytimes Program is a coordinated effort in participation with OCLC WebJunction in which a group of six trainers from around the state received training and then offered five-week online courses for library staff. State Library staff coordinate outreach and promotion for these trainings, and also host a learning community for attendees using Moodle.
Progress Toward Project Targets. The first target associated with the project indicated that by 2021, the number of public libraries offering early literacy programs will increase by 20%. Annual report data provided by the New York State Library indicated that the following number of libraries offered early literacy programs, by year: 651 libraries in 2017; 657 libraries in 2018; 662 libraries in 2019; and 617 libraries in 2020. Thus, there were small increases in the number of libraries offering early literacy programs between 2017 and 2019, though these increases fell short of the target of a 20% increase; the number of libraries offering early literacy programs declined in 2020,[18] most likely due to the pandemic. In addition, the State Library provided support to the public library systems for early literacy programs through the Family Literacy Library Services Program.
The second target stated that by 2021, the number of public libraries involved in local collaborations to enhance early childhood school readiness would increase by 20%. Data provided by the New York State Library reported that 484 public libraries were involved in local collaborations in 2017. This number declined in subsequent years measured (479 in 2018, 473 in 2019, and 384 in 2020), indicating that the target was not met. According to the NYSL, this was likely due to the pandemic.
The third project target indicated that by 2021, 85% of public libraries would have staff skilled in the provision of early literacy services. According to data provided by the New York State Library, by 2021, 650 of 1,067 outlets (approximately 61%), including main libraries and branches, had staff who attended trainings in the provision of early literacy services. Thus, this target was not met.
The fourth target stated that by 2021, 75% of staff at public libraries who received Ready to Read at New York Libraries training would report increased confidence in applying skills to improve early literacy services for families with young children and their communities. Surveys were collected from participants who attended webinars and presentations, as well as Early Literacy Summits.[19] The target associated with webinars and presentations was exceeded, as greater than 75% of respondents indicated strong agreement or agreement with the statement, “I feel more confident about what I just learned” (FFY 2017, 94%; FFY 2018, 94%; FFY 2019, 79%; FFY 2020, 93%). In FFY 2017, an Early Literacy Summit was held. Respondents indicated strong agreement or agreement regarding their confidence level about what they learned, exceeding the target (88%).
The data indicated significant reach through webinars. In FFY 2017, there were a total of 202 attendees across the webinars offered, which included topics on social media for promoting early literacy; and using the New York State Kids Well-being Indicators Clearinghouse (KWIC). In FFY 2018, webinars were offered on Supercharged Storytimes, early literacy partnerships, and introducing math and science concepts into story time, with a total of 336 attendees. In FFY 2019, there were 326 attendees across the various webinars offered, which included topics such as introducing babies to the library, training on a new professional development tool entitled “Raising the Bar,” and early literacy. A planned early literacy webinar on how early literacy develops in children was planned for March 2020 and had over 300 registrations, but was canceled due to COVID. The webinar was able to be rescheduled and offered in October 2020. In FFY 2020, there were a total of 616 attendees for the webinars offered. Webinar topics were diverse and included training about early literacy learners, the New York State parent portal, virtual and in-person programming for infants, summer reading resources for young readers with print disabilities, and oral narratives and stories. All webinars are recorded, close captioned, and archived on the Ready to Read website.
The fifth target indicated that by 2021, 75% of public library staff who received Ready to Read at New York Libraries early childhood training would state that they have applied what they learned to offer new or enhanced early literacy services. For the years in which survey data were collected, this target was exceeded. In response to the item “I intend to apply what I just learned,” 83% (FFY 2018) and 88% (FFY 2019) of respondents indicated agreement or strong agreement.
Implementation Barriers. The State Library staff interviewed reported that “Supercharged Storytimes” online courses associated with Ready to Read fill quickly, and there are not enough trainers to meet the high demand. With more trainers, staff would have the ability to offer additional sessions of the training to accommodate more participants.
The State Library staff also noted that the original group of trainers for the “Supercharged Storytimes” trainings has decreased due to retirements, which needs to be addressed moving forward.
Library Stakeholder Survey Findings Related to Goal 3
According to the findings of the library stakeholder survey, the Goal 3 activities endorsed most strongly by the field included fostering digital inclusion to ensure access, and increasing the quality and reach of literacy programs for youth and families. Specifically, 73% of respondents indicated that fostering digital inclusion is a high priority, and 57% of respondents indicated that increasing the quality and reach of literacy programs for youth and families is a high priority.
Of note, the NYSL began offering training on digital equity and digital inclusion during the period being evaluated, including sponsoring Digital Equity Summits in Spring 2021, which included expert presentations and topical breakout discussion groups. A combined total of 460 individuals participated in these summits.
Goal 4 was partially achieved.
Activities associated with Goal 4 include increasing visibility of statewide programs and services of the State Library; sharing information about the impact of LSTA funds; seeking public and private partnerships to support Five-Year Plan activities and to improve library programs and services; using emerging technologies to provide more effective support services to libraries and library systems; providing grants to library systems and libraries as funds are available; revising State Education Department policies, regulations, and guidelines in accordance with the Five-Year Plan; improving online planning and reporting tools; participating in national, state, and regional partnerships; improving access to the Office of Cultural Education’s research collections; providing timely access to requested copies of materials and information from State Library collections; improving statewide access to electronic federal and New York State government documents; expanding access and information provided by the State Library, both on site and online; and promoting user self-service (see Appendix E for a full list of activities associated with Goal 4).
The New York State Library implements several projects to support the goal of effectively leveraging private and public funding to support its other key goals. Projects aligned to Goal 4 include the following (note that programs marked with an asterisk align with the other goals above, and were discussed in those sections): Training and Outreach; Data for Decision Making*; Expanding Library Networking and Collaboration for Improved Learning and Access; Advancing Technology; Customer Support to Libraries; Customer Support to Individuals; Digitization and Preservation Program*; Partnerships in Resource Sharing and Access*; Discovery Services*; Building Strong Library Leaders for New York*.
Training and Outreach. Project targets were partially met.
Project Overview. Through the Research Library’s Training and Outreach project, the New York State Library provides training workshops, webinars, and presentations to assist its customers in using resources and collections. Educational programming is also offered on a range of topics, including genealogy, local history, author talks, and services for patrons with disabilities. New York State library staff are also trained by other library staff and by trainers from outside organizations.
According to the State Library staff interviewed, when the project began, it was designed to promote the library’s collections. It has since grown in scope to support the goal of improving the knowledge of New Yorkers. State Library staff were able to expand access to high-quality educational research programs led by faculty speakers.
Progress Toward Project Targets. The first target stated that 75% of State Library librarians will complete at least 12 hours of work-related training annually. According to data provided by the NYSL, all staff completed at least seven hours of work-related training annually. However, complete data were not available to evaluate if more than seven hours were completed, and thus the target cannot be evaluated.
The second target indicated that Research Library staff will provide at least 25 public classes or programs annually from 2017 through 2021. This target was met in each year, with the exception of FFY 2019 (33 public classes in FFY 2017; 42 in FFY 2018; 24 in FFY 2019; and 55 in FFY 2020). Programming was held virtually and in person. Of note, the programming was well received by participants based on survey findings, with between 90% and 100% of survey respondents indicating agreement or strong agreement that they learned something from participating in the training activity each year assessed.
The third target stated that by 2021, 60% of Research Library customers participating in public classes or programs will indicate through surveys that they learned something by participating in the activity, and that they are confident about what they have learned. This target was met for the three years in which data were available.[20] In response to the item, “I learned something by participating in this library activity,” 100% of respondents agreed or strongly agreed in FFY 2017 and FFY 2018, and 97% agreed or strongly agreed in FFY 2020. In response to the item, “I feel more confident about what I just learned,” 95% (FFY 2017), 100% of respondents (FFY 2018), and 97% (FFY 2020) agreed or strongly agreed.
Although this is not included in the target, a substantial number of consultations and reference transactions (relevant to the Research Library) were noted in FFY 2018 (290), FFY 2019 (238), and FFY 2020 (305).
Implementation Barriers. The interviewed State Library staff noted that outreach can be a challenge. Although they are advertising public classes and programs, including through newspapers, they may not be reaching as many people as possible, and view this as an area that requires further attention.
The State Library staff indicated that in-person trainings in Albany had been a challenge due to restricted parking for attendees. Initially, there was resistance from some staff regarding offering online trainings, but this waned as staff became comfortable with hosting trainings online. During the pandemic, there was a need to pivot to online offerings, leading to some perceived advantages and a greatly expanded suite of offerings, due to being able to engage high-quality speakers from around the country. Availability of online programs and effective use of social media also improved program attendance in 2021.
Expanding Library Networking and Collaboration for Improved Learning and Access. Cannot be assessed (targets not defined).
Project Overview. Through the Expanding Library Networking and Collaboration project, the New York State Library provides leadership, planning, coordination, consultation, and support services to allow library systems and libraries of all types to create and expand networking activities, partnerships, and collaborations. Specific focus is on strengthening the capacity of local public and school libraries to collaborate, maximize their resources, and offer programming and services that increase knowledge and skills of New Yorkers in a manner that promotes equity. According to the State Library staff interviewed, project staff engage in monthly meetings with public library and school systems to inform this audience about current initiatives that would impact their work.
Progress Toward Project Targets. The New York State Library did not record a specific target for this project. Thus, progress toward targets could not be evaluated. According to the SPRs, the project includes three activities: the School Library Systems activity; the Public Library Systems Leadership and Support activity; and the Minimum Public Library Standards activity. Through these activities, the New York State Library provides support through conference calls, face-to-face and virtual (Zoom) meetings, site visits, presentations, orientations, written materials, webinars, and in-person and virtual instruction.
Implementation Barriers. The State Library staff noted that staff turnover in the school library systems and the public library systems was worsened during the pandemic. One State Library staff member noted that at one point, there were concerns about the State withholding funding to libraries, resulting in some organizations laying off workers. Turnover during the pandemic was seen as worsening an upcoming staffing challenge due to retirements. The State Library staff described challenges in reaching consensus among libraries and library systems of different sizes who serve diverse populations (i.e., urban versus rural communities).
Advancing Technology. Project targets were partially met.
Project Overview. The Advancing Technology project is designed to ensure that New York libraries of all types have the technological resources to serve their users now and in the future, particularly regarding electronic environments. Specifically, the New York State Library partners with federal and state agencies, as well as academic and private entities in programs that include E-rate and broadband adoption.
According to State Library staff interviewed, webinars are provided to distribute information about E-rate and to encourage libraries to participate. Activities are also directed at supporting libraries to increase their broadband connections. In addition, a recent goal has been to make the State Library’s Division of Library Development website more accessible, allowing the public to more readily find information.
Progress Toward Project Targets. The first target associated with the project stated that by 2021, New York State will have a statewide e-book platform for use by all New York State libraries and all New Yorkers. Building on the May 2016 study Electronic Books and Public Libraries in New York State, the State Library began exploring options for creating a statewide e-book platform. Initial efforts included informal discussions with the library systems, vendors, and other e-book experts. Topics discussed covered areas such as assessment of need, project scope, currently available technological options for creating such a platform, resources needed for implementation, and estimated costs. According to New York State Library staff, after these initial discussions, the New York State Library determined that, given other priorities and the lack of sustainable resources to support such a statewide project, it was not advisable to pursue the project during the time period of this five-year plan. Thus, this target was not met.
The second target indicated that by 2022, the New York State Library website will demonstrate a 10% increase in use over 2017. Although the reporting period is ongoing, to date, a 10% increase has not been observed and the target has not yet been met. However, there were increases in users, sessions, and page views each year for the Division of Library Development website, when compared to 2017. Based on data provided by the New York State Library, in 2018–2019, there was a 7.8% increase in users, a 6.2% increase in sessions, and a 6.4% increase in page views compared with 2017–2018 data. When comparing 2019–2020 data with the 2017–2018 data, there was a 5.3% increase in users, a 2.3% increase in sessions, and a 1.5% increase in page views.
The third target indicated that an average of 350 libraries and library systems per year will receive E-rate discounts between 2017 and 2022. Data from the public and association library annual reports indicated that many libraries are part of consortia for E-rate benefits (391 libraries in 2018; 416 libraries in 2019; and 409 libraries in 2020), thus exceeding the target set. In addition to those receiving discounts through the consortia, some libraries and library systems received E-rate discounts directly. Specifically, in FFY 2017, 100 libraries and 30 library systems directly received E-rate discounts in the approximate amount of $7.8 million. In FFY 2018, 46 libraries and 26 library systems obtained E-rate discounts in the approximate amount of $6.3 million. In FFY 2019, more than 49 libraries and 21 library systems received E-rate discounts in the approximate amount of $5.8 million. Finally, in FFY 2020, more than 97 libraries and 24 library systems received E-rate discounts in the approximate amount of $8.8 million.
The fourth target stated that by 2021, 80% of public library facilities will offer the public access to minimum broadband speeds of 100 mbps. Although the 80% target was not met, the SPRs provided evidence that the percentage of libraries with minimum broadband speeds of 100 mbps steadily increased in each year of the project. In FFY 2017, the number of libraries with broadband download speeds of 100 mbps grew from 8.2% to 23% (248 out of 1,068 buildings, including library branches). It grew to 43% in FFY 2018, to 49% in FFY 2019, and to 52% in FFY 2020 (558 out of 1,067 buildings had broadband download speeds of 100 mbps or more).
The fifth target indicated that by 2021, 50% of libraries and library systems using the e-book platform will indicate that it is meeting the library’s/system’s needs and will improve library services to the public. This target was not met, as an e-book platform has not yet been implemented.
Implementation Barriers. State Library staff associated with the Advancing Technology project indicated that with a change in the E-rate program at the federal level, as of July 1, 2019, libraries and library systems can no longer apply for funding for telephone service. Since the cost of internet services is now somewhat lower, libraries may place a lower priority on applying for funding for internet services.
Customer Support to Individuals. Project targets were not met.
Project Overview. Through the Research Library’s Customer Support to Individuals project, the New York State Library uses resources and technology to make information available for scholarship and research. Specifically, reference, informational, and technical assistance are provided to individuals on site, as well as by phone, mail, and email. According to the State Library staff interviewed, requests through the website and by email are most common.
Progress Toward Project Targets. The target associated with the project indicated that the State Library would demonstrate an annual increase of 5% in the use of Research Library services and collections from 2017 through 2021, including but not limited to virtual and on-site visits, reference, circulation, and program attendance. There was a decrease in the number of items circulated: 6,452 items in FFY 2017; 5,290 items in FFY 2018; 4,012 items in FFY 2019. An increase in items circulated is observed in FFY 2020 (5,261 items). Of note, although not specifically defined in the target, there was a substantial increase in the number of items digitized in FFY 2020 (1,348 items, compared with 435 in FFY 2017). Based on data provided by the NYSL, there were 34,260 reference transactions in FFY 2017, 32,586 in FFY 2018, 17,004 in FFY 2019, and 12,500 in FFY 2020, which did not meet the target. In terms of virtual visits, the Research Library recorded between 500,312 and 769,283 page views each year from 2017 through 2021,[21] though the page views decreased over time.
Implementation Barriers. According to NYSL staff, the Research Library was closed for on-site use during the pandemic. When it reopened, COVID restrictions may have discouraged patrons; in addition, patrons may have been unsure about whether the Research Library was open. The interviewed State Library staff noted that a small book budget has resulted in the loss of resources for the New York State Library, creating barriers to meeting the needs of customers.
Of note, while State Library staff are constrained in the manner just described, they took pride in their ability to provide other specific services in great depth, such as assisting customers who need to trace laws for various reasons.
Customer Support to Libraries. Project targets were not met.
Project Overview. Through the Research Library’s Customer Support to Libraries project, the New York State Library uses resources and technology to make information available for scholarship and research to all library types throughout New York State through interlibrary loan services.
Progress Toward Project Targets. The target associated with this project stated that the State Library would demonstrate an annual increase of 5% in the use of Research Library services and collections from 2017 to 2021, including interlibrary loan. This target was not met, as there were some decreases in items circulated via ILL in some years assessed. SPR data indicated that 12,138 ILL items were circulated in FFY 2017; 14,410 in FFY 2018; 5,539 in FFY 2019; and 6,694 in FFY 2020. Of note, user satisfaction with ILL services was high in the two years in which surveys were collected, with 96% of users indicating agreement or strong agreement regarding the statement, “I am satisfied that the resource is meeting library needs” in FFY 2018, and 96% of users indicating agreement or strong agreement in FFY 2019.
Implementation Barriers. According to the State Library staff interviewed, the reduced book budget created a barrier to implementation. In addition, NYSL staff indicated that many libraries were closed during the COVID-19 pandemic, which reduced interlibrary loans. When libraries reopened, they often had reduced services and thus were not submitting interlibrary loan requests.
Library Stakeholder Survey Findings Related to Goal 4
According to the findings of the library stakeholder survey, the Goal 4 activities endorsed most strongly by the field included providing grants to library systems for innovative library programs and services, as federal funds allow. Sixty-two percent (62%) of respondents indicated that this is a high priority.
Question A-2. To what extent did your Five-Year Plan activities achieve results that address national priorities associated with the Measuring Success focal areas and their corresponding intents?
The Five-Year Plan activities achieved results that address several of the Measuring Success focal areas and their intents: particularly lifelong learning, information access, and institutional capacity. The table below highlights one project that addresses these three focal areas, and summarizes relevant results. See Appendix F for a full list of projects that align to each focal area.
| Measuring Success Focal Area | Project Aligned to the Focal Area | Example of Relevant Results |
|---|---|---|
| Lifelong Learning | Training and Outreach | Library customers are participating in public classes and programs due to the project. The target of 60% of customers indicating that they learned something new by participating in the project was exceeded in each year in which data were available. |
| Information Access | NOVELny | Library and library system staff who attended a training session on products of the statewide database program, designed to support information access, indicated agreement or strong agreement (exceeding the 50% target) in response to a survey item indicating “I feel confident about using what I have learned” in all four years assessed. |
| Institutional Capacity | Building Strong Library Leaders for New York | Library and library system staff who attended trainings provided by the State Library and its partners indicated agreement or strong agreement (exceeding the 60% target) that they learned something by participating in the training activity, and that they are likely to apply what they learned to improve library services to the public in the three years in which data were available. |
Question A-3. Did any of the following groups represent a substantial focus for your Five-Year Plan activities: library workforce (current and future); individuals living below the poverty line; individuals that are unemployed or underemployed; ethnic or minority populations; immigrants/refugees; individuals with disabilities; individuals with limited functional literacy or information skills; families; children (ages 0–5); school-aged youth (aged 6–17).
The majority of LSTA-funded projects serve a general audience. Two projects target a more specific audience but do not meet the “substantial focus” threshold, which indicates that at least 10% of total resources are committed by the overall plan across multiple years. These projects are Summer Reading at New York Libraries, which is targeted to children (ages 0–5) and school-aged youth (aged 6–17) and Ready to Read at New York Libraries, which is directed toward the library workforce.
Question B-1. How have you used any data from the State Program Reports (SPR) and elsewhere (e.g., Public Libraries Survey) to guide activities included in the Five-Year Plan?
According to the library administrators and leaders interviewed, SPR data are discussed internally, but not in a formal manner. Several State Library staff indicated that substantial time and resources go into the data collection process, leaving limited time to disseminate and discuss the findings.
One State Library staff member expressed interest in collecting data that might more meaningfully inform the work of the New York State Library and its partners. A staff member also suggested putting the data that is collected into context to show how libraries are performing important services for their communities.
The library administrators and leaders noted that the findings of the previous Five-Year Plan evaluation, including survey data that was collected through that process, helped to shape the development of the current Five-Year Plan (2017–2022) (discussed further in Question B-3).
Question B-2. Specify any modifications you made to the Five-Year Plan. What was the reason for this change?
According to the library administrators and leaders interviewed, there were no modifications to the Five-Year Plan.
Question B-3. How and with whom have you shared data from the SPR and from other evaluation resources? How have you used the last Five-Year Evaluation to inform data collected for the new Five-Year Evaluation? How have you used this information throughout this five-year cycle?
As discussed above, SPR data were collected and reviewed internally by State Library staff throughout the five-year cycle, though dissemination was more limited. Some projects, including NOVELny, were evaluated externally to assess impact and to improve implementation, and findings from these reports were shared with the leadership and with staff involved in the relevant projects.
The library administrators described using the previous Five-Year Plan and the evaluation of that plan as a starting point for developing the 2017–2022 LSTA Five-Year Plan. After reviewing the evaluation, the previous plan, and other planning resources, staff met to develop the next plan. A process of stakeholder engagement then followed. Input was sought from staff, the Regents Advisory Council on Libraries LSTA Committee, stakeholder group leaders on the stakeholder list developed by staff, and the library field. One administrator noted that the Five-Year Plan tends to be revisited around the time when SPR with accompanying data is completed. Staff and administrators consider whether activities were completed and whether metrics were met. The administrator suggested that perhaps this is too late in the year to be analyzing the success of the project. Another administrator noted that the Five-Year Plan might be a more effective living document if it consisted of smaller, more attainable goals that reflect a larger strategic plan. It would also provide an opportunity to add new projects that are important to the State Library and its stakeholders.
Question C-1. Identify how you implemented an independent Five-Year Evaluation using the criteria described in the section of this guidance document called “Selection of an Independent Evaluator.”
Lauren Polvere, Ph.D., Founder/President of Policy Strategy Consultants, LLC, conducted an objective external evaluation with methodological rigor. Consistent with the criteria, Dr. Polvere is independent of those being evaluated, and she has professional competency in evaluation. She is a developmental psychologist, evaluator, and policy researcher with over a decade of experience leading mixed methods research studies and evaluations for clients in the United States and in Canada. She has led evaluations for numerous and diverse organizations, including state and local governments, schools and universities, and nonprofit organizations. She is well-versed in designing and implementing evaluations that include document analyses, surveys, key informant interviews, focus groups, observations, and other methods. Dr. Polvere is highly proficient in analyzing quantitative and qualitative data. As a skilled facilitator, she successfully engages diverse stakeholder groups throughout the evaluation, using a participatory approach.
Question C-2. Describe the types of statistical and qualitative methods (including administrative records) used in conducting the Five-Year Evaluation. Assess their validity and reliability.
Stakeholder Survey: In collaboration with the New York State Library, the evaluator developed a library stakeholder survey, which was disseminated to library stakeholders through a listserv. In total, 494 respondents completed the survey, representing the following affiliations: academic libraries, public libraries, public library systems, reference and research library resources councils, school libraries, school library systems, special libraries (i.e., hospital, law, business, museum, and research libraries), and state agency libraries. The survey respondents included librarians, library directors or managers, system directors, other library staff, and trustees. The evaluator calculated descriptive statistics to analyze data from the electronic stakeholder survey. A content analysis was performed to summarize the findings of an open-ended survey item, which asked respondents to provide feedback for the New York State Library as it begins the process of drafting the next Five-Year Plan. The analysis of the open-ended item resulted in the development of a list of emergent themes, which were summarized and provided to the advisory committee.
Document Review: State Library staff provided State Program Report (SPR) data, existing evaluation reports, and web addresses to the evaluator. The evaluator organized relevant data from these sources into a spreadsheet, organized according to the goals of the LSTA Five-Year Plan, for analysis.
Key Informant Interviews: The evaluator conducted key informant interviews with State Library stakeholders, including administrators and project leaders associated with the projects that align to the Five-Year Plan goals and activities. Interviews were conducted virtually and were recorded and transcribed, with permission of the participants. A content analysis was performed on the qualitative data emanating from the key informant interviews. Analytic matrices and cross-case displays[22] were developed, resulting in data summaries and the identification of emergent themes.
Addressing reliability and validity: Two evaluators collaboratively coded and analyzed the qualitative data to ensure inter-rater reliability. Dr. Polvere worked closely with an advisory committee comprised of New York State Library staff to ensure alignment of the proposed evaluation approach to the Five-Year Plan and activities. Validity was also strengthened through the mixed method approach to the evaluation, in which findings were triangulated[23] (compared, contrasted, combined, and synthesized) across the above-mentioned data sources. Additionally, the evaluation process assessed perspectives from multiple stakeholder groups through the interviews and the survey to improve the comprehensiveness and trustworthiness of the data.
Question C-3. Describe the stakeholders involved in the various stages of the Five-Year Evaluation. How did you engage them?
Throughout the evaluation process, the evaluator met with the New York State Library advisory committee to obtain contextual information needed for the evaluation, and to ensure that the evaluation was designed to be as useful as possible. The advisory committee chair provided SPR and other relevant data, as well as a list of key informants to be interviewed. The full committee worked collaboratively with the evaluator to develop the survey.
Library stakeholders (New York State Library staff associated with the key projects and library leaders/administrators) were engaged through the key informant interviews and provided further context regarding their projects, as well as feedback on priorities for the next Five-Year Plan. Additionally, the survey engaged stakeholders from library systems and libraries by eliciting their perceptions of the level of priority of current projects and initiatives; by asking for their reactions to recommendations from a recent needs assessment, which will inform the upcoming Five-Year Plan; and by gathering their recommendations for New York State Library staff as they prepare to develop the next Five-Year Plan.
Question C-4. Discuss how you will share the key findings and recommendations with others.
The New York State Library will use the key findings and recommendations from this report to inform the development of the next Five-Year Plan. This report will be submitted to IMLS and will also be disseminated to Regents Advisory Council on Libraries, State Library staff members, library systems, and other stakeholders. This report will be posted on the State Library’s website.
| Person Interviewed | Role |
|---|---|
| Lauren Moore | State Librarian |
| Liza Duncan | Principal Librarian |
| Carol Desch | Coordinator of Statewide Library Services and Director of Library Development |
| Person Interviewed | Project |
|---|---|
| Barbara Lilley | Partnerships in Resource Sharing and Access |
| Carol Desch[24] | Expanding Library Networking and Collaboration for Improved Learning and Access |
| Ian Duckor | Discovery Services, Making New York History Materials Accessible |
| Mary Beth Farr | Targeting Library Services to Meet Changing Needs for Literacy, Education, and Outreach |
| Natalie McDonough | Targeting Library Services to Meet Changing Needs for Literacy, Education, and Outreach |
| Sharon Phillips | Ready to Read at New York Libraries, Summer Reading at New York Libraries, Teen Services |
| Frank Rees | Building Strong Library Leaders for New York, NOVELny |
| Jane Minotti | Data for Decision Making |
| Amy Heebner | Advancing Technology, NOVELny |
| Sarah McFadden | Advancing Technology |
| Amy Peker | Digitization and Preservation Program |
| Cara Janowsky | Customer Support to Individuals, Customer Support to Libraries, Training and Outreach |
Please tell us about your role.
The New York State Library, in partnership with the Regents Advisory Council on Libraries, recently commissioned a needs assessment to create a vision and plan for New York’s libraries. The needs assessment consisted of an environmental scan and a survey, as well as focus groups and interviews with New York State Library staff and representatives from libraries across the state.
The following priority areas emerged from a process of stakeholder engagement.
Please indicate your opinion regarding the priority level of each area identified in the recent needs assessment:
The New York State Library would like your input on a number of areas identified in the current Five-Year Plan (2017–2022).
The areas below address improving access for all New Yorkers (Goal 1 from the Five-Year Plan).
Please indicate your opinion regarding the priority level of each area identified in the current Five-Year Plan (2017–2022):
The areas below address delivering new and improved library programs (Goal 2 from the Five-Year Plan).
Please indicate your opinion regarding the priority level of each area identified in the current Five-Year Plan (2017–2022):
The areas below address fostering the perception of libraries as community learning spaces (Goal 3 from the Five-Year Plan).
Please indicate your opinion regarding the priority level of each area identified in the current Five-Year Plan (2017–2022):
The areas below address ensuring that New Yorkers will benefit from statewide programs and services (Goal 4 from the Five-Year Plan).
Please indicate your opinion regarding the priority level of each area identified in the current Five-Year Plan (2017–2022):
Please provide your opinion regarding the role of the New York State Library.
| Measuring Success Focal Area | Projects that Align with the Focal Areas |
|---|---|
| Lifelong Learning | Summer Reading at New York Libraries Training and Outreach |
| Information Access | NOVELny Making New York History Materials Accessible Discovery Services Partnerships in Resource Sharing and Access Digitization and Preservation Program Data for Decision Making Targeting Library Services to Meet Changing Needs for Literacy, Education, and Outreach Expanding Library Networking and Collaboration for Improved Learning and Access Advancing Technology Customer Support to Libraries Customer Support to Individuals |
| Institutional Capacity | Building Strong Library Leaders for New York Ready to Read at New York Libraries |