LSTA: Evaluation of NOVEL
Evaluation of New York's Library Services and Technology Act
Evaluation of the New York Online Virtual Electronic Library (NOVEL)
Himmel & Wilson, Library Consultants
February 18, 2007
|
This document
also available in .PDF
|
Contents
The Evaluation
Maps
Summary of Focus Groups -- Librarian and End-User Sessions
Summary of Interviews with Library Professionals
Summary of Interviews with End-Users
Maps 3-12: NOVEL Usage 2005
Responses to Specific Questions Raised in the Request for Proposal for the NOVEL Evaluation
Appendices -- NOVEL Evaluation --
Executive Summary
The New York Online Virtual Electronic Library (NOVEL) offers residents of New York State access to a wide variety of authoritative online information resources at no direct cost to the end-user at the point of access. People can enter the digital databases from computers located in libraries and schools as well as from their home or their workplace. The resources available range from basic information intended for young school children (e.g., Searchasaurus) to business resources (e.g., Business & Company Resource Center) of interest to specific segments of the adult population.
The NOVEL program is coordinated and managed by the New York State Library (NYSL), which is part of the Office of Cultural Education, within the New York State Education Department. However, funding for NOVEL is solely dependent on temporary Federal funding made available to the New York State Library through the Library Services and Technology Act (LSTA).
In 2006, the New York State Education Department issued a Request for Proposal (RFP) to conduct an independent evaluation of the NOVEL program. There were several purposes for conducting the assessment. First, the evaluation of NOVEL is part of an ongoing effort to be highly accountable for the expenditure of public dollars. Second, the assessment was designed to evaluate the program’s performance carried out under the State’s 2003 - 2007 Five-Year LSTA Plan and to inform decision-making regarding improvement of the program as the next LSTA Five-Year Plan is developed.
Himmel & Wilson, Library Consultants, a firm with extensive experience working with state library agencies throughout the United States and considerable familiarity with LSTA, was selected to conduct the evaluation of the NOVEL program. The consultants were asked to address seven questions. Short responses to these seven questions follow in this executive summary and these answers are expanded upon in the body of the report.
In an effort to ascertain the answers to these questions, the consultants carried out a wide variety of data-gathering activities. Focus groups with librarians and end-users of the NOVEL databases were held in nine locations throughout the State. A web survey that attracted more than 1,200 responses from librarians and end-users was conducted. More than thirty librarians who were very familiar with the program were interviewed as were a number of end-users.
The consultants also reviewed background documentation regarding the NOVEL program and analyzed both usage data and information regarding the more than five thousand schools and libraries that are officially registered for the program. The consultants also reviewed efforts that have been made to raise public awareness of the NOVEL program and to train librarians and library staff, educators, and the general public to use the databases effectively. Following then, is an overview of the evaluation structured around the seven questions originally raised in the RFP.
How has NOVEL helped the user, both librarians and their end-users?
- NOVEL equalizes access to authoritative information
- NOVEL is available 24X7
- NOVEL is not limited by geography
- NOVEL expands both the breadth and the depth of the information resources that are available
- NOVEL is cost-effective
Are users getting what they need?
- NOVEL has resources that are relevant to a broad cross-section of the population
- Most needs that are not being met are either at the “low end’ (very basic resources) or at the “high end” (technical, scholarly resources)
- Other libraries, library systems, and library consortia fill some, but not all, of the low and high end needs
Is the State Library communicating well with users about NOVEL?
- The State Library has been highly effective in communicating information about NOVEL to the library community
- Communications efforts have reached many sophisticated library users
- Communications efforts have failed to reach a large percentage of the general public in spite of considerable effort
- The NOVEL Statewide Education and Information Program that is in its early stages has great promise for improving communications with the general public
Did the State Library expand resources to all users including both large and small libraries?
- Users of small and large libraries have better access to authoritative information resources because of the NOVEL program
- Small libraries gain basic resources
- Large libraries redirect funds they would have spent on basic electronic resources to purchase/license other valuable resources
- Some libraries benefit by “building” on the NOVEL base (licensing more in-depth electronic resources from the same vendors)
How has the NOVEL program benefited libraries (and their users) through cost savings?
- There are both direct and indirect cost savings to libraries
- Library users have access to expanded resources at a lower cost
- Libraries indirectly benefit because they do not need to purchase, process, store and maintain certain kinds of resources (they need to own fewer journals/periodicals)
What do non-participating libraries and borderline participants need to be able to participate?
- Ongoing training and public awareness efforts will be needed
- Mass market advertising that would raise public awareness and drive user expectations will be needed
What next steps in regard to NOVEL should be considered for the new 5-year plan?
- Increase public awareness of the NOVEL program (The NOVEL Statewide Education and Information Program is an important step in this direction)
- Work even more closely with other consortia that license databases (such as Westchester Academic Library Director’s Organization [WALDO], the SUNY and CUNY libraries, and the New York State Higher Education Initiative [NYSHEI] to coordinate licensing efforts to ensure maximum benefit to all New Yorkers (both from a cost standpoint and depth/breadth of resources)
- Exert greater effort in an attempt to build a coordinated continuum of resources (create users in primary grades and provide a path that continues database use through adult years)
- Continue to work with partners (libraries/library systems) on training and staff development related to NOVEL databases
- Encourage State investment to expand/extend resources now licensed using LSTA funds
- Work collaboratively to reduce the number of interfaces used to access the NOVEL databases
- Expand access to federated searching while continuing to offer “expert” systems
New York State Library staff members have been, and continue to be, active in their efforts to improve the NOVEL program. Examples include pilot projects featuring federated and simplified searching and the improvement of marketing efforts through the NOVEL Statewide Education and Information Program.
As was indicated earlier, the NOVEL program is entirely supported with Federal Library Services and Technology Act funds. NOVEL addresses four of the six purposes for the LSTA program that are outlined in the Act’s authorizing legislation. These purposes are paraphrased on the following page.
The LSTA purposes addressed by the NOVEL program are:
- Expanding services for learning and access to information and education resources in a variety of formats, in all types of libraries, for individuals of all ages
- Developing library services that provide all users access to information through local, State, regional, national, and international networks
- Providing electronic and other linkages among and between all types of libraries
- Developing public and private partnerships with other agencies and community-based organizations
Measured against these high level goals and purposes, NOVEL is clearly an outstanding success. The consultants found many specific examples that serve to illustrate increased educational opportunity, expanded access to authoritative information, increased usage of networked information resources, exemplary cooperation between and among various types of libraries, library systems and the state library agency, and some new partnerships between libraries and community organizations.
Between 2003 and 2005, the total number of NOVEL online sessions (Gale and EBSCO combined) increased from 5,190,171 to 10,236,541, an increase of over ninety-seven percent (97.22%). Between 2003 and 2006, the number of NOVEL searches conducted increased by an even greater percentage (214.53%) from 9,894,816 to 31,122,781. This translates into approximately 1.61 searches per capita!
Furthermore, data gathered from focus groups, interviews, and surveys serves to underscore the importance of the NOVEL program in the lives of individuals. The consultants found ample evidence to support the conclusion that NOVEL is an exceptional resource that supports learning from childhood through adulthood and that the information gleaned from database searches contributes to both the quality of life and the educational, occupational, and personal success of countless New Yorkers.
The Evaluation
Introduction
The New York Online Virtual Electronic Library (NOVEL) offers residents of New York State access to a wide variety of authoritative online information resources at no direct cost to the end-user at the point of access. People are able to find and retrieve information on a wide variety of topics ranging from health and business information to full-text versions of journal and newspaper articles. Access to these resources is provided over the Internet through computers located in schools and libraries as well as those in the homes of New Yorkers from Buffalo to Montauk and from Plattsburgh to Jamestown.
New York is among a large number of states that offer their residents some access to a set of high-quality online databases. From Alabama’s “Alabama Virtual Library” to Wyoming’s GoWYLD, most states have recognized the importance of electronic resources to the educational, intellectual, and economic vitality of their populace. While a majority of states in 2007 are providing some sort of state database licensing program, all do not. Furthermore, the depth of the resources and the breadth of access to the resources vary tremendously from state to state. The mechanism used to fund the licensing of digital resources also varies tremendously.
The common thread among the programs is a tried and true principle followed by libraries for centuries. By aggregating demand for intellectual property and by developing a mechanism for shared access to that intellectual property, the state can offer a greater number of people the benefit of the content of information resources.
At different times in history and as information has been stored in different formats, the “aggregation and access” models that have been used have varied. Libraries have served as central repositories where original manuscripts were copied and distributed to a wider audience. Libraries have established monumental buildings where people could go to view books that have been acquired from commercial sources. In the late 20th and early 21st centuries, libraries have added the licensing of digital information resources to their content acquisition and delivery arsenal.
In the age of Google™, it is fair to ask why the licensing of costly databases is necessary. Many people believe that nearly everything in the way of information is just a simple Google ™ search away. There is no question that the advent of Internet search engines has radically changed the way people seek and retrieve information resources. At the same time, it is fair to characterize search engines such as Google™ and Yahoo™ as “good enough” information resources. People can find a great deal of useful information using these tools. However, we also know that these tools find and deliver whatever is placed on the World Wide Web regardless of its quality and/or veracity.
A few sobering facts demonstrate the need for information resources that are reliable. If you conduct a Google search on “Britney Spears,” you will find twice as many references as you will for a similar search on “Albert Einstein.” A Google search on the phrase “miserable failure,” displays a result that lists the official White House biography of President George W. Bush as the top entry in terms of relevance. This is due to the fact that Google results have been manipulated by the public through a practice known as “Googlebombing.” A variety of factors, including advertising expenditures, can influence the placement of hits on popular search engines. The information available through Google and other search engines is not necessarily factual or unbiased.
You may ask, “what about resources like Wikipedia?” Again, these resources have a place; however, a recent study revealed that Wikipedia had 31 % more errors than the Encyclopedia Britannica. Encyclopedic information, whether in print or online, is only as good as the experts who contribute content.
“Good enough” resources may be acceptable to satisfy our curiosity or to find an address or phone number of a retail outlet. However, these “free” resources are often not “good enough” for serious research purposes, for providing information regarding critical health or legal issues or for making fundamental decisions regarding investments. More and more frequently, Internet users searching for highly reliable information are faced with the words, “please enter your credit card number.”
The New York State Education Department - New York State Library offers the NOVEL databases for a variety of important reasons related to the information provided above. First, high quality research and decision making can only be achieved when researchers and those who make important decisions have access to authoritative information resources. Second, in a free society, high quality information must be available to all without a means test. When access is denied to information, access to education and success is also severely restricted.
Furthermore, if New York hopes to remain competitive in a global economy, it must provide its residents with the tools they need to contend. The labor force needs good information and the next generation needs to develop superior information literacy skills including the ability to discriminate between authoritative and questionable sources. “Good enough” may suffice when searching for information regarding a pop star but it is not adequate to prevail in the international marketplace.
There is simply no question that New Yorkers need online access to quality information resources. It is not an understatement to say that the educational and economic vitality of the State’s residents is at stake if such resources are not provided. The question then is not whether or not a program such as NOVEL should be offered. The question is more whether the program is as efficient and effective as it can be in achieving its goal of connecting people with up-to-date, authoritative information resources. The following evaluation explores this question in depth. The evaluation is not based on the consultants’ opinions. Rather, it is based on an analysis of data and on contact with hundreds of librarians, school library media specialists, teachers, and citizens who use the NOVEL databases on an ongoing basis.
The Evaluation Process
In February 2006, Himmel & Wilson, Library Consultants responded to a Request for Proposals issued by the New York State Education Department – New York State Library seeking a consultant to conduct an independent evaluation of the New York Online Virtual Electronic Library (NOVEL). Himmel & Wilson was subsequently awarded a contract to conduct the evaluation and work began in earnest on the project in May 2006.
The evaluation involved multiple approaches to determining the effectiveness of the NOVEL program. The consultants:
- met with New York State Library officials and staff involved in the program,
- reviewed and analyzed statistical data regarding the program,
- held a series of focus groups with librarians and school library media specialists using the NOVEL databases,
- held a series of focus groups with end-users of the NOVEL databases,
- conducted a series of web-based surveys with librarians using the NOVEL databases and with end-users of the databases, and,
- conducted personal interviews with librarians and with end-users of the NOVEL program.
While no single aspect of the evaluation provides a comprehensive picture of the successes and failures of the program, together, they offer an accurate picture of an extremely valuable program that, nevertheless has some weakness that should be addressed.
In all, the consultants had some contact with approximately 1,500 residents of New York State regarding their experiences with the NOVEL databases. This contact, achieved through the focus groups, interviews, and web surveys provides a look at the impact of NOVEL on the lives of real people. The locations of the focus groups provide one glimpse at the range of opinions that were sought. At least one session was held in each of the nine Reference and Research Library Resources Systems (3Rs) areas of the State of New York.
Sessions were held in:
- Batavia
- Canastota
- Farmingdale
- Fredonia
- Highland
- Ithaca
- New York City
- Saratoga Springs
- Watertown
The map below (Map 1) shows the distribution of the focus group sites. Focus group sites are represented by a red dot.
Map 1 -- Focus Group Sites
The web surveys also managed to gather perspectives from across the entire Empire State. Librarians and school library media specialists were encouraged to participate in the survey through messages shared on listservs and e-mail lists used by the State Library to communicate with the library community. Some contacts were also made with librarians who had been involved in “invitational grants” that trained teachers and end-users in how to get the most out of the databases. These individuals played a key role in encouraging end-users to participate in the survey. Similar surveys were aimed at librarians and school library media specialists, teachers, business users of the databases, personal users of the databases, and other users who did not fit nicely into one of the aforementioned categories. In all, approximately 1,200 individuals participated in the initial set of web surveys. A separate web survey was aimed at NOVEL users who access the databases through the State Library’s portal to the NOVEL databases. Map 2, below, shows the distribution of survey respondents by postal zip codes. As you can see, the web surveys also achieved a wide distribution of responses. Zip codes represented by survey respondents are shown in purple.
Map 2 -- Web Survey Respondents
Individual interviews were held with librarians and with end-users both in person and by telephone. The analysis of statistical data involved an examination of the databases used to track institutions that participate in the Novel program and reports on usage provided by the database vendors for 2003, 2004, and 2005.
Summary of Focus Groups – Librarian and End-User Sessions
Between July 10th and July 18th 2006, Himmel & Wilson, Library Consultants conducted a series of focus groups in the nine Reference and Research Library Resources System areas of the State of New York, generally an end-users’ group and a librarians’ group in each region. One end-user session was cancelled because there were no participants. Total attendance at the seventeen focus group sessions was 119 participants. The nine librarians’ sessions had a total attendance of ninety; the eight end-users’ sessions had a total attendance of twenty-nine. Four of the end-user sessions had only one participant each. Unfortunately, the project schedule resulted in the need to schedule focus groups during a period of time when many individuals schedule vacations. The schedule also limited the opportunity for the participation of school personnel and students. Because of these realities, attempts were made to gather information from these important constituencies using other methods such as web surveys and interviews.
More extensive summaries of the focus group sessions are provided in Appendix A (Librarians) and Appendix B (NOVEL End-Users).
Key findings from the sessions with librarians:
The NOVEL databases are a major resource, essential in some libraries and complementary or supplementary in others.
- Public librarians reported that reference questions were increasingly being answered with electronic resources and that fewer paper reference materials were being purchased.
- A medical librarian pointed out that printed information might already be two years old and added that she had no room to house back issues of periodicals in paper.
- School library media specialists and librarians from some small school library media centers and small public libraries said they had no databases other than those available through NOVEL.
- Even librarians and school library media specialists who said the databases were supplementary went on to say that the currency of the databases made them an important supplement to the print collections.
- An academic librarian explained that the databases were complementary; their focus is on the graduate school level, but the library also serves faculty and students on the personal level as well.
- Having NOVEL databases, “augment what we can afford to pay for.”
- Several participants from various types of libraries emphasized that consistency across all types of libraries was important to students as they moved from elementary to high school to college to using public or special libraries. “We can help students become knowledge information literate; the NOVEL databases raise the bar.”
Database use is frequently “self-taught” although the grants that have been offered for training have been extremely helpful.
- Training needs to be ongoing because staff members change, the databases change, and even those with training forget how if they use the databases infrequently.
- Training needs to be provided at the teachable moment or in relation to a specific subject of interest to the user. One-on-one training seems to be highly effective in training new users.
Participants find people/potential users do not understand the difference between databases and the Internet. Users often “Google” to get information instead of turning to more authoritative sources such as NOVEL.
If the NOVEL databases disappeared tomorrow, many libraries and their users would suffer.
- Many small school library/media centers and public libraries would have very limited (or no) electronic resources. “It would decimate our resources.”
- “It would increase the gap between haves and have-nots.”
- Because the databases have been available, librarians have stopped purchasing print materials in some areas; consequently, collections would have limited materials in some areas. “It would create a serious gap in our informational resources; I’d have to cut something else.”
- “We wouldn’t find a lot of answers for many of our customers.” (Reference service would suffer.)
- Participants thought the users would miss the health resources and the newspaper databases most.
- “It’s very useful in academic libraries, but academics wouldn’t be impacted as much; the academics buy (license) a lot of things on their own or through consortiums.”
- The image of libraries and librarians would suffer. “NOVEL makes me look good to my staff and students. You’d look bad to the public if you didn’t have those electronic resources.”
Participants gave several reasons they believe some people choose not to use the databases.
- Potential users don’t know what the databases are. “It’s an awareness issue.”
- Some find searching difficult. (the majority of librarians and end-users expressed a high level of satisfaction in regard to ease of use; however there is a small percentage of users who find searching at least some of the NOVEL databases, difficult.)
- Driver’s license access is helpful to some, but a deterrent to students without a New York State driver’s license, either because they are too young to have a driver’s license or, in the case of some college students, may be from out of state.
Many suggestions were offered for electronic resources to be added to the NOVEL databases; however, the suggestions seemed to vary by the clientele served.
- School library media specialists often said “any good encyclopedia” or named a specific encyclopedia. However, some librarians from other types of libraries did not think encyclopedias were heavily used by their clients even though they had tried different ones.
- School library media specialists also suggested that databases coded for reading levels would be very helpful.
- Public librarians suggested genealogical resources and test preparation databases along with newspapers and various other topical databases.
- Academic and medical librarians wanted more technical and/or “higher level” databases.
Suggestions for marketing the databases to the general public included
- Market NOVEL as being a safe source of information
- Get out where the users are to inform them of what is available
- Conduct a statewide public awareness campaign (although some participants disliked the idea of spending money on marketing; they preferred that funds be spent on content.)
- Strengthen the “brand recognition” of NOVEL (although some felt that starting over with a new, more descriptive name made sense)
Other issues arose and were discussed during the focus group sessions as well. Among the issues raised were:
- Getting usage statistics for the databases is difficult at some levels.
- Databases are complex to use: protocols are not standardized; spelling is a challenge for some people and search strategies are difficult for some people to develop.
- Terms such as databases and NOVEL confuse potential users.
- There is disagreement over the usefulness of federated searching at this point.
Key findings from the sessions with end-users:
Gathering end-users for the focus groups was a difficult task for the libraries and systems. This may have been in part a matter of scheduling; the sessions were held during the day when many people might be expected to be at work. Some librarians also suggested it would have helped if there had been some incentive (mileage, for example) available to people to travel, sometimes long distances, to attend the sessions. Nevertheless, those who attended the sessions had useful information to share. Writers and library trustees made up a good number of those who came to the sessions. In Manhattan, the participants were people who were interested in signing up for computer classes to learn about various programs; their exposure to the databases was somewhat limited. However, this group’s interest in the databases was extremely high.
The end-users made many comments and suggestions that supported or echoed what the librarians had said in their sessions.
- EBSCO, Gale, NOVEL, and “databases,” are all labels that “don’t mean much.”
- People are looking for a seamless way to get into the databases; they want to search quality resources more “like Google.”
- Participants described their difficulties in searching. Difficulties seemed to be related to differences in the screens from different access points and different searching strategies required by the various products. End-users did not like having to key in their ID number each time they moved to a different database (as is necessary when entering the databases through some access points).
- Databases that end-users would like to have available include HeritageQuest, a good basic encyclopedia, specific periodicals, and test preparation materials.
- End-users often said they got their training in using databases and software at the library; however, like many of the librarian participants, many end-users seemed to be largely self-taught; they just keep trying different things, trial and error, until they find what they’re looking for.
- What’s great about the databases is that they are “free!”
- The focus group participants (who were probably more sophisticated searchers than many potential end-users) did know that databases were different from the Internet. One participant said “…databases are juried or carefully selected and therefore they are more trustworthy.” This person went on to say that the NOVEL databases are “better quality” than much of what is found on the Internet.
- Their recommendations were to make searching more user friendly; do training by topic/subject; and make the databases more accessible to people with disabilities.
- They think librarians are missing a great PR opportunity with the databases. A PR campaign needs to go beyond the libraries to bring new users in. One lady said, “Publicity about it (NOVEL) is severely lacking!”
Summary of Interviews with Library Professionals
The NYSL provided the consultants with a list of the names of 33 library leaders in New York who had been involved in various aspects of the development and implementation of the NOVEL program. Over the course of several weeks the consultants were able to interview all but three of the individuals named. A list of those interviewed as well as a more extensive coverage of the content of the interviews can be found in Appendix C.
Many of the people interviewed had been involved with the development of the NOVEL program and their comments were very helpful to the consultants’ understanding of how the NOVEL program has developed. The consultants asked each of the people interviewed about their involvement with the NOVEL program, what they saw as the successes, what has not happened as they had anticipated or wanted, what or which of the five initiatives outlined at the onset of the NOVEL program should be the next priority, and what they believed the impact of NOVEL has been.
There was both a wide variety in the responses and some general agreement.
Successes (and impact of the NOVEL program):
- The widespread geographic access to the databases
- Equitable access, i.e., a “level playing field”
- Cost savings
- Leveraging existing budgets
- For some libraries the NOVEL program makes databases available where they would not otherwise be.
What has not happened as anticipated or wanted:
- Driver’s license access complicated things, especially for students and schools (however, many thought it was a good approach to widening the public’s awareness of the program)
- The selection of databases is too general, “low-end” for academic institutions and not low enough for elementary schools.
- Usage data is inadequate or unclear.
- Access points differ too much from library to library.
- Some librarians are hesitant to train end-users to use the databases.
- Some find NOVEL hard to use; the interfaces are complicated and the state library website is “way too complex.”
- Potential users are unaware of what the databases contain and of how to use them.
Priorities for the future among the 5 initiatives in the NOVEL program:
- Many of those interviewed still place a high priority on the database program—refining it, extending it, etc.
- Most of those interviewed questioned the meaning and intent of the second initiative, expanding resource sharing.
- Digitization is not a high priority for those interviewed; coordination of digitization does have some support.
- While several people recognized that the lack of high-speed telecommunications was a problem for libraries in rural areas of the state, most interviewees did not think NYSL should have a major role in addressing this problem.
- Federated searching as a concept was widely supported; however, this initiative seems to be in its infancy in New York and people had a wait and see mindset about it as a statewide initiative. Some thought that the technology would have moved beyond the outcome being sought by the time federated searching was widely possible in New York libraries.
Summary of Interviews with End-Users
NOVEL end-users interviewed via telephone included educators, students, researchers, senior citizens, and a number of people who characterized themselves as “personal” users who used the databases to answer questions that arose in their lives. Some had been introduced to the databases through presentations made by librarians, but others were self-taught. It appears that some school library media specialists have worked successfully to integrate the databases into the work that teachers do.
Some suggestions for improvements were
- People generally would prefer to get full text rather than citations or abstracts.
- Sometimes changes happen in the databases that the end-users do not know about until they discover them online during another search. One person said she “couldn’t always find what I’d found the last time I searched.”
- End-users seemed hesitant to be critical when they were asked how the NOVEL program might be improved because some at least thought the “fault” lay with them and their searching skills rather than with the databases or the way access was structured.
- One gentleman was very specific in his call for improvements. He did not like that “…you have to keep putting in your login and pin number. Once you log in, that should be enough. Why do you have to use the pin in addition to the library card number? Why do we need the extra step?”
The comments that follow are excerpts from some of the interviews.
A high school teacher who has had NOVEL training said she uses the databases as a student, a Mom, and a teacher. The school library media specialist at the school where she teaches also helps her with the research she is doing as a graduate student. Her son goes to a different school. “We used the databases this weekend from home. They’re an EXTREMELY VALUABLE tool. I have a sheet listing the EBSCO databases on my computer and the school library media specialist gave all of us (teachers) a list of what’s in each database. The best way to learn is hands on, but the librarian comes in whenever I ask. She tells us what we need when we need it. She has created a package for each project my students do that says which database is best for each topic.”
A health care worker said she had not used databases much, but she “keeps the information handy. The health databases seem user friendly.”
A woman whose job has her working at a computer all day said she had taken courses at Mid-York Library System on the databases. Her job leads her to use the Business Resource Center for the 800 numbers. She likes EBSCO databases; her son also uses them in his driver’s education class. She uses the Health Reference Center for personal use; she likes the full text. She has also used the custom and New York State newspapers.
Many people do not make a distinction between databases available through NOVEL and other online resources offered by their local library, library system, or educational institution. The consultants found that this kind of confusion was widespread. It was encountered in focus groups, in personal interviews, and in the web surveys. While librarians had a much clearer picture of what was and wasn’t part of the NOVEL suite of databases, even a few librarians mistakenly associated particular databases with NOVEL.
A school library aide who used to be a teacher’s aide for special education staff said that the library media specialist gave her training and now she teaches people how to use the databases. The databases are “incredible!” She characterized her students as “needy.” “Searchasaurus is so right for them—6th, 7th, and 8th graders. They can understand it and use it for science, history, etc.” Now she’s teaching MasterFILE™ Select to 6th-12th graders. She uses the databases several times a day. She thinks the databases are really easy to get into, understand, and maneuver around in. She’s not afraid, although she sees others who seem to be; she also promotes use of databases. “They have to be aware that Google is not the answer.”
A retired academic librarian said she used the databases to prepare a 20 minute paper on Korea for a club she belongs to; the librarian had told her about the databases. She also uses the newspapers database, especially the New York Times.
Another woman reported using NOVEL databases “for job information and entertainment.” She learned by “playing around” with the databases. She wishes there were more weekly newspapers along with their backfiles (especially local papers) because she does genealogy and local history searching for her mother. Again, this end-user moved freely between talking about NOVEL resources and other databases and resources available on the web.
A retiree who is also involved with a computer users’ group said he uses Google “quite a bit.” He has favorite sites, is interested in podcasts, blogs, online magazines and newspapers. He stays up late, but doesn’t watch TV; has been a library user since he was a child. He accesses everything from home; gets medical stuff through the NYSL web page and through the Mid-York System. He has been using databases, including NOVEL offerings for two years. He said that he is “…curious about what’s out there.” He added, “We appreciate the links.”
A director of special education said she learned about using the NOVEL databases through the school library media specialist who sponsored a series of workshops for the faculty and provided them all with passwords. The special education director uses the databases professionally, e.g., when there is a new student with “issues” who’s taking medications, she researches the medications and the issue the student is dealing with. She also researches early literacy topics. She likes that “the databases provide access to a wealth of information, especially professional information, at our fingertips.” They allow her to do research and get more information on topics, “anything I need to help our kids.” While she does not see any negatives, sometimes she gets just an abstract when she would like full text. She could get the full text through the librarians, but there’s a lag time involved in doing that. “This is a poor rural community. We’d have to drive 20-30 minutes to get to this information otherwise. The databases are wonderful for the kids and teachers. We’re fortunate to have the level and scope of the databases available within our school.”
The special education director went on to say that “the school media person also does twice a month lunch time workshops with the IT person to get teachers involved, ‘technology Fridays,’ — a 45 minute session where you can eat your lunch and get training at the same time. All you have to do is give the librarian and IT person a topic you’re interested in.”
A man who described himself as “a parent” and who did software testing and web development from home said he had never had any training in the use of the databases, but he had lived in another state and used ProQuest there. His use of the NOVEL databases is mostly “personal interest,” reprints of New York Times articles, consumer reports, etc. He especially likes that it is “real time, especially for newspapers.” He described using the databases as “intimidating. I’m confused sometimes; the web page has several services and different search engines. Each one is different; you have to use a sequence query. I wouldn’t know how to explain how to find things to somebody else. I just do searches for people rather than trying to explain.”
Another library media specialist who had also worked in a public library said she is “self-taught on the databases” but has given training to others. She has made bookmarks, promoted databases, and used the health and business databases. She thought the learning curve with business resources was steep because she didn’t know about them or what was there although navigation wise they were no more difficult than other products. She wished people knew about the NOVEL databases and used them; she “personally uses and loves them.” She also teaches 7th grade information literacy and uses databases for that.
The editor of a weekly newspaper indicated that she uses New York Times articles about once a week; she generally searches by using a couple of keywords. She would like the pictures to be there as well; she does not look for specific articles, just uses the subject approach and has no particular difficulties. She is “quite happy with it; it’s nice to be able to ask for either an abstract or full article although I use the full article most of the time.” The oldest things she sees are from the late 1990s and it would be great if the file went further back. She used JSTOR in college; “it would be nice to have access to that, although it probably was expensive.” “The databases help me all the time!”
Summary of Web Survey Results
Overview
Over 1,200 librarians/school library media specialists, students, teacher/educators, business users, personal users, and other users responded to a web survey about the NOVEL databases between May 21st and October 31st, 2006. They lived in 454 different zip code areas. Seventy-seven percent (77.08%) of the respondents were women and fifty-one percent (51.34%) were between the ages of 35 and 54 years. Thirty-three (33.16%) percent completed the survey from home. (A compilation of the survey responses follows in Appendix D.)
Some key points include
Accessed NOVEL databases several times each week |
28.42% |
Accessed newspaper databases |
47.59% |
Success in meeting information needs with databases |
3.92 on 5-pt scale |
Had no difficulty in finding information sought |
86.74% |
Learned about databases from librarian or school library |
44.07% |
Respondents also said what they liked best about the NOVEL databases are accessibility, ease of use, and the fact that they are “free.” They also liked that at least some of the databases are full-text and that they cover a wide variety of topics.
Background and Methodology
As a part of the evaluation of the NOVEL program, the New York State Library Division of Library Development (DLD) promoted participation in a web survey about the program to various groups of people who were expected to have some familiarity with NOVEL databases. The groups invited to participate were librarians/school library media specialists, students, teacher/educators, and general users of the NOVEL databases who had the option of identifying themselves as users of the databases in relation to their job or business or users pursuing a personal interest.
Who participated in the survey?
During the time the survey was available, 1,244 people completed it.
Category of users |
Total # of |
% of |
Librarians/School Library |
642 |
51.61% |
Students |
182 |
14.63% |
Teachers/Educators |
92 |
7.40% |
Business Users |
98 |
7.88% |
Personal Users |
188 |
15.11% |
Other Users |
42 |
3.38% |
Not surprisingly, librarians and school library media specialists represented the highest percentage of respondents.
Respondents were asked to give their home zip code. While some individuals did not offer this information, 1,145 respondents identified 454 different zip codes. The breakdown of home zip codes by category of users was:
Category of users |
Total # of |
# Unique |
Librarians/School Library Media Specialists |
623 |
387 |
Students |
156 |
51 |
Teachers/Educators |
80 |
64 |
Business Users |
89 |
53 |
Personal Users |
163 |
112 |
Other Users |
34 |
31 |
The zip codes with the largest number of respondents were typically the most diverse in terms of types of respondents as well. For example, the 37 responses from zip code 11215 in Brooklyn included 9 librarians/school library media specialists, 9 business users, 9 students, 7 personal users, 2 teacher/educators, and 1 “other” user.
Two hundred and seventy-six, or fifty-seven percent (57.14%) of the zip codes had only one respondent living in that code area. There were only seventeen zip codes which were given by both teacher/educators and students. The consultants surmise that this number is low because the ages of the students (see below) tended to indicate college aged students rather than high school students. The teacher/educators were likely to be at the high school level. Also giving a home zip code rather than the school or work zip code may have separated individuals as well.
Seventy-seven percent (77.08%) of the respondents were female. The breakdown by category was:
Category of users |
Female |
Male |
Librarians/School Library Media Specialists |
86.22% |
13.78% |
Students |
71.15% |
28.85% |
Teachers/Educators |
75.61% |
24.39% |
Business Users |
59.57% |
40.43% |
Personal Users |
61.85% |
38.15% |
Other Users |
66.67% |
33.33% |
While one might expect the librarians/school library media specialists and teacher/educator categories to have a higher percent of female respondents based on the general proportions of women in those professions, the high percents of female respondents for the other categories is somewhat surprising.
Fifty-one percent (51.34%) of the respondents overall were aged 35 to 54 years. This age group was the largest for all of the categories of users except the students. The largest age category for the students was 35 or older, which included 23.90% of the student respondents, followed closely by 22.01% who were aged 25 to 34 and another 20.75% who were aged 18-20 years.
Thirty-three percent (33.16%) of the respondents accessed the survey from home and another thirty percent (29.76%) accessed the survey from work.
How often do they access one of the NOVEL databases?
Overall, the highest number (266 respondents), which represented 28 percent (28.42%) of the respondents overall said they typically accessed one of the NOVEL databases “several times each week.” The categories most frequently mentioned were:
How often |
% |
Several times/week |
28.42% |
Several times/month |
17.63% |
Daily |
14.64% |
These numbers are highly impacted by the large number of librarians/school library media specialists completing the survey relative to the other groups. Forty percent (40.00%) of the librarians/school library media specialists said “several times each week.” The most frequent response among students and teachers/educators was “several times each month.” The most frequent response among business users was “several times each week.” The highest percent of the personal users, twenty-two percent (22.07%) said “have never used before.”
Which NOVEL database subject areas did they access most recently?
Overall the relative rankings were:
Subject/type |
% identifying this subject |
User category(ies) for which this is the most used resource |
Newspapers |
47.59% |
Librarians/School Library |
Health |
40.51% |
|
General Reference |
36.74% |
Students; Teachers/Educators; Personal Users; |
Literary criticism or authors |
27.33% |
|
Current Events |
26.69% |
|
History |
26.13% |
|
Business |
25.48% |
Business Users |
Science |
24.68% |
|
Social, political or economic issues |
20.58% |
|
Overall newspaper databases were cited by the highest percent as being the database that they used the last time they accessed the NOVEL databases. Sixty-one percent (61.41%) of the librarians/school library media specialists gave this response. However, the highest percents for students, teachers/educators, personal users, and other users said they had accessed general reference databases the last time. The highest percent of business users cited business databases as the database they had last accessed.
To what degree was their information need met during their last use of NOVEL databases?
Respondents were asked to rate how well their need was met by using a five point scale with 1 indicating that the need “was not met” and 5 indicating that the need “was completely met.” Overall the mean score was a 3.92, indicating a fairly high level of success. A 3.0 would be an “average” or midpoint on the scale, neither poor nor good. Not surprisingly, librarians gave a mean score of 4.10 and teachers/educators gave a mean score of 4.00. Both of these groups might be expected to have received training in the use of the databases and to be fairly knowledgeable in searching the databases. Personal users gave a mean score of 3.83; students gave a mean score of 3.62, business users gave 3.60. Only the “other users” group gave a mean score below the 3.0 midpoint; their mean score was 2.72.
Did they encounter any difficulty finding the information they were seeking?
The respondents overall seemed highly successful in using the databases. In general, the databases appear easy to use.
Category of users |
% indicating no difficulties |
Overall |
86.74% |
Librarians/school library media specialists |
88.69% |
Students |
84.11% |
Teachers/Educators |
86.59% |
Business Users |
79.07% |
Personal Users |
89.09% |
Other Users |
66.67% |
Overall eighty-seven percent (86.74%) said they had no difficulty. Among the categories of users this percent varied from a high of successful librarians/school library media specialists (88.69%) to a low for other users (66.67%). The small number of other users makes it difficult to identify what their challenges might have been. Twenty-one percent (20.93%) of the business users indicated difficulties in using the databases. It may be that the difficulties are related to the specific database(s) that business users in particular need to access. On the other hand, their perceived difficulty may be due to their experience using other resources that have less complicated searching mechanisms.
Some representative comments on the difficulties of using the databases include, “It would be helpful to have one search hit all the databases.” (an educator) “Often find abstracts when I want the full article.” (a librarian) “As usual, searches are only as good as the user’s ability to construct good searches: selecting keywords, subjects, using the thesaurus. Not all the database work as well as the others. The federated search feature at BPL is not very useful.” (a librarian) “Hard to know which database to use for newspaper articles.” (a librarian) “I teach in a K-4 school and the database for this level are few and not very kid friendly.” (a librarian) “Not enough high end research information—this is what is needed for work in sciences.” (a librarian) “The EBSCO databases I needed were not part of the databases provided by NOVEL.” (a librarian) “It’s too hard and takes too long to get started.” (an other user) “I couldn’t work our system.” (a personal user) “Unable to find good health information.” (a personal user) “Get good results all of the time.” (a student) “Difficulty finding full-text articles.” (a student) “Too much information that was useless and hard to obtain.” (a student)
What did they like best about the NOVEL databases?
Users in all categories cited accessibility, ease of use, and the fact that the databases are free to the end user as what they liked best about the NOVEL databases. They also cited full-text resources and the variety, scope, and wealth of resources as positives.
How did they find out about the NOVEL databases?
Overall forty-four percent (44.07%) learned about the databases through a librarian or school library media specialist. The librarian/school library media specialist was named by the highest percent of respondents in each category of user. Among the student users another nineteen percent (19.31%) indicated they had learned about the NOVEL databases through a teacher. Nineteen percent (19.08%) of the personal users and fifteen percent (15.48%) of the business users said they had learned about the NOVEL databases through the State Library website.
Other comments?
Respondents were also able to add other comments at the end of the survey. The reader is referred to the comments in Appendix D. While a few of the comments are simply “no,” meaning they had no comments to add, many of the comments are very positive, saying things like “Keep it up!” “Thank you for this important resource.” “A terrific use of tax dollars!” It is also interesting to note that most of the users, and many of the librarians, who talked about the NOVEL program as a good investment of tax dollars assumed that it was funded by the State of New York rather than by LSTA funds. The text report also includes many suggestions for additions to the set of databases offered and for improvements in how the databases are accessed.
Findings
The New York Online Virtual Electronic Library is supported entirely with temporary Federal funds allocated to New York State through the Library Services and Technology Act (LSTA) program administered by the Institute of Museum and Library Services (IMLS). NOVEL builds on EmpireLink, an LSTA funded pilot project, started in 1999, that offered a limited set of online resources. The more robust NOVEL project greatly expanded both the scope of resources provided and the mechanisms available for accessing the resources.
The NOVEL program specifically addresses one of the goals of New York State’s Library Services and Technology Act Plan. That goal is:
All New Yorkers will have reliable and equitable statewide electronic access to information resources through the creation of NOVEL and through enhancement and expansion of libraries’ technology capabilities to meet users’ informational needs.
The goal supports 5 initiatives established by the New York State Board of Regents. They are:
- Create NOVEL, the New York Online Virtual Electronic Library, to deliver high-quality, reliable digital information to all New Yorkers.
- Ensure that all New York’s students are information literate by providing strong school library media programs that include appropriately certified professional staff, adequate resources, and technology.
- Strengthen the ability of New York’s libraries to help library users acquire basic English literacy, information literacy, and computer literacy skills in their communities.
- Enhance access to the specialized resources held by New York’s academic, special, and research libraries to improve educational achievement, economic development, and health care for all New Yorkers.
- Support and enhance a skilled library workforce to meet the information needs of New Yorkers.
NOVEL also addresses four of the six purposes for the LSTA program that are outlined in the original authorizing legislation. These LSTA purposes are paraphrased below:
- Expanding services for learning and access to information and education resources in a variety of formats, in all types of libraries, for individuals of all ages.
- Developing library services that provide all users access to information through local, State, regional, national, and international networks.
- Providing electronic and other linkages among and between all types of libraries.
- Developing public and private partnerships with other agencies and community-based organizations.
Measured by all of these high level goals and purposes, NOVEL is clearly an outstanding success. The consultants found many specific examples that serve to illustrate increased educational opportunity, expanded access to authoritative information, increased usage of networked information resources, exemplary cooperation between and among various types of libraries, library systems and the state library agency, and some new partnerships between libraries and community organizations.
Between 2003 and 2005, the total number of NOVEL online sessions (Gale and EBSCO combined) increased from 5,190,171 to 10,236,541, an increase of 97.22%. Between 2003 and 2006, the number of NOVEL searches conducted increased by an even greater percentage (214.53%) from 9,894,816 to 31,122,781. This translates into approximately 1.61 searches per capita!
Furthermore, data gathered from focus groups, interviews, and surveys serves to underscore the importance of the NOVEL program in the lives of individuals. The consultants found ample evidence to support the conclusion that NOVEL is an exceptional resource that supports learning from childhood through adulthood and that the information gleaned from database searches contributes to both the quality of life and the educational, occupational, and personal success of countless New Yorkers.
However, the NOVEL program does not fare nearly as well in achieving some of the key outcome targets established in the State’s LSTA five-year plan (October 1, 2002 – September 30, 2007). In particular, it would appear that the targets that state that 30% of New York adult residents will indicate that they have heard of NOVEL, that 15% of New York residents will have used NOVEL and that 5% will say that they have benefited from access to the core collection were much too ambitious.
The NOVEL program is not the first statewide database to suffer from a lack of public awareness. In fact very few, if any, states have been successful in “branding” their database program in a way that has made it a household name. Efforts to create widespread awareness of such programs have generally been significantly underfunded largely due to a general aversion on the part of the public and law-makers to spending taxpayer dollars on any kind of publicity. This is indeed unfortunate because many excellent programs are underutilized because too few people are aware that they exist.
An article entitled Making “E” Visible that appeared in the June 15, 2006 issue of Library Journal [131, No. 11 (June 15, 2006): 40 – 43]offers several suggestions for increasing the visibility of electronic resources. Perhaps the most relevant to the New York State Library is a suggestion that appears in a supplemental section entitled “The Vendor Connection.” It advocates the position that vendors need to take a more active role in advertising directly to end-users. The consultants believe that New York, and every other state with database licensing programs such as NOVEL, should be demanding that vendors commit to conducting a robust advertising campaign as a condition of securing a licensing contract. At least one of New York’s vendors, EBSCO, has recently been reaching out to some end-users through sponsorships on National Public Radio programming. This is a start in the right direction; however, it barely scratches the surface of what needs to be done.
Public Awareness of the NOVEL Program
The consultants found that the “branding” of the NOVEL name has been largely unsuccessful among the general public. In fact, many of the users of the NOVEL databases (who were identified by librarians) were unfamiliar with the NOVEL name. Many simply knew that they were using online databases that can be accessed through their local library’s website. Furthermore, the NOVEL name was frequently confused with “NOVELIST,” a “readers’ advisory” service that is not part of the NOVEL suite of resources. As was mentioned earlier, the consultants found that this kind of confusion was widespread. It was encountered in focus groups, in personal interviews, and in the web surveys. Librarians had a much clearer picture of what was, and was not, part of the NOVEL suite of databases; however, even a few librarians occasionally mistakenly associated particular databases with NOVEL.
While the NOVEL name was widely known by librarians, many felt that the NOVEL name itself was confusing to end-users. In fact, the librarians cited terminology as creating difficulties in “selling” database use to the public in several different ways. Numerous focus group participants and interviewees made the same or similar points. They argued that on the one hand, “database” sounds intimidating and leaves the impression that the contents are technical in nature. On the other hand, many said that the “NOVEL” name leads the casual observer to conclude that the program is about fiction and, in some cases, that the content is trivial.
When asked directly whether the NOVEL name should be changed, most felt that it should and that marketing efforts to date have had only a minimal effect on encouraging use by the general public. Comments from end-users seemed to bear out this opinion.
Of the NOVEL users participating in the web surveys, over forty-four percent (44.07%) said that they found out about NOVEL through a librarian. Among students, over two-thirds (67.55%) said that they learned about the NOVEL databases from a librarian or a teacher. Almost thirteen percent (12.97%) discovered the NOVEL program through the New York State Library website and another seven percent (6.81%) found NOVEL through other (presumably library) websites.
Less than 3% of users said that they discovered NOVEL through an ad, flyer, or through the media. Most of those who did not credit a librarian or teacher with introducing them to the databases indicated that they had learned about NOVEL from a library website, a professional organization (such as a school library system, public library system, or one of the 3Rs), or by word of mouth from a friend or relative.
It should be noted that the State Library has recently embarked on a new effort called the “NOVEL Statewide Education and Information Program.” A contract has been awarded to the Ivy Group, a marketing firm with considerable experience with libraries, to work on this effort. One of the components of this project will be a new communication plan for the NOVEL program. The State Library is likely to have a much better sense of what needs to be done to increase public awareness of the NOVEL program as well as specific strategies to reach the general public as a result of the Statewide Education and Information Program.
Participation of Institutions/Organizations
While efforts to inform the general public about the databases have been only marginally successful, efforts to engage libraries in the program have been exceptional. Because of its population base and the great diversity of the State of New York, simply identifying all of the institutions that are potential participants in the program is a challenge.
Given the enormity of the task at hand, the New York State Library has, in collaboration with the various types of library systems, done a truly remarkable job. As of the end of 2005, the vast majority of school library media centers and public libraries were registered for the NOVEL program (most at the individual building level). Over 300 academic libraries were registered for the program and approximately 200 special (mostly hospitals) and non-profit organizations had been profiled. It is estimated that there are more than 7,000 libraries of various types in New York State. Of these, some 6,300 are eligible for participation in the NOVEL program. At the time of this writing, 5,322 libraries were registered. This translates into nearly eighty-five percent (84.48%) of the eligible libraries.
Building on this outstanding registration effort, more and more institutions have become active participants in the program each year. Following is an accounting of the number of institutions/organizations that showed activity (as reported by the major database vendors) in each of the last three years.
EBSCO |
Year |
Active Institutions |
2003 |
2.,071 |
|
2004 |
2,532 |
|
2005 |
2,672 |
Gale
|
Year |
Active Institutions |
2003 |
1,775 |
|
2004 |
2,050 |
|
2005 |
2,247 |
Activity by Institutions/Organizations
While some of these institutions/organizations showed minimal activity in 2005, 1,922 institutions/organizations showed 10 or more searches of Gale databases and 2,265 institutions/organizations showed 10 or more searches of EBSCO databases. A total of 790 organizations/institutions logged more than 1,000 searches on EBSCO databases and 322 institutions/organizations generated more than 5,000 searches.
Tracking exact usage between and among organizations in a comparable way is difficult because of the variety of authentication methods that are in place. In some instances authentication of users happens at the building level while in other cases, verification of eligibility to access the databases is handled at one site for many branches, buildings, or individual schools. In spite of this, there is no question that NOVEL database usage is expanding significantly.
Between 2003 and 2005, the total number of sessions (Gale and EBSCO combined) increased from 5,190,171 to 10,236,541, an increase of 97.22%. Between 2003 and 2006, the number of searches conducted increased by an even greater percentage (214.53%) from 9,894,816 to 31,122,781.
When one considers that New York ranks fifth in the nation in the number of public library reference transactions per capita (1.48 per capita in FY 2004 [Public Libraries in the United States: Fiscal Year 2004. National Center for Education Statistics. August 2006.]), it is remarkable to note that New York residents conducted 31,122,781 database searches. This translates into approximately 1.61 searches per capita!
Furthermore, data gathered from focus groups, interviews, and surveys serves to underscore the importance of the NOVEL program to individuals. The consultants found ample evidence to support the conclusion that NOVEL is an exceptional resource that supports learning from childhood through adulthood and that the information gleaned from database searches contributes to both the quality of life and the success of countless New Yorkers.
Distribution of Users
While NOVEL users come from throughout the State, it is clear that regional organizations in some areas have been more effective in spreading the “gospel” of NOVEL than others. The following series of maps (Maps 3 - 12) provide a look at the distribution of database use by zip code. The maps are based on 2005 usage statistics provided by the vendors. The vendor data indicates the number of searches associated with specific institutions. The consultants matched this data with the New York State Library’s database of registered libraries and assigned zip codes to each library or school represented.
Areas shown in white on the map had a relatively low level of NOVEL database use. Increasing activity is displayed by deepening shades of green. The darkest shade of green represents very heavy usage in a given zip code area. The scale on each map indicates the number of NOVEL searches that were associated with libraries located in a particular zip code.
It should be noted that some zip codes that show no use (displayed in grey) may, in fact, generate some database use. This is because statistics for quite a number of institutions is tracked by a regional organization or by the main or central library of a larger system of libraries. The usage represented on the map shows activity based on the zip code of the library through which users gained access to NOVEL, not the zip code in which the users reside.
The maps give some indication of how users are distributed throughout the state; however, they understate use in some areas and, to a lesser degree, overstate use in others. Note that areas shown in grey, white or in lighter shades of green should not be considered areas in which usage is necessarily poor. A zip code may be displayed in grey because the postal code area may not include a single library or school that is an access point. An area shown in white or light green may have fairly high per capita use if the area is sparsely populated.Map 3 – NOVEL Usage (Searches) by Zip Code – Entire State (2005)
Map 4 – NOVEL Usage (Searches) by Zip Code – Western New York (2005)
Map 5 – NOVEL Usage (Searches) by Zip Code – Finger Lakes Region (2005)
Map 6 – NOVEL Usage (Searches) by Zip Code – Rochester, Syracuse, Watertown Area (2005)
Map 7 – NOVEL Usage (Searches) by Zip Code – Adirondack Region (2005)
Map 8 – NOVEL Usage (Searches) by Zip Code – Capital Region (2005)
Map 9 – NOVEL Usage (Searches) by Zip Code – Catskill Region (2005)
Map 10 – NOVEL Usage (Searches) by Zip Code – Greater New York City Area (2005)
Map 11 – NOVEL Usage (Searches) by Zip Code – New York City (2005)
Map 12 – NOVEL Usage (Searches) by Zip Code – Long Island (2005)
Training
The consultants believe that much of the recent success in increasing the use of the NOVEL databases has been due to the training efforts that have been undertaken by the State Library and by organizations that received NOVEL “Invitational Grants.” The web surveys, focus groups and interviews all gave evidence that the training sessions (both for library staff and for the public) achieved results. However, it was repeatedly stressed that training needs to be ongoing. Changes in staff and changes in the methods used to search specific databases often leave libraries without the expertise they need to assist and train the public.
Tracking the impact of the training is somewhat difficult because most of the training was done by library systems while the actual usage is typically ascribed to individual institutions. A better method for aligning training offered and subsequent usage through the institutions involved in the training is needed.
One of the outcome targets established in the LSTA five-year plan was that 90% of library and library system staff who have attended a NOVEL training session will indicate in a focus group that they feel confident in promoting and facilitating use of NOVEL 24/7 core collection resources with their customers. The consultants found ample evidence that many librarians and school library media specialists had participated in training and that they valued the instruction that they received; however, it would be overstating the case to say that 90% of the librarian participants who had received training felt confident in promoting and facilitating NOVEL use.
Nevertheless, it is clear that more librarians are using the NOVEL databases to assist their customers and, in some cases, to train customers to use the resources independently. The following comments from the end-user focus group in Batavia are somewhat typical:
“Classes at the Richmond Library fill the gap for the beginner.”
“The reference librarian showed me.”
“I found the classes on the Monroe County Library System site.”
“I took the classes.”
Comments from focus groups participants and survey respondents make it clear that training offered through the invitational grants offered through NYSL in 2004 made a difference. The consultants believe that ongoing training and staff development efforts will be necessary if the NOVEL program is going to continue to mature.
The Breadth and Depth of Resources Issue
The consultants were fortunate to have representatives from a full range of types of libraries in the focus groups. Participants represented public libraries, school library/media centers, school and public library systems and 3Rs, special libraries, 2 year-associate degree academic libraries, private colleges and universities and public colleges and universities. The question of the breadth and depth of resources that are available through NOVEL was a topic of discussion at nearly every session.
School library media specialists usually expressed their desire to have a quality “basic” encyclopedia (Grolier’s and World Book were frequently mentioned) as part of the NOVEL suite of databases. A number of individuals representing schools also expressed a desire for more resources on social issues (SIRS was specifically mentioned several times).
At the other end of the educational spectrum, representatives of large colleges and universities (as well as some representatives of large public libraries) pointed out that the NOVEL databases, while appreciated, represented only a small percentage of the electronic tools that their institutions provide for their students. Products like MasterFILE™ Select were often seen as too basic to support the research needs of their students. They expressed a desire for more in-depth resources such as MasterFILE™ Premier and JSTOR.
Public participants in focus groups also had lists of resources they would like added to the NOVEL package. In some cases, an assumption was made that all or most electronic resources that are offered by their local libraries are provided through the NOVEL program. The two offerings that seem to fall into this category most frequently are “Novelist” and “HeritageQuest.” In instances in which it was noted that these products are not part of the NOVEL package, participants thought that they should be included. Other individuals specifically mentioned auto repair manuals as being a desirable addition to the NOVEL suite of resources.
The consultants believe that the school library media specialists and academic librarians as well as current NOVEL end-users make valid points in urging the State Library to consider expanding resources at both ends of the spectrum. It would certainly be less confusing to end-users if they were not confronted by several different “levels” of similar products (such as MasterFILE™ Select and MasterFILE™ Premier). However, we also recognize that the cost of providing resources for a large audience that seldom uses certain resources is likely to be prohibitive.
In some states in which the higher level resources are provided, one argument that is offered is that doing so provides a “continuum” of electronic information resources. Students learn to use a resource during their K-12 years, continue to use the same resource in college, and on into adulthood. Balancing cost and depth of resources is obviously a difficult task. The consultants recognize that the NOVEL program, like EmpireLink before it, is solely dependent on temporary Federal LSTA funds. New York is not unique in respect to funding databases with Federal dollars; however, a good number of states have managed to make the transition to funding their programs with State dollars, thereby freeing up Federal dollars for more innovative purposes rather than ongoing operational expenses.
The New York library community needs to continue to work to convince State lawmakers that online resources are not an add-on. They are part and parcel of basic library service in the 21st Century. They ensure that children have more equitable access to a quality education, they stimulate economic growth and they enrich the lives of citizens. In spite of the fact that the NOVEL program has some shortcomings, New York State without such a program is unthinkable. A very good case can be made that in a world without NOVEL, Alabama, with its Alabama Virtual Library, would have a clear competitive advantage over New York. The same could be said of Pennsylvania, New Jersey, Ohio, Indiana, and so forth.
New York residents DO need access to more rather than less in the way of online resources. A statewide program that offers greater breath and depth of resources makes sense. It was clear from interaction with focus group participants that several different buying consortia are already in place in New York. The Westchester Academic Library Director’s Organization (WALDO), the SUNY and CUNY libraries, the New York State Higher Education Initiative (NYSHEI), and other groups share some similar goals. While cooperation does exist between and among the State Library/State Education Department and these other organizations, even greater collaboration could lead to the availability of deeper and wider resources for all New Yorkers. This is certainly a worthy goal.
Several states have taken different approaches to addressing problems that are similar to those confronting New York. It would be worthwhile for New York to examine different models that exist in Ohio and Maryland in an effort to find an appropriate solution for New York State.
Interface Issues
Many of the frustrations expressed by NOVEL users relate to the large variety of different ways that the public accesses the same databases. It is clear that the public cares little about whether a specific database is “branded” as being a NOVEL database or an information resource offered by their local public or academic library or by their school library media center. The people simply want seamless access. Many of the different mechanisms used to access the databases result in the frustrating process of re-entering authentication codes repeatedly as the user moves from one database to another.
Given the complexity of the situation in New York State (the number, size, and diversity of organizations involved), it is much to the credit of the New York State Library that the system works at all! The task at hand is truly monumental. It is also remarkable that the NOVEL help desk manages as well as it does. Again, given the complexity of the situation, it is miraculous that the limited staff available for this purpose manages as well as it does.
It has already been noted that efforts to “brand” the NOVEL program have been relatively unsuccessful. While part of this is probably due to the program’s less than descriptive name, the interface issue is also a very important one. Imagine the reaction that a public relations firm would have to a potential client that brought a product to the firm that was going to be packaged in a different array of boxes, cans and cartons of all different shapes, sizes, and colors. This is essentially the situation that exists with NOVEL. It means different things to different people and acts differently for different people depending on their point of access.
Some states have implemented a single interface or portal for their database programs and force all users to access the resources in this way. Given the large number of other databases available to New York residents from other sources (library systems, individual academic institutions, consortia, etc.), this approach would probably result in less, rather than more, use of the databases. Nevertheless, simplification of access is essential. Again, an approach that involves all of the “major” players who are involved in database licensing is recommended.
The Federated Searching Issue
The library community is clearly split on the issue of federated searching. While many end-users and some librarians want database searching to be “more Google like,” others, primarily librarians and other expert users (researchers) fear that federated searching, at least federated searching using most of the products that are now available, will prove more confusing and less reliable than using the vendor supplied native search engines.
The consultants believe that the solution to this question must be the provision of parallel search systems; a federated system for first time users and novices and an expert system for librarians and experienced researchers. The experimentation with federated searching that has been taking place has been extremely useful. A refined version of the federated searching that has been piloted should eventually be made available to all libraries. The consultants believe that many small school library/media centers and public libraries would choose to offer the federated model as the “front-end” of choice.
If this standardized front end was adopted by a majority of small public and school library/media centers, this discrete and identical implementation of NOVEL could be marketed more effectively on a statewide basis. The consultants recognize that many libraries, particularly large ones that license a significant number of databases beyond those available in NOVEL, will continue to want to offer their own front end and the searching systems that are native to the individual products. The State Library should continue to facilitate this practice; however, the State Library should also encourage libraries to include a link to the State’s federated search front end as an alternative mechanism for querying the databases.
Responses to Specific Questions Raised in the Request for Proposal for the NOVEL Evaluation
How has NOVEL helped the user, both librarians and their end-users?
NOVEL is a great equalizer. NOVEL provides New Yorkers with access to authoritative information regardless of where they reside in the state or the depth of library and information resources that are available locally. NOVEL ensures that New York residents of all ages have access to knowledge that furthers their education and access to facts that help them make more informed life decisions.
The 24 X 7 availability of the service and the fact that access is not bound to a physical location are also seen as great benefits. One end-user of the databases said “Home access is wonderful and allows me to do research when the library is closed.”
Teachers and school library media specialists cited the importance of NOVEL to their efforts to develop high-level information literacy skills. Librarians cited both the cost savings that derive from centralized licensing of the databases and the extent to which NOVEL provides local access to resources that have not been available in the past. Libraries of all sizes have cost-effective access to a wide variety of authoritative resources, which in turn, provide a great benefit to end-users.
Some end-users mentioned the importance of the NOVEL databases to their work. Examples include journalists and health care professionals. One person said “For …small businesses, it’s a tremendous resource.” Other end-users cited the significance of the databases to their personal lifelong learning objectives. One individual called the NOVEL databases a “…reliable source of … information from established, authenticated sources” that is “more reliable and quicker than aimlessly wandering around web sites.”
Are users getting what they need?
Although there were many requests for making additional digital resources available through the NOVEL program, the considerable value of current offerings was clear. One person said “I can’t begin to subscribe to all of the periodicals that are part of NOVEL.” A librarian added “They (the NOVEL databases) have subject content that our patrons need and want.”
Although NOVEL meets many user needs, information gathered through the focus groups and web survey also reveals that NOVEL does not meet all of the information needs of users. Both end-users and librarians/school library media specialists identified some additional databases that they thought should be included in NOVEL.
These requests for other resources tended to be at both ends of a “complexity continuum.” On the one hand, school library media specialists often advocated for the addition of a basic encyclopedia. Academic librarians and end-users of resources in fields such as the health sciences wanted NOVEL to offer more scholarly and in-depth content. In fact, the consultants found that public libraries and school library media centers frequently were able to offer the very basic tools through licenses secured through library systems. Nearly all of the academic institutions and most of the special libraries the consultants encountered were paying for access to some digital resources beyond what was available through NOVEL. The consultants conclude that NOVEL has successfully targeted the middle ground and that it does a good job of meeting many of the information needs of a sizeable proportion of the population.
Is the State Library communicating well with users about NOVEL?
The New York State Library has done a very effective job of communicating information about the NOVEL program to librarians. However, in spite of considerable effort, it has not been able to make significant inroads into making the general public aware of the rich resources that are available to them.
The task of making the library community in New York aware of a program such as NOVEL is enormous. Whereas, the number of libraries of various types in many states is fairly small, the number and variety of libraries in New York is great. The New York State Library estimates that there are approximately 7,000 libraries in the Empire State of which about 6,300 are eligible for participation in the NOVEL program. This includes public libraries, school library media centers, academic libraries and special libraries such as hospital, non-profit agency, governmental, and business/commercial libraries. As of the date of this writing, 5,322 libraries or approximately eighty-four percent (84.48%) of those eligible were registered with NYSL for NOVEL use. Registration involves much more than simply contacting libraries. Contact information, IP addresses for authenticating use, and a host of other data must be collected from each entity. The consultants cannot overstate the enormity of the challenge this represents nor can they compliment the NYSL staff adequately for its accomplishment in this regard.
That said, a major weakness of the NOVEL program has been the degree to which the program has penetrated the potential market for the digital resources that NOVEL provides. New Yorkers who know about NOVEL tend to be relatively sophisticated library users or those who are fortunate enough to be users of libraries in which librarians and library staff have made the promotion of the databases a high priority.
A good number of individual libraries and library systems have been very effective partners with the State Library in spreading the word regarding NOVEL. An analysis of NOVEL usage data (logons and searches) shows heavy use in areas in which libraries and library systems have actively promoted the availability of the databases. More people are using the NOVEL databases in areas that have integrated the use of the resources into their ongoing program of service than in those that have failed to do so. This finding is underscored by the fact that so many web survey respondents (44.07 %) indicated that they had found out about NOVEL from a librarian or media specialist.
The New York State Library has made and is continuing to make a concerted effort to raise public awareness of the NOVEL program. “Invitational grants” offered to library systems willing to provide training and to conduct initiatives designed to increase awareness of the NOVEL program have had some positive impact. The endeavor to provide access to the databases using a New York State driver’s license as an authentication tool is innovative and has the potential for being highly effective. Nevertheless, short of a campaign that includes mass-market advertising techniques, the user base of the NOVEL program is likely to remain a relatively small subset of the general population that could derive a significant benefit from using the resources.
It should be noted that the State Library has recently embarked on a new effort called the “NOVEL Statewide Education and Information Program.” A contract has been awarded to the Ivy Group, a marketing firm with considerable experience with libraries, to work on this effort. One of the components of this project will be a new communication plan for the NOVEL program.
Did the State Library expand resources to all users including both large and small libraries?
The consultants found that the NOVEL program has enabled the State Library to expand resources to the users of both large and small libraries. However, the manner in which resources have been expanded typically differs depending on the size of the library. End-users of very small libraries (particularly small public libraries and small school library media centers), benefit by virtue of the fact that they can access information that had not been immediately available previously. Where in the past their libraries had little or nothing on a topic or had a limited number of periodical titles, now far more extensive resources are available through the NOVEL program.
Users of larger libraries have typically benefited more indirectly. Many of these libraries either already had licenses to the databases included in the NOVEL suite or, if NOVEL was not available, would be subscribing to services such as MasterFile Select. The fact that the NOVEL program provides public access to these resources at no cost to the local library typically results in the library’s purchasing additional resources (print and media) or in the local library (or library system) licensing more in-depth electronic resources or resources not included under the NOVEL umbrella.
One library system reported that “We went from MasterFile Select to MasterFile Premier. The library system saved $160,000 in one year because of the LSTA funding for NOVEL. If individual libraries had done that it would be 4-5 times that amount.”
In the case of both small and large libraries, end-users have access to expanded resources because of NOVEL.
How has the NOVEL program benefited libraries (and their users) through cost savings?
As was indicated above, the availability of the NOVEL databases at no direct cost to the local library has freed up funding to be used for the purchase of additional resources. In fact, the NOVEL program saves libraries and end-users in multiple ways. First, the unit cost for access drops significantly when licensing is done at the statewide level. The library system quoted in the precious section estimated that it would have cost individual libraries four to five times as much to license the same databases individually. As was also noted above, libraries and their users benefit from the availability of expanded resources.
However, there are even more subtle ways in which libraries realize cost savings through the statewide licensing of the NOVEL databases. For example, one of the librarians interviewed who worked for a library system said “…many (libraries) have based their periodical subscriptions on what’s available in NOVEL.” In other words, a good number of libraries have been able to discontinue subscriptions to periodicals and journals because they are now available through NOVEL. This action not only saves the cost of the subscription, but also the costs associated with processing the materials as they are received, claiming missing items, housing back issues, re-shelving, and so forth.
Another way in which libraries have derived a cost benefit is savings related to “upgrading” from “basic” databases to “premium” databases. One librarian said “If you have MasterFile Select, the basic version, you can buy the next level (MasterFile Premier). Many libraries do that. They can build on the State’s product by paying an additional amount.”
What do non-participating libraries and borderline participants need to be able to participate?
Some of what needs to be done to enable non-participating and borderline participants in the NOVEL program to become more active represents a continuation of what has been done in the past. Training librarians, library staff, and end-users has resulted in greater use in some areas of the State. This aspect of the NOVEL program cannot be seen as a one-time effort. Mechanisms must be found to offer ongoing training as staff turnover takes place in libraries across the State. It is also essential that school library media specialists be given the support and tools they need to be effective in training teachers and students to use the NOVEL resources.
Some of the training described will probably need to be supported through grant programs similar to the invitational grants that have already been offered. The State Library should also look at other ways that libraries and schools can be urged to offer training at the local level. Examples might include the creation of a closer linkage between the NOVEL program and other information literacy efforts in the State.
Another aspect of what must be done to encourage non-participating and borderline libraries to participate involves public awareness. Greater public awareness of the availability of the NOVEL resources will lead to greater demand. Higher expectations on the part of the public are likely to drive some tentative libraries to greater activity.
The consultants believe that one of the mechanisms that might make this happen would be mass market advertising by the vendors of the databases that suggests that electronic resources are available at the library. This approach might be characterized as the “pharmaceutical” model. We are all familiar with ads that urge us to “ask your doctor about (name of the drug).” There is no reason that “ask your librarian about (name of database)” wouldn’t work as well. EBSCO has taken some steps in this direction, particularly on public radio. The State Library needs to leverage its buying power to require that the vendors provide a certain amount of advertising as part of its licensing agreement.
What next steps in regard to NOVEL should be considered for the new 5-year plan?
- The New York State Library (NYSL) must Increase public awareness of the NOVEL program (The NOVEL Statewide Education and Information Program is an important step in this direction)
- NYSL needs to develop strategies to market the NOVEL program through interest groups (historians, business organizations, environmental groups, health professionals, etc.) This is an attempt to create point of need training opportunities and “teachable moments.”
- NYLS needs to work even more closely with other consortia that license databases (such as Westchester Academic Library Director’s Organization [WALDO], the SUNY and CUNY libraries, and the New York State Higher Education Initiative [NYSHEI] to coordinate licensing efforts to ensure maximum benefit to all New Yorkers (both from a cost standpoint and depth/breadth of resources)
- NYSL needs to exert greater effort in an attempt to build a coordinated continuum of resources (create users in primary grades and provide a path that continues database use through adult years)
- NYSL needs to continue to work with partners (libraries/library systems) on training and staff development related to NOVEL databases
- NYSL, in partnership with libraries of all types throughout the State, needs to encourage State investment to expand/extend resources now licensed using LSTA funds
- NYSL needs to work collaboratively to develop a limited number of effective interfaces that can be adopted by libraries throughout the State as the primary ways to access the NOVEL databases
- NYSL needs to expand access to federated searching while continuing to offer “expert” systems
- NYSL needs to work to leverage its buying power to encourage/require vendors to include mass-market advertising as a condition of contracting with the State (NYSL should also work with the other licensing consortia to adopt this approach)
New York State Library staff members have been and continue to be active in their efforts to improve the NOVEL program. Examples include pilot projects featuring federated searching and simplified searching and improving marketing efforts through the NOVEL Statewide Education and Information Program.
Conclusion
New Yorkers are very fortunate to have the resources that are offered to them through the New York Online Virtual Electronic Library (NOVEL). While the program can be improved, and while public awareness of the database program still needs to be increased, NOVEL nevertheless manages to address all of the high level goals set for the program. While NOVEL has failed to reach all of the targets set for the program in the 2003 – 2007 Five-Year LSTA plan, the NOVEL program is much stronger and better in 2007 than it was at the inception of the five-year planning cycle. Usage has increased dramatically and the program is known and appreciated by a vast majority of the library community.
It is disappointing, yet understandable, that NOVEL is still almost totally dependent on LSTA funding. Shifting some, or all, of the program to State funding should be considered a high priority.
Much work remains to be done to address issues related to public awareness, and on streamlining public access to the databases (federated searching and front-end issues). At the same time, ongoing effort must be made to ensure that librarians and other library staff are familiar with NOVEL resources.
Back to: Contents | LSTA Evaluation Documents and Reports Page | LSTA home page | Library Development home page