Tuesday, December 7, 2010

should academic libraries default conservative on discovery systems?

I've read two recent articles that each really draw the same conclusion: in many cases, libraries are better off sticking with proprietary, legacy (in terms of technology) products to support end-user services.

The first, "Next-Generation Library Catalogs and the Problem of Slow Response Time" takes a look at an important issue, the user experience with next-generation discovery systems. It is written by three librarians at the University of Colorado-Denver and appears in the December 2010 issue of Information Technology and Libraries. It does have the laudable goal of creating criteria, based on performance times, that can be used to assess library discovery systems. It also makes the reader aware of tools and methods that can be used to perform performance testing.

The authors do not explicitly address the role of the underlying ILS in attempting to assess discovery system performance. The fact is that in most cases, enhanced discovery systems sit on top of integrated library systems with architectures dating back to the 1990s - or earlier. These ILS products may lack standards support needed for data interchange between systems. The authors note that the University of Colorado Denver employs the Innovative Interfaces Millennium system. Marshall Breedings's Perceptions 2009 survey notes that Innovative's own customers, while giving the vendor generally high marks, are concerned about Millennium's "perceived closed architecture."

I have managed OCLC WorldCat Local for Washington State's primary discovery system for about 15 months. In my experience, the most likely culprit in lag time for WCL detailed record display presentation is the interoperability between WorldCat Local and the integrated library system. For a system like Millennium, that will likely be accomplished via screen scraping, due to a lack of workable standards support at the ILS level. It is an resource-intensive operation and my institution's upgrade of its integrated library system server in January 2010 had a significantly beneficial impact upon WorldCat Local performance in the detailed record display. It's hard to make a blanket quantification, but in some cases, the performance improvement was as high as 60-70 percent following the ILS server upgrade.

Would the testing being described in the article identify the true source of a performance problem, if the focus is solely on the discovery system product and benchmarking that product?

Additionally, the discovery solutions that graded out well in the article - Innovative Interfaces Millennium Web OPAC, web search for Ex Libris (previously Endeavor) Voyager are not, in my view, viable products in the long term. Millennium is simply too proprietary and non-standard for the 21st century and Ex Libris' long-term commitment to Voyager is questionable.

The second, "OCLC's WorldCat Local versus III's WebPAC: Which interface is better at supporting common user tasks?," is written by Bob Thomas and Stefanie Buck, librarians at Western Washington University and Oregon State University. It appears in volume 28, issue 4 of Library Hi Tech. There are likewise valuable parts of this article; it provides detailed information on WorldCat Local-related usability testing. The authors describe usability testing performed at Western Washington, in which users performed tasks in the Innovative Interfaces Millennium Web OPAC interface and the WorldCat Local system.

The article begins with what I view as a false premise: "The main advantage to the WCL interface is the collapse of three tiers of discovery: our local catalog, our consortium catalog (Summit) and WorldCat" (648). The testing then heavily or entirely depends upon tasks related to returnable items. Speaking for my institution, I can say that this advantage is significant, but the key strategic advantage of WorldCat Local implementation is enabling user access to articles, books and media items, and unique special collections through a unified discovery system.

I am very suspicious of usability studies that conclude, in effect, that information silos work for our users in 2010. ("When using three relatively distinct systems (WebPAC, Summit and WorldCat), the scope of each collection is more distinct" [668].) While Washington State still employs the III Millennium Web OPAC to enable some very important user tasks (such as searching reserves and location-based searching), we present WorldCat Local as the primary discovery system, because of the strategic advantage that I described above.

Both search options are presented on the Washington State University Libs home page - Millennium Web OPAC is still available to users. In the past 30 days, over 62% of the searches of WSU's Millennium system were machine searches supporting the circulation interoperability with WorldCat Local. That and significant increases in the use of article request services suggest to me that our WSU WorldCat service is working, that users are voting with their feet. It isn't a perfect approach, both in terms of the assessment results and the work with the vendor and the user community to improve the product. But I believe that this ongoing engagement, based upon a current technology discovery product, has tremendous value for our users.

1 comment: