worldcat local usability results from OCLC

I've had the chance to read the most-recent usability results for WorldCat Local. I found this document extremely helpful, particularly since this round draws upon usability testing performed on 40 faculty, student, and library staff users at five academic institutions. It's completely logical for OCLC to perform this testing at the network level; in today's tight fiscal environment for libraries, this work is more important than ever.

Washington State University's WorldCat Local service is constantly evolving. A good example of this is enabling access to electronic resources. On this topic, the report notes that users prefer hyperlinks to an OpenURL button on the WCL detailed record display. This is problematic, because in some cases, the links provided in the OpenURL resolver are more accurate than hyperlinks that appear on the detailed record display. A good example of this at Washington State is the PsycArticles database records: the Findit! button presents a link that provides reliable one-click access to the full-text article. The links on the detailed record display, on the other hand, take the user to the journal index page for the CSA and EBSCO databases. This finding underscores the importance of my institution's current work in populating the WorldCat Knowledge Base and using its links over those scraped from the legacy Millennium library system. The study shows that users did feel comfortable in clicking on the WCKB links for electronic resources (which include the words "Full-text" and info on the provider and database).

The concept of "local" receives a lot of attention in the report and it's especially important at the WSU Libraries, where one WorldCat Local instance supports multiple campuses (although the Vancouver campus has built a campus-specific WCL instance, other WSU campuses have not). Testing clearly revealed that users tend to read "local" as the local campus and, in this context, the WCL first-level holdings presentation can be confusing. This would be ameliorated by having a WCL instance for each campus and, as a unified discovery and fulfillment system, this approach makes sense.

There is a discussion on the editions support in WorldCat Local, which is still problematic. The report describes an OCLC goal for presenting editions: the most recent, locally-held edition should be displayed in the detailed record by default.

The report also describes some label changes that have been made in the last year, on several displays, and the fact that format facet support is now available for the "all editions and formats" display. I found it useful to review these, particularly in showing the ongoing evolution of the WorldCat Local service and in understanding the motivation behind each change.

There are two areas that aren't covered in this report, and which are serious issues in terms of WorldCat Local's acceptance at research and academic institutions. The first is duplicate records. This is becoming more problematic as more institutions, like the University of Washington and Washington State University, add third-party databases to their WorldCat Local default search. This leads to more duplicate records in search results. It's my understanding that OCLC is working on a de-duplication strategy that addresses duplicates at time of record loading. I see duplicate records as a manageable problem, though, because modern search services, like Google, can present duplicate entries to users.

The second area is more difficult - the lack of any hits-in-context support. Not only does this impede buy-in to WorldCat Local, but this lack of functionality makes WorldCat Local an outlier relative to other modern search services - which are presenting information on why a record was retrieved. I'm not certain that the seriousness of the hits-in-context problem is clear to the WorldCat Local discovery team. In contrast, the report notes that known-item searching performance is a problem that OCLC is working to address.

Despite these two omissions, this report is very valauble to my institution as the WSU Libraries moves forward to enhance search and fulfillment services. Assessment is another discovery/fulfillment-related task that can, to a significant degree, be moved to the network level. This is shown by the last two summary reports created and released by OCLC.

should academic libraries default conservative on discovery systems?

I've read two recent articles that each really draw the same conclusion: in many cases, libraries are better off sticking with proprietary, legacy (in terms of technology) products to support end-user services.

The first, "Next-Generation Library Catalogs and the Problem of Slow Response Time" takes a look at an important issue, the user experience with next-generation discovery systems. It is written by three librarians at the University of Colorado-Denver and appears in the December 2010 issue of Information Technology and Libraries. It does have the laudable goal of creating criteria, based on performance times, that can be used to assess library discovery systems. It also makes the reader aware of tools and methods that can be used to perform performance testing.

The authors do not explicitly address the role of the underlying ILS in attempting to assess discovery system performance. The fact is that in most cases, enhanced discovery systems sit on top of integrated library systems with architectures dating back to the 1990s - or earlier. These ILS products may lack standards support needed for data interchange between systems. The authors note that the University of Colorado Denver employs the Innovative Interfaces Millennium system. Marshall Breedings's Perceptions 2009 survey notes that Innovative's own customers, while giving the vendor generally high marks, are concerned about Millennium's "perceived closed architecture."

I have managed OCLC WorldCat Local for Washington State's primary discovery system for about 15 months. In my experience, the most likely culprit in lag time for WCL detailed record display presentation is the interoperability between WorldCat Local and the integrated library system. For a system like Millennium, that will likely be accomplished via screen scraping, due to a lack of workable standards support at the ILS level. It is an resource-intensive operation and my institution's upgrade of its integrated library system server in January 2010 had a significantly beneficial impact upon WorldCat Local performance in the detailed record display. It's hard to make a blanket quantification, but in some cases, the performance improvement was as high as 60-70 percent following the ILS server upgrade.

Would the testing being described in the article identify the true source of a performance problem, if the focus is solely on the discovery system product and benchmarking that product?

Additionally, the discovery solutions that graded out well in the article - Innovative Interfaces Millennium Web OPAC, web search for Ex Libris (previously Endeavor) Voyager are not, in my view, viable products in the long term. Millennium is simply too proprietary and non-standard for the 21st century and Ex Libris' long-term commitment to Voyager is questionable.

The second, "OCLC's WorldCat Local versus III's WebPAC: Which interface is better at supporting common user tasks?," is written by Bob Thomas and Stefanie Buck, librarians at Western Washington University and Oregon State University. It appears in volume 28, issue 4 of Library Hi Tech. There are likewise valuable parts of this article; it provides detailed information on WorldCat Local-related usability testing. The authors describe usability testing performed at Western Washington, in which users performed tasks in the Innovative Interfaces Millennium Web OPAC interface and the WorldCat Local system.

The article begins with what I view as a false premise: "The main advantage to the WCL interface is the collapse of three tiers of discovery: our local catalog, our consortium catalog (Summit) and WorldCat" (648). The testing then heavily or entirely depends upon tasks related to returnable items. Speaking for my institution, I can say that this advantage is significant, but the key strategic advantage of WorldCat Local implementation is enabling user access to articles, books and media items, and unique special collections through a unified discovery system.

I am very suspicious of usability studies that conclude, in effect, that information silos work for our users in 2010. ("When using three relatively distinct systems (WebPAC, Summit and WorldCat), the scope of each collection is more distinct" [668].) While Washington State still employs the III Millennium Web OPAC to enable some very important user tasks (such as searching reserves and location-based searching), we present WorldCat Local as the primary discovery system, because of the strategic advantage that I described above.

Both search options are presented on the Washington State University Libs home page - Millennium Web OPAC is still available to users. In the past 30 days, over 62% of the searches of WSU's Millennium system were machine searches supporting the circulation interoperability with WorldCat Local. That and significant increases in the use of article request services suggest to me that our WSU WorldCat service is working, that users are voting with their feet. It isn't a perfect approach, both in terms of the assessment results and the work with the vendor and the user community to improve the product. But I believe that this ongoing engagement, based upon a current technology discovery product, has tremendous value for our users.