The Failure of Middleware, Part 3: How do we measure up?

So now that we know what our population wants, we can see how our OPACs measure up. I created a screencast using my alma mater’s OPAC to illustrate some of the challenges users have when using catalogs. Obviously some catalogs are more sophisticated or offer more flexibility in searching, but some are even more difficult to use. In this screencast, I pretend to be a student who has to write a paper about The Brothers Karamazov. She is looking both for the book itself and for books about The Brothers Karamazov. Let’s see how she does. [watch the Flash movie]

So the first problem we see here is that searches are left-anchored. Young people these days are not accustomed to those limitations, so if they put the authors name first in a query and get a zero result, they might actually think the library doesn’t have any of the author’s work. The next problem is the fact that some catalogs, though fewer, still don’t ignore articles and will come up with a zero result if you enter a, an, or the. Google and Amazon have no problem ignoring articles in a search, so why can’t our catalogs? While a savvy user might realize that a University library must have a copy of The Brothers Karamazov, some might give up when they get zero results.

Then we have the subject search. The vast majority of students know nothing about Library of Congress subject headings. Subject searches are really dependent on users entering the right term, because there can be a number of ways to describe an idea or a concrete object. Many users do not understand that they cannot just enter any term. While most systems also have title keyword and subject keyword searches, most users are not going to distinguish between those options and the default subject and title searches. They’re simply going to search using the default subject or title searches, which require an exact title (or at least the beginning of one) to be entered.

Traditional OPACS require a lot from users. They expect that users understand how search engines work. They expect that users can properly articulate their information need in words and can properly spell those words. They expect that users understand Boolean logic, when study after study has shown that Boolean searching is counterintuitive to users accustomed to Google and Yahoo.

The truth is quite different. According to a recent Penn State protocol study [Novotny, E. (2005) “I Don’t Think I Click: A Protocol Analysis Study of Use of a Library Online Catalog in the Internet Age.” College and Research Libraries — password required] where users were asked to find materials on certain topics in the OPAC, the author found that:

• “Both experienced and novice users demonstrated a somewhat hazy conception of what is in a library catalog.” They didn’t quite know what they’d find in the OPAC, if it was just books or electronic journals as well.
• “Although they were occasionally puzzled or frustrated, they rarely took the time to explore why the results were unsatisfactory.” Users would just try something else rather than figuring out what went wrong.
• Most users do not search for synonymous terms. Meaning that if they search for teen smoking, they might not search for adolescent smoking as well. They think synonymous terms will always be mapped to one another.
• Users often click on links (usually the LC Subject Headings) as they are accustomed to in search engines. And this rarely leads to a satisfactory result, as anyone who’s tried it will attest.
• Users do not search systematically. It’s very much a berrypicking model. They will try one thing and when that doesn’t work they will try something else, but there is rarely a method to their madness.
• Users stated that they would like a “find related” button when they found a resource that met their needs. Then all they need to do is find one useful resource to get to related ones rather than continue searching.
• Users don’t want to know how the system works, they want results. We librarians might like searching. We might like the chase. For users, the catalog is a means to an end. They want to satisfy their information need and that’s it.

We really need to ask ourselves if users should need to understand how library OPACs search the collection. If so, then the failing is the users’ and they simply require education. Outside of the library world, the goal of information retrieval systems is to make the underlying architecture invisible to the user – in the sense that the user should not have to know anything about search strategies to use them. Why should it be any different in libraries?