As I was working on the book and it started getting longer and longer… and longer, it became clear to me that I was going to have to cut some topics that I’d planned to cover. My book proposal included a few things that one may not define as social software but that I thought fit into the larger dialogue about the Read/Write Web and the evolving view of how Web pages/applications/middleware should be designed for our users. As my word count got longer and longer, some of those things had to be left out. One of those was Web services. I had written a chapter on Web services and did an interview for the chapter with Casey Bisson, who does amazing things with liwbrary middleware (see his WPopac and an explanation). And while I was not particularly sad to see my stunningly inadequate description of Web services go by the wayside, I was very sad that people would not have the opportunity to read Casey’s insights into why our systems suck and what Web services could mean for libraries.

So after talking with Casey last Friday night, he e-mailed me and let me know that I could blog the interview. So here it is!

MF: How are our systems broken?

CB: Our library systems are broken in three ways: usability, findability, and remixability. We’ve been talking about usability for some time now, it’s the source of the hugely famous lipstick on a pig quote (proper thanks go to NCSU’s Andrew Pace for giving us the appropriate words), and it’s a reflection how little we’ve used technology to improve the search experience since we created our systems as a near copy of paper-based card catalogs 30 or more years ago.

Part of this is that our patrons have growing expectations of self-service (and pre-established information search behaviors as a result of growing internet use), but our systems have developed in different directions. The challenge is to find ways to invest our online services with the value libraries bring to in-person services. Allowing comments (and pingbacks and trackbacks) in our catalogs allows us to move some of our discussions online (where they can be indexed and found by search engines).

In an online world driven by search engines, findability means making our services findable by patrons who don’t yet know how we can help them. Search engines offer a huge opportunity to meet needs of the growing number of internet users, but libraries are underrepresented there because our systems are largely invisible to their crawlers. Building systems that can easily be linked to means that our users can exchange those links wherever they are online. Students can link to their favorite book in their Facebook profile or in their LiveJournal, and we adults can use those links in our research, our reading groups, or elsewhere. Those links tell search engines and their crawlers what online resources are most valuable, and building systems that can be linked to and indexed makes libraries more findable online.

The third problem we must address is the remixability of our content. The biggest lesson of the past few years of development on the web may be that those who control or provide content don’t necessarily know best how it could or should be used. Every organization has limited resources and so cannot pursue every new idea about how to use or present the content they control. Enabling remixability by offering web services and employing broadly supported standards, like RSS and OpenSearch, opens the door to allow passionate individuals and creative organizations — the same forces that have given us most of what we now recognize as the web — to develop those new interfaces to that data. Amazon, in particular, has been enormously successful with this and now claims over 140,000 registered developers building applications that vary from ecommerce websites like MasterWish.com to personal library catalogs like LibraryThing.com. The technology that enables this is called web services, and they offer the rather revolutionary opportunity to separate the applications that store and maintain an organization’s data from the applications used to display and manipulate it.

MF: What could web services mean for libraries?

CB: Libraries face the challenge of trying to integrate information from a growing number of applications and sources. Alongside the catalogs we maintain in our ILSs we’ve added licensed databases from a number of providers and many of us keep special collections or digital archives separate in specialized databases. Our reasons for this are clear to us, but difficult to explain to our patrons, who must learn not only how to select which search box is most appropriate for which question, but also how to use the extended features each vendor may (or may not) provide.

“Integration,” as it turns out, means more than simply putting a common header and footer on each page of each application, it means making all of those many applications work in similar ways. How frustrating must it be for the patron who learns that the “book bag” she has been maintaining in the OPAC isn’t connected to the “my research” collection in the database she’s been using, and more frustrating yet to learn that the bookmarks she’s been collecting in her browser all this time no longer work because none of our services allow bookmarking that way.

Web services offer a promising solution to this. By separating the data in our ILSs and databases from the OPACs and other applications we use to display and manipulate it, we create an opportunity to advance the state of each separately. A search and retrieval application could be built or offered by a vendor that allows us to search from all services — our ILS, databases, digital collections, etc — using one interfaces and one set of features to learn. This search/retrieval app could include comments/pingbacks/trackbacks, browser bookmarkable permalinks, indexability by search engines, and all the other features we’re coming to expect in our software today — without regard for which database the content was housed in (so long as they’re all web services accessible).

MF: What are the barriers to improving our library systems?

CB: Our efforts to improve our online services are burdened by the enormous costs of trying to work with systems that weren’t built for remixing. Allowing patrons to comment on our catalog items in the OPAC means waiting for vendors to offer that enhancement, to hack our catalog enough to bolt comments on, or to completely replace our OPACs. And if we do find a way to hack in a comments in our OPAC, we take on the costs of having to secure it, build an administration interface, and build a mechanism to combat spamming. The problems with building and maintaining a comment system aren’t unique to libraries, but the complexity of our existing systems makes it difficult to apply a generalized solution. So instead, we end up investing development time on problems that are common to all online services, instead of problems that are unique to libraries. Worse, the work we invest with one system often can’t be easily applied to other systems because each has it’s own hurdles and barriers.

Another issue is the size of the library community and our ability to support programming efforts. Much of the success we’ve seen with online services outside the library community is based on standards that are uncommon within libraries. Though quite a number of the 140,000 registered Amazon developers could offer tips and advice — or even entire applications — to those within the library community, we can do little with it because our standards isolate us from their work. Outside libraries, Amazon’s web services are the de facto standard for exchanging bibliographic information, but within the much smaller community of library programmers, we must face dozens of metadata standards (MARC, MODS, METS, DC, the list goes on and on) and a handful of application interface standards (z39.50, SRU/SRW, vendor specific standards, etc) to accomplish the same tasks.

The OpenSearch standard extends RSS with features necessary for searching, like a means to return suggested alternate searches (as for spelling corrections) or faceted search refinements. In the year since its release, over 300 OpenSearch targets have been announced and now it’s being integrated into Internet Explorer 7. The rapid development of OpenSearch reflects the general interest both inside and outside libraries to solve the difficulties of making information systems work together; the opportunity here is to find more common ground with those outside libraries and solve more problems together.

You just can’t help but love this guy!