A Facebook friend messaged me yesterday with a library 2.0 assessment-related question that I didn’t have the answer to, but was curious about myself. So I thought I’d put it out to the hive mind:
“[I was] wondering if you knew of any libraries who are doing a great job at assessing the impact or use of web 2.0 technologies that they’ve implemented? We’ve been doing some research, but haven’t seen any good examples yet.”
Does anyone know of any libraries that are doing real assessment work to gauge the impact of the social technologies they’ve implemented? If so, please comment or email me. It seems like it isn’t happening much in libraries.
I would bet that libraries aren’t on the social software bandwagon. I’m very much part of the social apps developer network out in the Bay Area, and I can tell you no one is thinking about libraries. I develop in the education space though, so if there is something you are thinking about in specific, get in touch.
The social software developers may not be thinking about libraries, but libraries are definitely thinking about social software.
I’m also very interesting in any responses you get.
Thanks for sharing!
I’m not aware of anything immediately available, but expect an ARL SPEC Kit in the future on social software in libraries. I just completed the survey yesterday for Univ of KY, and assessment was addressed in the survey. More about the survey: http://www.arl.org/stats/specsurveys/
Here are some of my results. They are bit old, but I can update them:
http://gathernodust.blogspot.com/2007/05/library-20-usage-results.html
I should read your book. The funny thing about the book on the right is: I’m thinking I should chat with you and pick your brain, but you’ve done all this work to put your brain in a book. So I should read that before I ask you the basics like “How do you see social software used in libraries?” What do you all think of CourseSmart and Questia?
Based on my own review in prep for teaching the Social Software course at the University of Arizona last semester, most libraries are falling short on program evaluation of social applications and programs, but there are several studies in the literature on the effectiveness of some particular things, such as virtual reference. Most of these are predominately encounter-based rather than outcome-based. I suspect most forays into social software are more or less ad hoc and underfunded, so I’m not surprised. But it’s a good question to raise. I had the class spend three weeks or so looking at literature on project management, the Logic Model, outcome-based evaluation and related topics. Their final project needed to include an evaluation plan, as most grant-funded programs (even from foundations) require it.
If you want, you can follow my work. It is actually how you define Web 2.0 technology.I am on a dissertation mode developing a Design, Development and Evaluation of a Library’s Enquiry Automatic Feedback (LEAF) system. The product is up and running on the library site. It is just now left with the documentation “how I customised and deliver such product into a library environment.” This LEAF system has a little bit of business intelligence, knowledge based, able to move information into social sites (like facebook and friendster, CRM and marketing tools. In my paper, I will also just justify it the level of acceptance. You can check it out on my site, just click thesis cat. There are also other library projects, that I am developing, such as Business Library Blog (Corporate Style) and Gateway Page to all various library digital resource (Database 2.0). As, I am developing these, I will be doing survey and studies on the usuability portion.
At Michigan State, we’re still mainly counting tick marks when it comes to assessing our Web 2.0 efforts — individuals who are our fans on Facebook, number of photo views and contacts on Flickr (~3,000 and ~30 respectively, w00t!) — but I would love to see examples of some more substantial assessment efforts. How do we pin down impact and not just outputs?
At my small business college library we are using both a typepad blog for our library webpage and a wiki for our subject pages. So far it is going reasonably well. We are still trying to work out some bugs though.
In my research on library blogs which I just completed last month, I found that very few are assessing the impact of blog use.
I’m definitely interested in the responded you get, especially the “how” aspect. At my library we’re using usage stats. That’s how we measure the success of our other endeavors like programing (how many people show up). I’d really like to see us asking people how they found out an event, etc. Our ventures in social networking are fairly new, so it’s hard to say the impact it will have over time.
I imagine that when most libraries set up social network profiles, blogs, wikis, etc. they do so in experimental mode. There is no real design process involved. It’s more of a “let’s put it out there and see what happens”. Therefore assessment would be difficult if not impossible. You generally can’t assess something if there were no outcomes from the start. What is there to measure? If we stated our outcome as “reference questions will increase twofold and the new activity will come from facebook links” (not just user stats – but what made a difference)or “english 101 students will report improved academic success in response to a semester-end survey as a result of using the library’s wiki” then you could conduct assessment. It’s no different from information literacy. You need to start with outcomes that are measurable. Then, you just might be able to assess whether or not the outcomes were achived. Otherwise, it’s all guesses and anecdotal evidence (if that).