Behavior vs. belief and changing culture

At LOEX of the West this summer (a fantastic conference, btw), Joan Kaplowitz did a session where she started by asking attendees what words they associate with assessment. I won’t list the litany of negative terms that came from the audience, but I will say that the most positive word used to describe assessment was “necessary.” I remember once feeling that way about assessment. It was just something we did because we had to, not because we got any value from it. And given the types of assessments we were doing at the time (satisfaction surveys essentially), I don’t see how we could have gotten much value from it. But we could say to accreditors “hey, look! We’re doing assessment.” And at the time, that seemed good enough. Several years ago, I saw the light and finally “got” assessment. I now see it as an a tool to help us make better decisions and better serve our patrons. I see it as a tool not only to improve my instruction, but to tell the story of our instruction program. Once I internalized the value of assessment, my focus shifted from doing assessment to learning from assessment, which led to better assessment design. But can I point to what it took to get to that point? Unfortunately not.

For the Web 2.0 class I teach at San Jose State University, my students recently had to do an assignment where they critique the social media marketing presence of a library and suggest improvements as if they were a consultant. (It’s a great exercise, by the way, to allow students to really put into practice what they’ve learned about social media best practices.) They all did a terrific job, but my biggest takeaway in reading all of the critiques was the growing gulf between libraries that truly embraced social media and those that are just using social media. Libraries like the Toronto Public Library, Saginaw Public Library, Topeka and Shawnee Libraries, Columbus Metropolitan Library and University of Southern California understand the value of social media and the shift in communication strategy (along with the lowering of barriers/hierarchies between patron and library) that should occur in these spaces. And as a result, their presence in those spaces is vibrant, engaging and conversational. You can see how the library is connecting and how their patrons are responding. There is real personality, humanity and humility in their presence. But many libraries are using Facebook, Twitter and blogs solely as one-way communication media and their communication strategy in those spaces seems to mirror that on their website. Consequently, while their presence has plenty of news and they may even have a good number of people who’ve friended or followed them, their space feels kind of empty. There are no comments, there’s no conversation and there’s no humanity there. Lots of libraries use social media because they know their patrons are using it, not because they value the sorts of connections they can make with their patrons via these technologies.

You can see this same dichotomy in the use of social media in teaching and learning. In my article, “Participatory Technologies, Pedagogy 2.0 and Information Literacy,” I write —

in many cases, these technologies are deployed within a traditional educational context. Some case studies showed that the deployment of participatory technologies did not result in increased collaboration amongst students, likely the result of not adopting pedagogies that encouraged participation, supported collaborative learning and facilitated the creation of knowledge communities… Participatory technologies are not transformative in and of themselves. If a class is still largely focused on a hierarchical model where content from their instructor and his or her views are considered more valuable than student contributions, technologies like blogs and wikis will not create true collaboration… In order to reap the benefits of participatory technologies, an instructor must create an environment in which collaborative learning can more readily occur.

It’s one thing to use social technologies in a class, another entirely to embrace a pedagogical model that truly unlocks the affordances of social technologies. And while there’s nothing wrong with using social technologies to engage students or as a system for easy content creation, without a change in pedagogy, the technology will not magically change the classroom environment. Similarly, a library having a blog or a Facebook page isn’t suddenly magically cooler or more connected to its patrons.

What do all of these anecdotes have in common? They illustrate the gap between doing/using something and internalizing its value. Changing behavior is shallow; it doesn’t necessarily equal a change in culture and will not necessarily result in real organizational change. I’ve been doing a lot of research on what it takes to build a culture of assessment and a big part of that has focused on the business literature related to organizational change and culture change. And what I’ve found is that there is a huge yawning gap between changing behavior and changing culture. It’s not so difficult for administrators to force librarians to do assessment, but it doesn’t mean they will do it well or get anything out of it if they don’t grasp the value of assessment. And while the value of assessment seems obvious to those of us who are assessment geeks, I remember being on the other side of that equation where I actually saw assessment as something we had to do for reasons totally external to student learning. I found a lot of works in the literature that addressed the need for changes that impact the organizational culture or described the type of organizational culture that is high performing and embraces change, but few that actually suggested ways to get there. The three that really spoke to me were —

• Kotter, J.P. (1995), “€œLeading change: Why transformation efforts fail”, Harvard Business Review, Vol. 73 No. 2, pp. 59-67.
• Deardorff, M.D. and Folger, P.J. (2008), “€œMaking Assessment Matter: Changing Cultures, Improving Teaching, and Transforming Departments”,€ Annual meeting of the American Political Science Teaching and Learning Conference, San Jose Marriott, San Jose, California, Feb 22, 2008
• Allen, J. (2007), “Introducing Assessment: Strategic Adaptations and Early Successes”, Assessment Update, Vol. 19 No. 2, pp. 5-7.

Kotter explores the incremental steps needed to build excitement for and buy-in for change and to embed that change in the organization’s culture. It’s an amazingly practical and brilliant model for change and inspired me to read his book that was inspired by the HBR article.

While lots of the literature of libraries and higher ed talk about this ideal culture that is needed in the organization to develop a culture of evidence, Deardorff and Folger write about ways to work towards building a culture of assessment even in the absence of that culture.

Instead of having faculty focus on creating perfect assessments or assessing things that might be seen as daunting, Allen suggests that to get people started with assessment (after which they will hopefully recognize its value) they focus on things that are easier to assess. This “perfect is the enemy of good” article really resonated with me.

So many of the things I read in the library literature basically suggested that without the ideal culture, building a culture of assessment simply isn’t possible. Which would mean that the vast majority of libraries are basically out of luck. That’s why I gave the talk I did at LOEX of the West on Building and Sustaining a Culture of Assessment in Your Instruction Program which explored the resources, administrative support, grassroots buy-in and change leadership needed to anchor assessment work in the library culture. My slides from the presentation are below and I recently submitted an article I wrote on the topic for peer-review (so hopefully it’ll be out late this year or early next year).

I’m simply not willing to believe that libraries that don’t exhibit that perfect culture described in so much of the library literature about assessment can’t achieve a culture of assessment without completely changing their culture. It certainly is easier for libraries with the sort of culture described by Lakos and Phipps (2004) to achieve it, but I do believe it’s possible to create a culture of assessment simply by getting librarians and administrators to internalize the value of assessment. But doing that is easier said than done. Still, I suspect it’s easier to do that than to change the culture.

But how do we get there? In spite of having read an MBA’s-worth of business articles, I still feel like there’s so much mystery surrounding what it takes to get an individual to internalize the value of a change; to make that change more than just behavioral. It’s happened to me and even I can’t articulate how it happened, nor can I foster it in someone else. Even Kotter seemed the least clear in his discussion of anchoring a change in the culture. If anyone has found a brilliant article on how to get people to internalize the value of assessment/social media/change/etc. or has been able to achieve it themselves, I’d love to hear about it!