By Meredith Farkas | September 7, 2008
The part of my still new-ish job that causes my the most worry is assessment. I’d hoped to have time this summer to do some serious research on information literacy assessment and get some good ideas for how to best assess library instruction this Fall. However, I got too busy with other, more pressing, tasks to spend the time doing any research. The only thing I went into the semester knowing for sure was that I wanted to change the way we did assessment, especially in our English 101 sessions.
I think my Director would like to see assessment that is consistent across instruction sessions. There are good reasons for this. For accreditation, it’s good to show that you are using consistent measures of instruction. Also, how will you really be able to compare the effectiveness of an English session with a Business session if the measures are different? However, I worry that creating any assessment that is going to work for everyone will end up not giving us as meaningful results. Last year, we used an assessment tool for English 101/102, and in the social sciences, architecture and engineering. It asked students to rate us on the following statements:
- The purpose of this instructional session was explained to me.
- The information presented was clear and well organized.
- There were opportunities to ask questions during the session.
- I will be able to use these research tools by myself.
with a strongly agree to strongly disagree scale. Looking at the assessment stats for last year, only a very small percentage actually chose “disagree” or “strongly disagree” on any of the questions, leaving us with very little useful feedback. There was a final question that asked for open-ended feedback, but we almost never got any. It is nice that we can look at most of the classes taught last year on a single spreadsheet and see what students thought of them, but looking at those stats tells me absolutely nothing about what I (or any of my colleagues) should be doing better or covering in more depth. It also doesn’t give me any sense of whether they absorbed what I taught them.
My big push for this year is to incorporate active learning into every session we teach. To that end, I created a worksheet that students are meant to complete during the class session(s) we’re teaching. The worksheet contains things like an area for students to brainstorm keywords for their topic, a place to record their search strategies and what the results were, and a place to record useful books and articles they found. In class, we’re supposed to go over a topic, like finding books, and then give the students time to complete that part of the worksheet with our help. Then we go onto the next topic and give them time to practice those skills. I think it’s helpful for them to practice the skills we’re teaching during the session, because I know how easy it can be to zone out when someone spends an hour lecturing. But when you’re forced to actually do something, the chances of remembering what you learned later on are much greater. Also, the worksheet can act as a “roadmap to research” later on, reminding them of what they’ve already tried when they’re in the thick of their research.
The worksheet template is being used in all EN 101 sessions and all of the social science sessions this semester. It’s just a template and it can be tweaked to fit the topics being taught. For example, I recently taught a session that covered creating search queries, finding tertiary sources, and finding peer-reviewed articles in History. For that, I got rid of the section on finding books and added a section on reference sources that was appropriate to what I would be teaching. As a result, each class will probably have a slightly different worksheet, because we’re preparing them for different assignments and, thus, should be teaching different skills.
Even within English 101, it’s difficult to create any sort of consistent worksheet for students to use. My predecessor’s goal had been to create a consistent list of things that we would teach in every EN 101 session. It was a good goal, but difficult to do when each instructor assigns a very different research assignment. I’ve seen English 101 classes where students had to find peer-reviewed articles on environmental topics, where students had to find articles related to topics discussed in Flowers for Algernon, where students had to research and write about a controversial issue, and more. And the resources they were required to use also varied widely. With so many different topics, it’s almost impossible to cover the same set of resources in every EN 101 session. I know there’s a push here to standardize the EN 101 curriculum, but until then, I think we have to teach to the assignments students have.
A colleague of mine gave me the idea of using the worksheet as an internal assessment tool as well. We could grade the assessment using a rubric I created and then record each student’s score on each of the individual questions on a spreadsheet. That way, we could see not only how well the students in the class absorbed what we were teaching, but what specific topics they had the most difficulty with. If there is a certain question that the average student score was very low on (for example, justifying why an article is of sufficient quality to use in their paper), we would know that we need to cover that topic better next time. It should really give us a sense of our effectiveness as instructors and what specifically we need to do to improve.
Obviously, if we all have somewhat different worksheets, we won’t be able to compare the scores exactly from one class to the next. Hopefully, each worksheet will contain some common elements (like, hopefully each person is teaching the students how to use library databases), and we can compare those scores. But, what I think will be nice, is that we’ll be able to see where, in a given session, we didn’t get through to the students.
I know consistency is important, but what I really hope to accomplish this year is to get some measure of our effectiveness as instructors. And I just don’t know if any single tool will help us to learn that. I want to know that we’re really giving the students the skills that they need to do research for their assignments. Then, once we know we’re ok on that, I’d like to focus on creating more consistent assessment. But I really think that making our instruction work effective is far more important than measuring everything in the same way.
I’m curious how other libraries assess individual information literacy sessions. Does every instructor use the same assessment tool in every class? Are there specific assessment tools for specific disciplines? Do you assess students satisfaction or student learning? Do you feel that the assessments you get back are instructive to you?