Comments on: Consistency or effectiveness in instruction and assessment

By: Stephanie R.

Stephanie R. — Tue, 23 Sep 2008 23:29:13 +0000

I think you have to decide what you want to assess. Are you assessing the students’ perception of the quality of instruction, in terms of performance? Do you want to find out if they feel more comfortable using library resources/talking to a librarian? Or are you trying to see if you taught them a discrete skill? After you know what you want to find out, then you create an instrument. I just read this really excellent book, Radcliff, C. et al (2007).A practical guide to information literacy assessment for academic librarians.

By: Mary Ellen P.

Mary Ellen P. — Wed, 10 Sep 2008 20:40:55 +0000

I am not in a career which requires the sort of massive assessment that you are talking about. But I have taught a few courses and I have lot of opinions about surveys and assessments.

It impossible to get really useful results from surveys without some varied and open ended questions. You can have a set of 5 to 10 closed questions that are the same in every class, and which can be used for statistical purposes to report to bosses. For the rest have open ended and varied questions. The most useful to me as an instructor are questions like: Did you learn A, B, C, and D. When students say they didn’t, I can emphasize that material more.

I remember taking a survey about health that was done by the local county health services. One of the questions was along the line of, “Did you know that walking 10,000 steps per day can improve your health.” On NPR just a month earlier a reporter had said that this statistic was used by pedometer makers to sell the devices. The statistic was based on no research. But there was no answer to the question that equated with, “I disagree with this question.” Closed-ended questions will always have this sort of problem that they cannot reflect the perceptions of people who don’t see the world the same way as the people writing the questions.

By: Sandra R.

Sandra R. — Tue, 09 Sep 2008 14:23:35 +0000

I agree that this is a very difficult task, but one that most of us librarians have had to grapple with a lot recently. I have used a combination of surveys and pre- and post-tests. The problem with the tests, is that the faculty have to be on board with this project. So, it is pretty much ad hoc with those faculty who agree to participate. I do seem to get some good information out of these assessments, but some of it does not really tell me anything insightful about student learning or our teaching. I think the ideal situation, is for the administration to mandate an information literacy test for all freshmen upon entering college and another test to test their competency as seniors before graduating. How many college/university adminstrations would support this kind of thing at this point is questionable. But I see it as setting baselines and the best way of measuring students learning of info. lit. skills.

By: stevenb

stevenb — Mon, 08 Sep 2008 14:42:54 +0000

Take a look at Chapter 5 in Joseph Matthews book “Library Assessment”. There is a section on IL assessment that you may find helpful. Possibly different from your own accreditor guidelines, Middle States recommends using a variety of methods to assess student learning – not just one. So pre-test/post-testing, portfolio analysis, citation review, rubrics, etc. are all possibilities. I always found this publication to be a good source of understanding and examples:
http://www.msche.org/publications/Developing-Skills080111151714.pdf

Assessing student satisfaction and student learning would be quite different. The latter is much more difficult, but of much greater value to the library and institution. I would often use pre/post testing with courseware (e.g., Bb assessment tools) to learn not only if students were retaining information learned in my sessions, but to fine tune my content. By analyzing the results to each question I could see what the vast majority were getting right, what they getting wrong – and why they could be getting it wrong based on their wrong answers. This always helped me to revise my sessions by dropping out content they seemed to easily “get” and increasing time spent on concepts they didn’t get. Good luck with your assessment efforts.