ICT Assessment, the Librarian’s ViewICT Assessment, the Librarian’s View

Eileen Stec, Instruction & Outreach Librarian, Rutgers University

April 18, 2007

What are our choices?What are our choices?

Multiple choice or similar questions ICT online assessment or print assignments/in

class presentation using authentic tasks Electronic or analog portfolio

Answer this questionAnswer this question

1. What is a list of books, journal articles, or other materials about a certain topic?

CHOOSE ONLY ONE ANSWER.BibliographyKeywordLibrary catalogResearch database Subject heading*

*Question from Project SAILS Validation Phase III

Answer to question 1Answer to question 1

What is a list of books, journal articles, or other materials about a certain topic?

CHOOSE ONLY ONE ANSWER.

Bibliography (correct)KeywordLibrary catalogResearch database Subject heading

SAILS multiple choice assessmentSAILS multiple choice assessment

Pro’s Simple to score Easy to compare

individuals or cohorts Lower cost to build &

administer than ICT Designed using ACRL

Information Literacy Standards

Con’s Proof of memorization

skills Can’t be generalized to

real-world tasks Disagree with many

answers

Authentic Tasks allow assessment ofAuthentic Tasks allow assessment of

“Lower Order” thinking skill:

Outcome 2.2.a. Identifies keywords, synonyms, and related terms for the information needed.

“Higher Order” thinking skill:

Outcome 3.3.b. Extends initial synthesis, when possible, to a higher level of abstraction to construct new hypotheses that may require additional information.*

*The Information Literacy Competency Standards for Higher Education.American Library Association, 2000.

Authentic tasksAuthentic tasks

Authentic tasks can be created electronically, as they are in theICT assessment:http://www.ets.org/ictliteracy/

OR…

Authentic TasksAuthentic Tasks..in print… IRIS Online Catalog Assignment.  1. Briefly describe your topic __________________________________________________2. List below, keywords you would use to locate a subject heading. (Method 1.)____________________ _____________________ ____________________________ 3. Use any of the three subject heading search methods demonstrated in the Handbook to locate a useful book you can check out on your topic. As you do so, place an X in front of all the methods listed below that were helpful:___ Method 1. Keyword to find a useful subject heading.___ Method 2. Browsing alphabetically for a subject heading.___ Method 3. Finding hidden treasure: Using a book to help you find other books. 4. Provide information about the book you located and checked out:Title ___________________________________________________________Call Number _____________________________________________________Library that “holds” the book ________________________________________A helpful subject listed in the book’s record ____________________________ 

Authentic TasksAuthentic Tasks

With a rubric:

Very Good. All of the conditions in “Acceptable” are met, however, this student has provided additional information for the assignment, e.g. located more than one book, shown evidence of thoughtful concepts culled from reading the book, provided additional subject headings or narrower/broader keywords than required.

Acceptable. Each question (1-6) contains at least one correct answer and the book the student reports has the status “Checked Out”

Unacceptable. Not all of questions 1-6 are answered or the information seems “phony”

OR

The book the student was supposed to check out of the library is on the shelf or is a non-circulating book (Reference, etc.)

Portfolios – Authentic tasksPortfolios – Authentic tasks

Work samples anchored to performance objectives

Work samples collected during the process of instruction (no special tests created)

Regular assessment using a rubric, student responses evaluated and scored*

*The Systematic Design of Instruction. 2001. Walter Dick, Lou Carey & James O. Carey.

ICTICT Pro’s:-Can access partial

understanding.-Real world scenarios.-Assessment validated.-Designed using ACRL

Information Literacy Standards Con’s:-Expensive to construct.-Core level low on library-like

tasks.

Authentic –home grownAuthentic –home grown Pro’s:-Inexpensive to develop.-Real world scenarios;

individualized. Con’s:-Very time-consuming to

score.-Assessment not validated.

IRIS Online Catalog Assignment.

 

1. Briefly describe your topic __________________________________________________

2. List below, keywords you would use to locate a subject heading. (Method 1.)

____________________ _____________________ ____________________________

 3. Use any of the three subject heading search methods demonstrated in the Handbook to locate a

useful book you can check out on your topic. As you do so, place an X in front of all the methods

listed below that were helpful:

___ Method 1. Keyword to find a useful subject heading.

___ Method 2. Browsing alphabetically for a subject heading.

___ Method 3. Finding hidden treasure: Using a book to help you find other books.

 

4. Provide information about the book you located and checked out:

Title ___________________________________________________________

Call Number _____________________________________________________

Library that “holds” the book ________________________________________

A helpful subject listed in the book’s record ____________________________

Portfolio’s - AuthenticPortfolio’s - Authentic Pro’s-Individualized and real world

assessment.-Rubrics, properly tested and

applied can compare one students’ work to another.

Con’s-The most time-consuming of all

assessments.-requires work over time, not a

one-shot assessment

ResourcesResources

ACRL Information Literacy Standardshttp://www.ala.org/ala/acrl/acrlstandards/

standards.pdf Project S.A.I.L.S.https://www.projectsails.org/ ETS ICT Assessmenthttp://www.ets.org/ictliteracy/