more than “just a number”: assessment & collaboration in a first year experience...
TRANSCRIPT
More than “Just a Number”:
Assessment & Collaboration in aFirst Year Experience (FYE)/Information Literacy Program
Indiana University Libraries Information Literacy ColloquiumAugust 3, 2012
Presenters:
Hilary WrittBelinda Yff
Session Objectives:“What Will I Learn Today?”
Methods for successful
faculty/librarian collaboration.
The usefulness and relevance of
selected evaluation tools.
Some tools that can be used in assessing the
outcomes of an IL program.
The Beginning…
A decision was made to incorporate more information literacy skills into the College Success Skills course.
(Louisville) 1999: Opening of the new Sullivan University Library.(Lexington) 2001: Move to a new campus/improvements to the Learning Resource Center.
FYE 101: Goals and Objectives
Retention “The goal of FYE
101 is to “enhance student persistence… by addressing the academic, planning, and personal needs of students through group and individual
guidance.” (Dr. Daniel, 2007)
The goal of the library component is for students to effectively use library resources and conduct research so they can be successful in
their future classes, leading up to graduation.
Ideally, all 1st quarter students take FYE 101: Information Literacy.
Collaboration of Librarians and Faculty: Mutually Beneficial
Librarians ‘subject experts.’
Assist faculty. Increase student
recognition of library staff.
Shows that the faculty value the
library and information
literacy.
Makes implementation and assessment
possible.
Synergetic relationship that
filters to other programs.
Methods of Collaboration
Email MeetingsMentoring
& Liaisoning
Web resources
E-mails
Informal e-
mails
• Alert to recent website or database changes• Schedule class visits or tours• Clarify any questions
Formal
e-mails
• Submit longitudinal data • Deliver necessary start of quarter and end of
quarter instructions • Schedule campus meetings
Meetings
Informal
Formal
• All-campuses
• Present longitudinal data
• Discuss goals & objectives
• Introduce and evaluate future ideas (e.g. online testing, new assignments)
• Create time for faculty to share experiences
• Give updated materials to faculty for the quarter
• Revise assignments
Liaisons & Mentoring
• Librarian liaisons with General Education faculty. – The Louisville and Lexington campuses
have a dedicated FYE librarian who works with an FYE faculty coordinator.
• Partnership with new student mentor program. – Lexington librarian member of new student mentor
committee– Student mentors visit FYE classes
Web Resources: Information Literacy Page
Information literacy terms like: scholarly journals, invisible web, and Boolean operators
Library handouts and worksheets Supplemental activities
Examples of Collaboration
Librarian participation
in class
Scavenger hunts/
Library visits
Handouts/ Assignments
Library Participation in Class
INVO LVE M E
NT
• Librarians invited as guest speakers. • Orientation to library resources, instruction about databases, internet, library
resources.
ASSISTANCE
• Especially, for new instructors, librarians are resource persons. • May teach the library (information literacy) component if needed.
PARTIC IPATI
O N
• Create/participate in active learning activities. • Examples: “Library Olympics,” Dewey Decimal Races, Database Jeopardy,
Activity Supplements Folder, Were You Hoaxed?
AVAILABIL IT
Y
• Students and faculty can always reach out and/or work with a librarian outside of class.
• Research “coupons.”
$100
$200
$300
$400
$500
$100
$200
$300
$400
$500
$100
$200
$300
$400
$500
$100
$200
$300
$400
$500
“Other Databases” ProQuest EBSCOhost Where to Go
G
O
O
D
L
U
C
K
!
True or False?You cannot access the
electronic databases off-campus.
What is…False?
“Where to Go” for $100
Library Participation: Research Coupons
~~~~ FREE RESEARCH HELP~~~~
Present this coupon at the LRC Reference
Desk to get free research assistance.
NO EXPIRATION DATE
~~~~ FREE RESEARCH HELP~~~~
Present this coupon at the LRC Reference
Desk to get free research assistance.
NO EXPIRATION DATE
~~~~ FREE RESEARCH HELP~~~~
Present this coupon at the LRC Reference
Desk to get free research assistance. NO EXPIRATION DATE
Scavenger Hunts
Students explore the library on their own and complete a worksheet.
Individual
Surviving the Library
The Library Mystery Tour
Group
Gain knowledge about the library, services, and resources. Lessen “fear” of
the library.
Complete a
worksheet
Introduce themselves to a library
staff member
Go to the different
areas of the library
Scavenger Hunts - Individual
Welcome!• How well do you know your library?
• Can you locate print information quickly?• Or… is it a JUNGLE OUT THERE?
Accuracy and speed count in…
SURVIVING THE LIBRARY!
Scavenger Hunts - Group
“The Disappearance of Chef Ally Gratin”
Development of Handouts/Exercises/Assignments
Developed from information literacy
objectives and student needs.
2 different sets of assignments/exercises
Available in print and online
• Recognized Information literacy standards
• Overall FYE 101 objectives
• Alternate between quarters
• Convenience of faculty/students
• Online version on library’s website
Development of Handouts/Exercises/Assignments
Development of Handouts/Exercises/Assignments
Collaboration on activities is not always successful. But… it’s important to try!
• Examples of ideas that did not “quite work out”:
If at first you don’t succeed…
Group Wiki Activity
Independent Study Groups
Budgeting Component
Beginning of Quarter Visits
Assessment
Pre & Post Tests
Objective
SubjectiveEvaluation Surveys
Testing. . . 1, 2, 3
Objective 1: Search abilities
Objective 2: Ability to use evaluation
guidelines
Objective 3: Skill with the SUS
website (catalog and DDC)
Objective 4: Understanding
differences between Internet, databases,
and print
Objective 5: Ability to identify popular
and scholarly resources
Objective 6: Ability to demonstrate proper usage of etiquette
and protocol regarding plagiarism
Each question is aligned with one of the six objectives
Administered at the beginning and end of the quarter.
Scores analyzed to determine improvement
Examples of Objectives:
Test Questions
Objective 1
Objective 2
Objective 3
Objective 4
Objective 5
Objective 6
Revising
Who? • Input from
faculty members• Revisions made
by librariansWhen?• Tests
evaluated each quarter
Why? • Information
changed• Keep the integrity
of the test• Questions with
high-miss rates
Request
Request to administer tests totally online and receive results in a timely manner.
The online testing instrument must use resources already available.
Experimentation
Brainstorming with involved faculty, IT, and library staff.
Initial decision to use Google Docs as the vehicle for administration.
Results can be downloaded to a spreadsheet.
Security of tests was an issue.
Success!!
Library has a subscription to Survey Monkey (a survey tool).
Time limits and passwords can be set. Results can be compiled in a spreadsheet.
Testing has been done for several quarters with minimal problems.
Online Testing
Online Testing: Google Docs
Online Testing: Google Docs
Online Testing: Survey Monkey
Online Testing: Results
Online Testing: Results
Assessment: Evaluations
When:
Administered each quarter following completion of the library component of FYE
Who:
All students in FYE (both on-campus and off-campus)
Evaluations: Type of Questions
Presentation of IL component (was the information clear and
organized?)
Usefulness of information for future
classes
Classroom environment (freedom to ask
questions, equipment and databases easy to
use?
Understanding of information prior to compared to after IL
component
Speed and time allowed for component
Most helpful information gained and “what did you like the
most?”
Demographic questions (is this your first time to
take the class, etc.?)
Suggestions for improvements
Evaluations (Example)
Longitudinal Data: Statistics
Semi-annual meeting with FYE
& Gen Ed Faculty/ Librarians
Compile annual statistical
report
Test scores Student evaluation responses
Continuous “tweaks” as
needed
Communication (formal and
informal)
Evaluation Statistics
• Yearly statistical report created.
• Report compared to previous statistics.
• Use ranking system (1 to 5) with many questions.
Evaluation Statistics: Annual Report
Evaluation StatisticsExample: Averages of the evaluation questions from 2000 - 2011
What have we learned?How are the FYE students and faculty doing?
• Evaluations
• Overall, students appear satisfied happy with the information literacy (library component):
• Average evaluation questions ranking of 4.31 from Fall 2010 to Summer 2011 (5 is the highest ranking)
• From 2000 to Fall 2010, the rankings ranged from 4.13 to 4.36.
What have we learned?How are the FYE students and faculty doing?
• Evaluations• The majority of students feel that their knowledge of databases, internet, and
print resources has improved following the library component. My understanding of the ELECTRONIC DATABASES PRIOR to the library component was...
Answer OptionsResponse Percent
Response Count
Fall 2010-Summer 2011
Above average 13.0% 39 16.0%Average 39.9% 120 35.4%Below average 47.2% 142 48.6%
My understanding of ELECTRONIC DATABASES AFTER the library component was...
Answer OptionsResponse Percent
Response Count
Fall 2010-Summer 2011
Above average 47.6% 141 49.0%Average 50.0% 148 48.2%Below average 2.4% 7 2.8%
What have we learned?How are the FYE students and faculty doing?
• Test Scores• There is significant improvement from the pre-test
to the post-test.
• Scores ranged from 61 to 66% (pre-test) and 80% to 85% (post-test) from Fall 2002 to Summer 2011.
What are students saying?
Forcing students to use Sullivan's Library
was very much informative for me. I
had no idea of the options to retrieve
info before this class. This was all new to
me and I'm glad I ran into it.
Some people (like me) are scared to use a computer. My fears are more at ease
after this class.
I believe the library component was just
right for a 1st quarter student such as myself. We learned a lot and by having the handouts to go back and refer to in
the future is a great help.
This is my first year here at college. Everything I am learning is helping my education after twenty five years.
Before this class, i didn’t know anything about any of the databases but now I do, and I have already started using them as resources for other projects.
The class was perfect. My instructor made sure that I as well as others understood everything well. I feel that nothing should be taken away. Also, nothing really needs to be added. It's great, and I learned a lot more than I knew about the internet thanks to this class.
The Future?
Continue to make assessment count!
Continued evaluation leading to revision and
improvement
Continued communication and collaboration
Questions/Comments?
