![Page 1: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/1.jpg)
1
Work in Progress
Critical Thinking and Information Literacy: Assessing Student
PerformanceThe InfoSkills Research Team
Dr. Senay Purzer, Mr. Michael Fosmire, Ms. Amy Van Epps, and Ms. Ruth Wertz
Project Collaborators
Ms. Megan Sapp Nelson, Dr. Brian Dillman, and Mr. Austin Saragih
Presented at the 120th ASEE Annual Conference & Expo – Atlanta, GAJune 24th, 2013
InfoSkills, 2013
![Page 2: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/2.jpg)
2
Overview
Project Background – Information Literacy and Critical Thinking
Research Methods – RQs and Research Design
Project Results – Correlation of CELT and CAT
Project Conclusions – Implications & Further Study
InfoSkills, 2013
![Page 3: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/3.jpg)
3
A set of skills that enables the ability to recognize the need for information, and the ability to search for, access, evaluate, and
use information to fulfill a specific purpose.1
InfoSkills, 2013
DefinitionsProject Background
Information Literacy
The cognitive process of “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or
generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.”2
Critical Thinking
1. American Library Association. (2000). Information literacy competency standards for higher education. Retrieved from http://www.ala.org/acrl/sites/ala.org.acrl/files/content/standards/standards.pdf.
2. The Foundation for Critical Thinking. (2013). Defining critical thinking. Retrieved from http://www.criticalthinking.org/pages/defining-critical-thinking/410.3. Albitz, R. S. (2007). The what and who of information literacy and critical thinking. portal: Libraries and the Academy, 7(1), 97-109.
recognize need
accesssearch for/gather
evaluateuse/apply
synthesize
analyzeconceptualize
SET OF SKILLS3
COGNITIVE PROCESSES3
![Page 4: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/4.jpg)
4
Impetus for our research:• Important 21st century skills• ABET criterion 3i (lifelong learning)• Engineering decision-making
Primary focus of our research:• How do we assess information literacy and critical
thinking?
Impetus for Our Research
InfoSkills, 2013
Project Background
![Page 5: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/5.jpg)
5
InfoSEAD framework for item generation1
• Seeking – Gathering and access• Evaluation – Assess quality, credibility, etc.• Application – Using information to satisfy a need• Documentation – Citation and in-text referencing
Two Scenarios 18 total items• 16 multiple choice; 2 select all that apply
About the CELT(v2.1)
InfoSkills, 2013
Project Background
1. Wertz, R. E. H., Purzer, S., Fosmire, M. J., & Cardella, M. E. (in press). Assessing information literacy skills demonstrated in an engineering design task. Journal of Engineering Education.
![Page 6: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/6.jpg)
6
Research question:• How well does the CELT (v2.1) measure critical
thinking?
Research method:• Pearson’s correlational analysis between CELT
instrument and the Critical Thinking Assessment Test (CAT) developed by Tennessee Technology University.1
InfoSkills, 2013
Correlational StudyResearch Methods
1. Center for Assessment and Improvement of Learning at Tennessee Technological University. (2010). CAT technical information, from http://www.tntech.edu/files/cat/reports/CAT_Technical_Information_V7.pdf
![Page 7: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/7.jpg)
7
N = 44 first-year engineering students CELT reliability measures• KR-20 = 0.671
• N = 188 first-year students (Fall 2012)• Based on 16 MC questions
CAT reliability measures • 15 constructed response items; scoring reliability 0.822
• Cronbach’s alpha 0.702
InfoSkills, 2013
Population and SettingResearch Methods
1. Wertz, R. E. H., Saragih, A., Fosmire, M. J., & Purzer, S. (2013). An Evaluation of the Critical Engineering Literacy Test (CELT) Instrument through Item Analysis and Comparison to the Critical Assessment Test (CAT). Paper presented at the 2013 Illinois/Indiana - ASEE Section Conference, Angola, IN.
2. Center for Assessment and Improvement of Learning at Tennessee Technological University. (2010). CAT technical information, from http://www.tntech.edu/files/cat/reports/CAT_Technical_Information_V7.pdf
![Page 8: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/8.jpg)
8
Overall Correlational Analysis:• CELT total score was positively related the CAT
total score (r = 0.47, p <0.01)
Itemized Correlational Analysis:• Items 5, 11, 16, 17 had positive associations with
the CAT total score
InfoSkills, 2013
CELT ResultsProject Results
![Page 9: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/9.jpg)
9InfoSkills, 2013
CELT ResultsProject Results
Which of the following comparisons of omega3/omega6 ratios is most relevant in determining whether GE fish is equivalent to
its non-GE counterpart?
Item 11(r = 0.32, p <0.05)
What would help the review panel validate the data presented?Item 16
(r = 0.37, p <0.05)
Where would you likely find authoritative information on a typical omega-3 levels of salmon?
Item 17(r = 0.35, p <0.05)
[Evaluate] which of the citations is incorrect or incomplete?Item 5
(r = 0.51, p <0.01)
![Page 10: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/10.jpg)
10
14 of 18 CELT items positively correlated to some portion of CAT
Correlation between the CELT and CAT total scores is moderately strong (r = 0.47, p <0.01)
CELT items 5, 11, 16, and 17 provide insight into a possible subset of skills where information literacy skills and critical thinking overlap.
InfoSkills, 2013
General ConclusionsProject Conclusions
![Page 11: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/11.jpg)
11
What are the common elements in CELT items 5, 11, 16, and 17 that made them correlate to CAT?• Expand sample• Item-to-item analysis• Verbal protocol
InfoSkills, 2013
Future StudyProject Conclusions
![Page 12: Critical Thinking and Information Literacy: Assessing Student Performance Work in Progress Critical Thinking and Information Literacy: Assessing Student](https://reader035.vdocument.in/reader035/viewer/2022071807/56649dce5503460f94ac1ae9/html5/thumbnails/12.jpg)
12
Dr. Senay Purzer – [email protected]
Mr. Michael Fosmire – [email protected]
Ms. Amy Van Epps – [email protected]
Ms. Ruth Wertz – [email protected], 2013
THANK YOU!
Contact Information