assessing learning, critical reflection, and quality ......assessing learning, critical reflection,...

Assessing Learning, CriticalReflection, and Quality EducationalOutcomes: The Critical IncidentQuestionnaire

Donald L. Gilstrap and Jason Dupree

This research study incorporates Brookfield's Critical Incident Question-naire (CIQ) as a qualitative instrument to assess the ACRL InformationLiteracy Competency Standards for Higher Education in one library'sinstructional curriculum. A sample (n=348) of English Composition II stu-dents was studied over the course of two semesters during a four-sessioninstructional program. A methodological framework of critical reflection,incidents, and events was incorporated, as well as reflection on practice.Results of the study showed the CIQ was effective in supporting qualita-tive methods for assessment of critical reflection in generaland the ACRLStandards specifically during the research and learning process.

n 1989, the American Li-brary Association first main-streamed the importance ofcritical thought in informa-

tion literacy.' This focus on thinkingcritically was later developed further bythe implementation of the Associationof College & Research Libraries (ACRL)Information Literacy Competency Stan-dards for Higher Education in 2000.2 As aresult, critical thinking skills have becomeincreasingly important in library andinformation literacy programs over thepast 15 years. Recognizing the importanceand complexity of this pedagogical con-struct, committee members of the ACRLdid not define the term critical thinkingas a static and prerequisite "thing" that

exists in classroom settings waiting to bediscovered. Rather, thinking and discern-ing critically incorporates a process ofexperience and analysis toward whichwe guide our students as they becomeinformation-literate consumers as well asproducers of information. Encompassingthe related concepts of critical reflectionand critical incidents, the goal of this re-search study was to facilitate and assessthe process of critical thinking in a libraryinstruction program, as well as to reflecton our own practice of teaching informa-tion literacy.

In 2002, the Southwestern OklahomaState University (SWOSU) Libraries re-structured its entire library instructioncurriculum to integrate specifically the

407

Donald Gilstrap is Associate Dean of Librariesfor Technical Services, University of Oklahoma Libraries;e-mail: [email protected]. Jason Dupree is Head of Public Services, Southwestern Oklahoma State UniversityLibraries; [email protected]. The authors wish to thank Janet Grabeal, Assistant to the Director,SWOSU Libraries, for her assistance in compiling data used in this study.

408 College & Research Libraries

new ACRL Standards. The new cur-riculum identified three tiers of instruc-tional levels. Tier 1 targets first-yearundergraduates in Freshman Orientationand/or English Composition I courses,where library instruction focuses on thelibrary's services, resources, and Web siteduring a single session. Tier 2 consistsof multiple sessions, integrated into theEnglish Composition II courses thattraditionally consist of first- and second-year students. Library instruction in thesecond tier elaborates on the library'sservices and resources, emphasizessearching techniques, details the processof evaluating information, and allowsstudents time to work with resources andresearch concepts through self-directedand instructor-mediated environments.Research-based, upper-division, andgraduate courses constitute Tier 3. Thistier serves graduate and undergraduatestudents who are focusing on researchwithin their academic major. At this level,library instruction attempts to expanda student's knowledge of informationalresources within his or her degree field,stresses advanced searching techniquesand elaborates on critical thinking andreflection by asking students to considerlegal, social, and ethical uses of informa-tion. Each of the Tier levels in the SWOSULibraries Instruction Program utilizes acurriculum outline, following the ACRLInformation Literacy Competency Stan-dards for Higher Education. After we haddeveloped the curriculum, we wanted toidentify a method of assessing studentlearning, as well as to help us continuallyimprove our teaching. We have, therefore,chosen a method that assesses the ACRLStandards specifically and critical reflec-tion during learning in general.

Literature ReviewIn our development of a theoretical frame-work for assessing the critical thinkingprocess, this study relies on the works ofBrookfield, 3 Tripp,4 and Woods.5 Duringthe 1980s, Stephen Brookfield 6 produceda seminal work, The Skillful Teacher. This

September 2008

book has risen to a level of wide embracein teaching circles as a means for reflect-ing on practice. It has been incorporatedinto library science and educational the-ory courses, and it has become a staplein many university faculty professionaldevelopment collections. Brookfielddeveloped his theoretical framework forthe CIQ used as the assessment instru-ment for this study, by borrowing fromDaloz's7 focus on transformational learn-ing, Tripp's8 research on critical incidents,and Woods'9 development of critical edu-cational events. Consequently, Brookfieldargues for a more personalized approachto teaching that moves the instructor to-ward the role of active participant in thelearning process, much like the teachingof manipulative and cognitive processesin information literacy as described byHenri and Dillon.10 In Brookfield's view,those who teach can draw upon both theirstudents' and their own sources of criti-cal reflection through journaling, groupwork, and discussion of classroom events.The development of Brookfield's work inthis area seems to have coincided withthe emphasis placed on critical thinkingin the ACRL Standards.

Understanding Brookfield's devel-opment of the CIQ as the assessmentinstrument used for the study beginswith a brief overview of Tripp's" seminalwork on critical pedagogy. Tripp definescritical incidents, which emerge throughthe critical reflection process, in the fol-lowing way:

Critical incidents are not 'things'which exist independently of anobserver and are awaiting discoverylike gold nuggets or desert islandsbut like all data, critical incidentsare created. Incidents happen, butcritical incidents are produced bythe way we look at a situation: acritical incident is an interpretationof the significance of an event. Totake something as a critical incidentis a value judgment we make, andthe basis of that judgment is the sig-

Assessing Learning, Critical Reflection, and Quality Educational Outcomes 409

nificance we attach to the meaningof the incident.

12

Tripp also contends that critical in-cidents do not have to be dramatic orunusual. Oftentimes, critical incidentsseem nothing more than common ev-eryday events "but are rendered criticalthrough analysis,"13 as is perhaps thecase in many of the information literacycourses we teach.

Brookfield's development of the CIQalso relies on the work of Woods,14 whoargues for a more radical approach tostudies of critical reflection. In Woods'sview, critical educational events are cata-lysts for transformative development ofboth students and teachers. In case studyresearch, Woods and others have foundthat critical incidents have the potentialto be "highly charged moments and epi-sodes that have enormous consequencesfor personal change and development."'

5

Segal has also proposed a complimentaryperspective on critical incidents and criti-cal reflection during the learning process."6

Segal describes how assumptions andhabitual practices that are disrupted leadto reflection or defensiveness duringlearning. Equally, the more recent works ofMezirow" on perspective transformationshow how a disorienting dilemma servesas a catalyst for critical reflection, usuallyinvolving something dramatic. Conse-quently, Segal notes that the very processof critical reflection can cause feelings ofuneasiness among learners and can leadto apparent defensiveness in responsesduring the process of critical reflection, aphenomenon that is shown in more detailin the findings section of this article.

Several scholars in the social and be-havioral sciences have performed studiesof critical incidents through reflectionusing this framework for investigation inrecent years. Drawing from Sch6n,18 theworks of Thiel19 and Miller2" have utilizedcritical incident techniques in biographi-cal writing on classroom experiences toreflect on teaching practice. Weinreich's2"study incorporated Brookfield's CIQ to

assess the impact service learning hadon collaboration among students duringteam projects. Equally, Angelides andAinscow22 and Jackson and Wasson23

develop Tripp's framework of criticalincidents, addressing culture and howit impacts the learning environment.Additionally, Farmer 24 focuses on theunderlying social contexts that drivethe critical reflection process, whilePugh and Bergin's2" studies show howstudents reflect on knowledge gained inthe classroom and apply it in the outsideworld. Moreover, Walton and Nettleton 26

borrow from Sch6n's27 work on reflect-ing on practice to develop a conceptualmodel that focuses on critical reflection inthe classroom through the integration ofstudents' classroom experiences with theresearch process." And, finally, Brookfieldhas drawn on the use of critical incidentsin numerous autobiographical and eth-nographical studies of critical reflectionin areas ranging from teacher educationto the health professions.2 9

These types of research studies, there-fore, add further support for the use ofcritical incident methods to assess theteaching and learning processes of infor-mation literacy programs. Additionally,these studies relate uniquely to the ACRLStandards' focus on applying the knowl-edge gained from library instruction toinfluence events outside the classroomsetting."g However, attempts to assess thisbased on the ACRL Standards provideus with unique challenges and oppor-tunities. Although it is argued criticalincidents emerge in different ways, it isagreed that they are unpredictable, aredifficult to control, contain simultane-ous problems and solutions, and can lastfrom a very short period to many weeks. 31

Therefore, we feel that a qualitative tool ofdiscovery, such as has been developed byBrookfield with the CIQ, presents educa-tors with an opportunity to incorporatenew assessment methods that seek todevelop further understanding of thecritical reflection process in general, whileidentifying ACRL Standards, Outcomes,


and Performance Indicators in informa-tion literacy programs specifically.

Research DesignFor the purposes of this study, we havechosen a qualitative method as a basis forour research design. Brookfield's exten-sive investigation into the roles of criticalreflection in teaching and learning pro-cesses led to the development of the CIQ.32

The primary purpose of this instrumentis to assess student critical thinking andsubsequently reflect on these findings asa source of professional development byteachers. This study uses a modificationof the CIQ to collect data on the criticalreflection process during library instruc-tion sessions at our library.

All sections of semester-long EnglishComposition II courses at this universitywere chosen for study for a period of onefull academic year. We determined thatthe entire population (P=752) of EnglishComposition II students at this universitywould be advantageous for a number ofreasons. Primarily, we felt the populationwould provide more robust and compre-hensive data when analyzing the complex-ities of the critical reflection process in ournew, ACRL Standards-based curriculum.Equally, we had already introduced thetier approach to library instruction into theEnglish Department on campus, focusingon the Tier 2 level of our Library Instruc-tion Program for these courses.

A very productive and collegial rela-tionship had been developed betweenEnglish and Library faculty, and we hadintegrated a four-session curriculum intomost English Composition II courses.Students would come to the library dur-ing four different class periods, receivinginstruction on information literacy andacademic research. Library Instructionsessions included traditional lecture,group work, active learning activities,and a time for critical reflection at theend of each class. Additionally, due tothe typically one-session duration ofTier 1 freshman orientation and Tier 3upper division and graduate classes, we

September 2008

believed at the time it would be harderto integrate the CIQ instrument and stillcome away with rich and descriptive datathat incorporated the complexities of amultisession approach.

After students completed a short de-mographic questionnaire (see AppendixA) and informed-consent documents, thefirst instruction session focused on intro-ducing the research process and applyingit within the lecture material. Additionally,we incorporated a theoretical componenton database architectures: showing howdatabases store and retrieve informa-tion, introducing database searchingtechniques, and applying this knowledgethrough the use of the online catalog tosearch for books. The second sessionincluded a review of the first session,lecture and discussion on the differencesbetween popular magazines and scholarlyjournals, and instructor demonstrationsand student application of both serialsaggregation and journal article databases.The third session included a recursivereview, focused on news sources and theuse of relevancy ranking, and utilizednewswire services and government in-formation. And the fourth session beganwith review and then integrated groupactivities and discussion on the strengthsof the open versus the hidden Web, anexamination of search engines and toolsused to locate information on the Web andhow they operate, the process of evaluat-ing information sources, and a condudingdiscussion on examples where authorsfailed to conduct thorough research.

At the end of each of the four sessions,CIQs were distributed to students withfive basic questions (see Appendix B). Stu-dents had time to reflect on critical inci-dents they felt had taken place during thelearning process in that session and writebrief responses to describe these ideas. Li-brarians teaching the classes would thencollect the CIQs at the end of the class andplace them in an envelope. Before the nextday's session, each instructor would gleanthe CIQs and look for issues or ideas theyperceived to be critical incidents in their


teaching of information literacy. The nextsession would incorporate follow-up onthe previous lesson to ensure that compo-nents were being taught clearly.

In Tripps's view, reflective teaching isjust as important as reflective learningamong participants. Tripp also arguesthat, when studying critical reflection, itis difficult to "move from a conclusionto data rather than the other way,"'33 andthat continual analysis of the data aids inmore accurate representations of criticalincidents. This idea supports our choice ofqualitative investigation for this study andcomplements the framework from whichBrookfield originally developed the CIQ.Equally, Denzin and Lincoln" have beencited frequently in the literature in theirdescription of data triangulation as a va-lidity procedure in qualitative research. Todecrease the influence of the researcherson the study findings, we have followedtheir approach, using multiple methodsand multiple researchers throughout thestudy. During different stages of this re-search, instructors performed journalingactivities where we reflected on learningexchanges in dialogue between teachersand students--or procedural issues deal-ing with the study-and logged these inindividual journals. We also relied on themethods encouraged by Brookfield, 35

Hoover,3 6 and Sch6n 37 to reflect criticallyon our practice of teaching by meetingperiodically to discuss each other's ob-servations while integrating our viewsinto a coherent "master journal."

Library instructors were given fulldata sets that contained all of the CIQresponses. All student IDs had beenstripped from these files, and any in-formation that identified the instructorteaching the class or the correspondinginstructor was also removed. Instructorsthen worked independently with the datasets to develop clustering of the generalthemes that emerged from each of thesessions, similar to the approach chosenby Kracker and Wang.-" Equally, afterinstructors had developed their generalthemes, we met to compare and discuss

each other's themes as additional meth-ods for data checks suggested by Denzinand Lincoln. 39 Notes were recorded of theobservations of each of these meetings,and group clustering took place to syn-thesize the major themes we observed forthe findings section of this article.

Results and Discussion of FindingsGroup interpretation of the data led toclusters of themes that emerged through-out this study for each question. Out ofthe population surveyed (P=752), wereceived a total response rate of 58%(N=433), and a study response rate of46% (n=348) for those students whocompleted informed-consent forms. Weviewed this as a very successful responserate, particularly for a qualitative study.40

A portion of the data reflected shallowdescription for each of the questions.Answers ranged from "I understoodeverything" in Question I to "None" or"N/A" in Question 2, Question 4, andQuestion 5. However, inferences can bemade about these types of responses:students are apathetic about the grow-ing amount of survey research in highereducation, or that students' perceptionsof their research abilities are higher thantesting of these abilities would sug-

FIGURE 1Participated in Library Instruction

Session Previously

Didn't respond3%

Nos

. . .6%

......... ii !! i:.. .N o .......:i! i ii !ii:! i !ii.. .


FIGURE 2Responses by Age

140-

19

120-r

100-

1880-

20

140-4V_

20-

24 in 27 28 32 33 34 38

_ 22-1 --2217 19 21 23 25 27 32 34 45

60-'

gest, as has been shown by Dunn41 andMaughan.4 2 Therefore, arguments couldbe made that the critical reflection pro-cess is not taking place for these students,the process is not apparent in their de-scriptions, or there has not been enoughbackground content knowledge on thecritical reflection process. Conversely,it can also be inferred that librarianshave performed content delivery well,

which supports ACRL Information Lit-eracy Outcomes, and, as a result, fewerstudents related descriptions of criticalincidents. Equally, we expected that thenumber of students identified as havingparticipated previously in Tier 1 sessions(see figure 1) might have contributedto this phenomenon. Further researchwould be necessary to investigate thequestions surrounding the lack of depthin these particular responses.

However, since we chose an entirepopulation for our data collection, weviewed the fact that a large portion ofstudents responded with rich descrip-tion as a success for qualitative studies,as Denzin and Lincoln43 have shown. Asmost researchers conducting qualitativestudies will note, the richness of thesedata cannot comprehensively be ex-plained within the confines of an article.Some might suggest that qualitative dataanalysis provides avenues for infinite ob-servations and descriptions of participantresponses.44 Descriptive statistics havealso been used for exploratory purposesand to generate future hypotheses (see

FIGURE 3Responses by Sex

Didn't respond3%

Female56%

Didn'tWRespInd45 l

September 2008


figures 1-6). Through the process of cod- descriptive vignettes of student responsesing and recoding data, we have tried to that help support analysis of our observa-identify general themes that emerged tions and reflection on our teaching prac-during this study and provide very short, tice. Standards, Performance Indicators,

FIGURE 5Responses by Academic Level

250-

200 Fr

150o]

50-

Freshman Sophomore Junior Senior Didn't Respond

FIGURE 4Responses by Identified GPA

120

100V

80.

60-Z

03.5-4.0 3.0-3.4 2.5-2.9 2.0-2.4 1.9 or below Didn't respond


and Outcomes4" from the ACRL Infor-mation Literacy Competency Standardsfor Higher Education will be referred tothroughout the results of this study, andwe suggest referencing this documentwhile reading the findings.

Results for Question 1LearningThe main themes that emerged from re-searchers coding responses for this ques-tion deal with learning, searching, time,and evaluation of resources. The learningprocess seemed to be exemplified as themain source of critical reflection. Pri-marily, students responded most oftento issues surrounding the application ofknowledge. Many students referencedtheir feelings of confidence about under-standing the content of the day's sessionwhen they were able to apply what theinstructor taught. This proclivity towardkinesthetic learning by the students camein the form of applying new knowledgeto worksheet activities, interacting ingroup exercises, teaching the teacher,and hands-on computer activities, whichreflect standards 1.1, 2.2, and 3.6. Studentsnoted this through statements such as,"I understood what we learned whenwe did our activity." "when we got to

FIGURE 6Transfer Student

Didn't respond Yes3% 9%

....... .......... ........... ............ ............ .......... .....

.......... .....

.......................................I ...........................................................................................................................I ..............................................................................................................................................................

No88%

TABLE 1Results For Question 1: At What

Moment in the Class Today Did YouFeel Like You Most Understood the

Instructional Content?

Question I ThemeResponses,

n =

All/everything 95

Learning 76

Boolean/search strategies 67

Time 41

Evaluation 35

N/A or illegible 14

Miscellaneous 20

TOTAL 348

practice what we were being taught,"and "I think everything is becomingclearer now." Statements such as thesereinforce the power of in-class exercisesto help students synthesize the conceptsthey are learning and apply them to theactual research expected of them in theircoursework.

Visual learning also served as a cata-lyst for many of the students' reflectionson learning. Graphical representationsand models used in PowerPoint presen-tations appeared to be an effective formof conveying information theory-basedconcepts through visual media. As onestudent noted, "I really understood thepower point slide. It allows me to seewhat you're talking about rather thanto just hear it." Moreover, generativemetaphors were used in the teaching

process that helped bring out deeper criti-cal reflection on student learning. Whenteaching components of the researchprocess, a ladder was used as a visualrepresentation. When describing knowl-edge acquisition, this metaphor returnedrepeatedly. One student commented, "Ifelt like I most understood the slides, es-pecially the 'research ladder' slide." Usingvisual metaphors to teach poetry research,

September 2008


another student noted, "while looking atthe poetry, I realized what the instructorwanted us to do in our research."

Another recurring concept in thelearning theme was the use of reviewduring closing and subsequent sessions,as students noted this technique increasedtheir recall of previously learned knowl-edge. Students commented particularlythat the incorporation of role-playing,where students teach the teacher, wasan effective method to increase recall.And, conversely, reflections on feelingsof confusion emerged. Albeit these feel-ings were infrequent for this question,students showed that, although they didnot understand all of the content, theywere empowered in the classroom to re-spond to these phenomena. Equally, thesefeelings of confusion often accompaniedobservations about the complex natureof the research process while using theInternet, supporting Segal, Brookfield,Trip, and Woods.

46

SearchingIssues surrounding searching alsoemerged as a major theme for this ques-tion. We recognize the emergence of Web2.0 technologies and do not necessarilyencourage or force students to use Bool-ean searching when more effective natu-ral-language searching is available. How-ever, Boolean logic is an integral theoreticalcomponent of this library instruction cur-riculum to build foundational knowledgefor database architectures, and the issueof search construction came up most fre-quently for this theme. Librarians felt thisis one of the more difficult-or perhapsunexciting--aspects of the curriculumto teach. However, perhaps it is becauseof this focus on teaching Boolean logicthat instructors spend more time on it.Although we expected criticism of teach-ing Boolean logic, much to our surprise,students responded with descriptions ofclarity on this subject above many othercurriculum components in the searchingtechniques session perceived by instruc-tors to be less difficult. This was evidenced

by comments such as, "I felt like I under-stood the instructional content when theinstructor was talking about Booleanoperators," and "talking about Booleanlogic is what made sense to me." Again,we noticed the importance of standards2.2 and 3.6 to which students alluded intheir statements. Other related studentreflections on searching dealt with a betterunderstanding of when to use keywordand when to use subject searching, andhow to use article databases and Internetsearch engines effectively.

TimeIdentification of time periods during thesession such as "beginning," "middle,""end" -- were also considered moments ofunderstanding by the students. However,since descriptions were limited in theseresponses, it was difficult to identifyexactly which part of the instructionalsession to which students were refer-ring. Equally, since each instructor taughtcomponents at different times during thesession, generalization about the timeperiods could not be made. As a result,this theme had to be downplayed due toinsufficient data for analysis. However,some comments alluded to ideas thatwere usable for pedagogical reflection forthe individual instructor such as, "I un-derstood it all after she completed [his orher] lecture," in this case before applyingknowledge; or "the instructor explainedthe topics really well so I felt I understoodeverything at the end of the lesson," inthis case after applying knowledge.

Evaluation of ResourcesThe evaluation of resources emerged asanother common theme in the curricu-lum. Part of the curriculum emphasizesthe key differences between magazinesand journals and their uses for academicresearch. Based on our feedback fromthem, students tended to know relativelylittle about scholarly journals before par-ticipating in library instruction. However,after completing the session on scholarlyinformation sources, students commonly


TABLE 2Results For Question 2: At WhatMoment in the Class Today Did

You Feel Most Confused About theInstructional Content?

Question 2 ThemeResponses,

n=

None/never 182


Learning 37

Time 33

Evaluation 22

N/A or illegible 14

Miscellaneous 16

TOTAL 348

reported a better understanding of howpopular and scholarly publications dif-fer-emphasizing standard 1.2.d-andhow journals contribute to the researchactivities of university faculty. This sametype of reflection came from analysisof Web-based sources. After comparinggovernment and education domains withpropaganda and parody sites, such asmartinlutherking.org and whitehouse.net, students commented on their realiza-tion that there are indeed vast differencesbetween popular and research-based Websites available for free access. We felt thestudents' observations confirmed theimportance of standard 3.2.c, realizingprejudice and deception in informationresources. Although the publication ofscholarly information is critical to theadvancement of academic thought andprogress in higher education, we recog-nized how commonly we in universitysettings take for granted that students donot fully understand how very differentpopular and scholarly literature can be.The students' concrete identification ofthe importance of evaluation of sourcesin research emphasized their positivereaction and deeper understanding ofthese differences when provided withthis curriculum.

Results for Question 2SearchingAlthough 67 students responded that thesession on Boolean logic and search strat-egies was the moment where they mostunderstand the instructional content, 44,or 13%, of the students also respondedthat this led to their most confusing mo-ment. However, this confusion seemedto emerge among students for differentreasons than in Question 1. One studentimplied that Boolean logic was not auser-centered form of searching, noting:"...the and, or, and not I understood, butit seems like too much work." Standard3.4.e addresses this statement, particularlyin regard to determining the limitations ofinformation-gathering strategies. Anotherstudent identified that, in spite of the com-plexity of user interfaces, they are not ableto infer what the user means: "...when it

came to using the Boolean operators in theactual search, I couldn't think of synonymsfor Oklahoma and statehood." Another ap-proach instructors used was to provide amathematical metaphor to explain Booleanlogic, but one student noted: "...when shestarted applying it to math with the FOILmethod, I just got lost." These commentshelped us to reflect not only on our teach-ing strategies but also on the limitationsin the design of systems with which weaccess information.

TimeFor this question, we were able to distin-guish more information regarding timesegments during the class period. Ratherthan the brief answers that appeared inQuestion 1, students included informa-tion in Question 2 that helped us identifywhat section of the class session or whythat section stood out, such as "...at thebeginning of class because I didn't knowany of the content" or "... at the beginningtalking about [word games]." Moreover,students addressed issues of time, deal-ing with the instructional content. Somestudents commented there was a lot ofinformation presented in a short amountof time; a dilemma with which most of

September 2008


us teaching library instruction can sym-pathize. "S/he went too fast on the slides.I didn't have enough time to write downone of the last slides."

We found reflections such as these to betelling statements. On one hand, we real-ized we might need to adjust the contentand approach to teaching this section oflibrary instruction. Yet, on the other hand,we found that these types of commentsshowed engagement in the instructionalsession at a level where she or he feltfrustrated by not being able to recordall of the knowledge being shared. Wealso feel it is important to note that thesephenomena emerged in Question 2 whereconfusion is identified. Supporting theworks of Brookfield, Tripp, and Woods,critical reflection takes place as a resultof critical incidents and events.47 Criticalreflection by the students on these issuesseemed to take place at a deeper level, asa result of their confusion or frustration.Equally, they expanded on why thesefeelings were important to them.

Evaluation of ResourcesEvaluation of resources was another ma-jor theme that emerged in both Question1 and Question 2. "I am kind of confusedabout all of the Web sites as far as findingthe correct information I may need," and"I was most confused about using bal-anced or biased resources. It looks like itwould be hard to distinguish." Anothertelling statement was "I was most con-fused when we talked about magazinesand journals, since I don't read either ofthose." We recognized that, although thecontent being taught in the curriculum isimportant, statements such as these mightreflect a much different methodologyused to access information, principallyfrom the World Wide Web, by new gen-erations of college students and that wemight need to adjust the way we approachthe section on evaluating magazines andjournals to reflect how online resourceschange student perspectives. The centralfocus of standard 3.4.e speaks to thegenerational differences between the Net

Generation and previous generationsin their determinations of the nature ofinformation by questioning its originsand limitations.

LearningFor some students, the application ofknowledge learned in the sessions wasnot as easy as it appeared. We found thiswas particular to difficult topics wherekeywords are so specific or narrow thatdatabases produced few results. Review-ing search strategies and informationalsources, and subsequently integratingnew knowledge, is the core methodologyof standard 3.7.b-c stressed in our cur-riculum. Other students noted feelingsof information overload such as, "exactlywhere to go when I find what I am look-ing for. There was a lot of information inthis session, and I kind of got it all mixedup." This supports the idea that researchis a difficult process, but we felt that, forsome students, the amount of informa-tion might be too much. Despite thispossibility, standard 1.1.c-d is designedto alleviate anxiety and frustration by en-couraging students to explore resources,develop familiarity with their topics, andachieve a manageable focus regardingtheir research tasks. Some students alsodid not understand the group activities

TABLE 3Results For Question 3: What WasYour Most Rewarding Experience

in Today's Class?


n1=

Confidence 97


Evaluation 63

Learning 61

N/A or illegible 13

Library resources 24

Miscellaneous 13

TOTAL 348


or metaphors used to convey meaningof the curriculum concepts. We found,however, that comments of confusion onthe types of pedagogy used in the sessionswere spread evenly between kinesthetic,auditory, and visual learning. This tendsto agree with the reliability of usingmultiple methods when teaching to ac-commodate diverse learning styles whilefocusing on experience in learning.4 Thisphenomenon of diverse learning stylessupported our original curriculum objec-tive of including several different learningactivities and teaching methods to sup-port the multiple ways in which studentsdiffuse knowledge.

Results for Question 3ConfidenceResults from Question 3 detailed studentperceptions of their most rewarding ex-perience in the day's library instructionsession. By far, the most commonly emer-gent theme for this question surroundsstudents' descriptions of improved con-fidence in their library research skills,exemplifying standards 1.2.a, 1.2.c, and1.4.b. Responses included confidence inlibrary research in general: "I feel moreconfident about doing research in thelibrary" and "I feel better prepared toperform library research"; confidencein database searching: "I feel confidentI can perform searches more efficiently,"underscoring standard 2.1.d; and stan-dards 3.4.a and 3.7.a develop the students'confidence in synthesizing and applyingknowledge reflected in several statementssuch as: "When I was able to complete theworksheets by myself!"

Evaluation of ResourcesEvaluation of resources was also iden-tified by students as being their mostrewarding experience. These responsessupported aspects of standard 3, wherestudents learn to evaluate information,think critically, and incorporate newinformation into their knowledge base.The following student reflections are wellfounded in 3.2.a: "learning where to find

September 2008

credible sources for references on the In-ternet" and "that when you use some do-mains, you want to check for accuracy andauthority of the author, because some arecredible and some aren't." But responseson this question also identified commonmisperceptions we might have aboutstudents: "going through the Web sitesand seeing how a professional opinionwould be better than a college student'sopinion." Standard 3.4 highlights thepossibilities for students to compare newinformation with prior knowledge whileformulating new conclusions, and stan-dard 3.5 encourages students to reconciledivergent viewpoints and judge what isappropriate or inappropriate. Clearly,these students reflected their understand-ing of these concepts after engaging in theinstructional sessions.

Although at face value, this might ap-pear obvious to librarians, we realizedhow important a concept this has becomein the age of blogging and facebook.com.We noted it was interesting that so manystudents identified evaluation of resourcesas their most rewarding experience. Whenwe might be too apt to write off newergenerations of students in their use of sus-pect resources, we were reinforced that, iflibrarians provide instruction on evalua-tion-and utilize methods that generatemeaning for students -then the studentswill come to value it to a high degree.

The Learning ExperienceThe learning experience was identified asanother rewarding theme of the instruc-tional session. One aspect of the learningexperience students noted as importantwas the focus on recursive learning. De-scribed by Doll4 9 as a continuous returnto previous content and experiencesin order to create new knowledge andmeaning, we actively included reviewand discussion of the previous session'scontent to provide a foundation for thenew session. This was evidenced throughresponses such as "the review" and "be-ing able to understand what we did inthe last class better." In standard 4.1.b,


knowledge and skills are revisited fromprevious experience, and, more specifi-cally, standard 1.1.d-e addresses the in-formation requirement of the student bydefining and modifying that need whilestandard 2.4.b refines this need based onexamination of the data or their absence.Other responses identified visual, kines-thetic, and auditory learning as the mostrewarding experiences, with commentsranging from students' appreciation ofthe well-constructed PowerPoint pre-sentations, the lecture content, and theworksheet and group activities. A fewstudents noted their future application ofthis knowledge through statements suchas, "when I perform searches anywhereelse, I will know how to improve theirquality," a close association to standard3.7.c; "learning valuable information thatwill benefit me not only with this researchbut with other research in the future"; and"I actually learned something that is go-ing to be helpful in the future." Again, wefound support for the ACRL InformationLiteracy outcomes objective of applyinginformation literacy skills outside theclassroom -specifically in standards 1.1f,2.2.a, 2.4, and 4.1.b. As a result, we felt


n =

None, N/A, or illegible 93

Learning 52


Evaluation 36

Complexity of research 26

Confidence 24

Engagement in class 22

Library resources 17

Miscellaneous 40

TOTAL 348

we had accomplished one of our primarycurriculum objectives.

Results for Question 4Learning: Synthesis and ApplicationWhen evaluating responses for this ques-tion, we noted that learning was identifiedas the most surprising experience. Stu-dents responded in a way that focused ontheir newly found abilities to synthesizeand apply the knowledge of the lectureand group activities when constructingsearch strategies and undertaking theresearch process. Many students ex-pressed their surprise after recognizingthe similarities and differences in datafields that existed among databases,such as thesauri and descriptors. Thoseresponses reflected the beneficial use ofstandards 2.2.c-f and 2.5.d. Students alsocommented on how the use of Booleanoperators, truncation and wildcards, andlimiters could refine their searches, refer-ring back to standard 2.2.d.

Students also expressed feelings ofsurprise in the idea that their researchimproved greatly by applying thesenew concepts of searching. During thefirst session, many students identified,both in their CIQ responses and throughcomments recorded by instructors afterclasses, that they felt confident in theirability to perform research with searchengines and did not see a priority for at-tending library instruction classes. Aftercompleting the sessions, however, stu-dents noted surprise in how much moreeffective their research strategies hadbecome. Search engine proselytes seem-ingly became library database converts!Reflections such as "[I was surprised] howmany books are available on my topic,"or "how much information I found on mytopic through the databases here in thelibrary," exhibited the students' recogni-tion of the value of library resources. Thisvalue assigned by the students highlightsstandard 2.1.c, where students examinethe scope and organization of informationretrieval systems, drawing conclusionsbased upon their findings as stressed

TABLE 4Results For Question 4: What WasYour Most Surprising Experience

in Today's Class?


in standard 3.4. And, finally, standard5.1.b facilitates contemplation of free vs.fee-based accessible information and thesubsequent issues students do not realizeexist. Other students expressed happinessor gratitude in being able to apply thisnew knowledge to future research. Onestudent noted that "[after this class] I willbe able to look up information for otherpapers, because now I think it is mucheasier to do." As educators, we were alsosurprised and encouraged that we weremaking a difference and contributing tothe learning process in ways that wereunexpected by the students.

EvaluationResponses on this question also revealeddeeper reflections on the evaluation ofInternet resources. Again, at the begin-ning of the library instruction curriculum,several students had commented on theirconfidence in performing research - usingWorld Wide Web search engines--sup-porting the University of California andCalifornia State University system librariesinformation literacy studies.5 ° However,librarians provided lecture and group ac-tivities, incorporating a map of the WorldWide Web and an evaluation of falsifiedresearch, parody, and propaganda Websites.51 Many students expressed their

TABLE 5Results For Question 5: Was ThereInformation Presented in Today'sClass That is Still Unclear to You?

If So, Please Describe


n=

None, N/A, or illegible 318

Learning 9


Library resources 4

Evaluation 3

Miscellaneous 7

TOTAL 348

September 2008

surprise at the amount of misinformationand disinformation on the Internet, stress-ing the importance of standards 3.2.a, 3.2.c,and 3.6.a. Equally, as a result of groupactivities in the classroom and subsequentapplication of these criteria for analysis,students commented that they now feltmore confident about evaluating Internetresources due to a better understandingof domain use and identifying authorcredentials, reflecting standard 1.3.b.

Results for Question 5The majority of responses for Question5 were either left blank, or studentsresponded with "no" or "none." Thosestudents who did respond, however,ranged in their comments from specific tocomplex aspects of the research process.Many students discussed specific aspectsof library resources, such as limiters in theonline catalog or full-text retrieval optionsin subscription databases as still beingunclear. Understandably, students stressedsome confusion over when to use particu-lar resources, such the use of CQ Researcherover EBSCO's Academic Search Premierand vice versa. Standards 1.2.c, 2.2.e-f,and 2.3.a are designed to guide studentsto differentiate between resources, applysearch strategies in various informationretrieval systems to increase success, andincorporate discipline-specific techniqueswhen appropriate. Other students, how-ever, noted confusion over Boolean searchstrategies or the evaluation of resources.

We found it difficult to develop themesfor this question, as the students' respons-es related to specific aspects of the day'sclass and were often contingent on theirown perceptions of the day's events inrelation to their student peers, instructors,and/or teaching methods. However, weidentified this phenomenon as supportingTripp's argument that critical reflectioncannot be socially and culturally extri-cated from our students and that the class-room realities created by students can besocially constructed.5 2 Additionally, thecomments that students chose to recordon information that was still unclear at the


C)

0

C)

tC)

~ E .~u a0'.2

C- 00 ý. - "

0

C)

0

C)C)CtEaC)

0

�C)

a0

C)0

0

C)

C)C)Ct

C)0

�C)

0

C)0

CIDC_

CZ

C)

0-COS-

73 1rr_

C13a)Ij

Ona

C)

CC

r-

-aC)

CCC)

� En-C)

0...

a

CC

Lu

aaCCaCC

Lu

C"C"

a0•

CC

a0

CCaCC

Lu

C)>� C.)CCa'-0

.0

C)

CCa'�0.0

00

U

C)

C)C.

end of the day's session were incorporatedby instructors to review for the next day'sclass, highlighting the recursive learningprocess integrated into the curriculum.Moreover, we found that those studentswho did choose to respond seemed tonote consistently their realization that theclassroom content was not as easy as theyhad expected and/or that they would needto practice what they had learned outsideof class when they had more time to applytheir new knowledge. Those reflectionsreinforce aspects of standards 1.1, 1.3,2.2, 3.4, and 4.1.

ConclusionFor the methodological framework of thestudy, we feel the use of the Critical Inci-dent Questionnaire supported the worksof Brookfield53 and Tripp' throughout thisresearch. Responses generated by studentsconsistently seemed to uphold the theorythat critical reflection takes place as a resultof critical incidents and events. We alsoidentified Segal's5" work in the responsesof students who noted disorienting ordisrupting incidents in the classes andhow these led to rich descriptions in thecritical reflection process. As an example,when students were confused, criticalreflection by the students on particular is-sues seemed to take place at a deeper levelas a result of their confusion. Through theprocess of writing their responses, theyexpanded on why they were confused,exhibiting an iterative learning cycle thatdeveloped through the course of the entireblock of instructional sessions. At anotherlevel, the things students most understoodin Question I also came up as emergentthemes for concepts they found most dif-ficult to comprehend in Question 2 and/orQuestion 5. Moreover, their reasons forconfusion seemed to differ significantlyfrom those who identified the same ideasas most understandable. We contend thisfurther supported Tripp's concept of thesocial creation of knowledge during theprocess of learning. Furthermore, wesuggest that, when evaluating students,using methods that focus on quality in

C)I-C)

I-C)

CuC)

C)

3

3-C)


education, critical reflection analysis tendsto support why and how learning is tak-ing place as opposed to measuring onlylearning content.

Another phenomenon that developedduring this study is the emergence ofthemes across questions (see Table 6).Learning, search strategies, and evalua-tion were themes that remained constantin each of the questions. Library resourcesas a theme spanned Questions 3, 4, and5, while time was identified in bothQuestions 1 and 2. Confidence equallyemerged as a theme in both Questions 3and 4. We found it particularly compel-ling that the concepts of learning andevaluation spanned all of the questionsand helped lend support for: 1) the useof the CIQ instrument as a method forassessing the critical reflection process,and 2) that learning and evaluation, twothemes one could infer would elicit criti-cal reflection, emerged as a result of thecurriculum and pedagogy. We are unsurewhy the themes of library resources, time,and confidence emerged consistently intheir respective questions and believemore research would be necessary toinvestigate these phenomena.

It is inferred that Woods5" was sup-ported in some aspects of the studythrough reflections that were more radicalin nature. However, we do not believe thissupport was strong for this study. Thiscould be due to the limitations of the CIQinstrument and the limited time frame forobservation, reflection, and discovery ofmore radical critical events. An equallyfrustrating dilemma we faced was thelarge number of "nonresponses" fromrespondents. These ranged from "N/A"to "yes" or simply blank responses forsome of the questions. We can attempt tomake assumptions about why this hap-pened: apathy toward survey researchby students, a lack of engagement in theclass, or full learning engagement and un-derstanding of the curriculum outcomes.However, we feel further research wouldneed to be conducted to help explain whystudents responded in this way.

September 2008

Another aspect of this study thatbecame somewhat problematic for theresearchers was the size and scope of thepopulation. On the one hand, we felt itimportant to extend participation to anentire population to gain description ofreflection at a level and scope extensiveenough to analyze the critical reflectionprocess. Although this was not an insur-mountable attempt, we quickly realizedthat the size of the data sets from thesample (n=348) were quite large andpresented a lengthy period of analysis.We feel that it might be beneficial to in-corporate the use of the CIQ only once,at the end of the summation of sessions.However, we first addressed this pos-sibility at the beginning of the study, andwe believed that a true assessment of thecurriculum would require the use of theCIQ after each session. Otherwise, studentrecall of the previous sessions would beminimized, skewing the data primar-ily toward reflection on the last sessionwhen the CIQ would be performed. Inattempting to apply the CIQ as an as-sessment instrument at other institutionsthat incorporate multisession informa-tion literacy or instructional technologycurricula, we might recommend the useof stratified sampling among differentcourses to increase the timeliness of dataanalysis. Additionally, we believe the useof the CIQ might be valuable as an effec-tive assessment method for informationliteracy curricula that incorporate singlesessions for each course.

The main limitation of this study sur-rounds the concept of critical reflection ingeneral. The theoretical framework relieson the qualitative interpretations of theresearchers. Although multiple methodswere utilized to limit the influence of theresearchers' subjectivity throughout thisstudy, it cannot be discounted.5 7 Equally,it should be recognized that each campusand each student brings with them uniquesociocultural perspectives and academicexperiences that influence student re-sponses in a study such as this. Althoughgeneralizations about the findings of


this study to other populations mightbe inferred, it is not our intent to applythis knowledge across undergraduatestudents in library instruction. Moreover,descriptive statistics were used onlyfor exploratory purposes for our ownreflection on practice and to assist in de-veloping hypotheses for future studies ofinformation literacy.

In conclusion, we believe the CIQproved to be an effective qualitativeinstrument to assess critical reflec-tion and critical incidents during theprocess of learning. Although thisstudy focuses on information literacycurriculum, similar research couldbe performed in any program thatis complementary in nature in termsof time segments and instructionalobjectives, such as in writing centers,instructional technology programs, orcomputer literacy courses. Equally, wefeel this assessment approach wouldprove effective for semester-long

courses in information literacy, LibraryScience, or other social and behavioralsciences disciplines. As was detailedin the responses to each of the ques-tions, themes that emerged showedthe process of critical reflection takingplace consistently between instructorsand throughout each of the sessions.These themes identified the key cur-riculum components of the programand expanded upon them, suggestingthat the curriculum was effective inboth content and design to support theACRL Information Literacy Compe-tency Standards for Higher Educationspecifically and to elicit critical thoughtduring learning in general. Perhaps justas important, the themes we identifiedwere instrumental in our reflections onpractice. The knowledge that we gainedfrom this study helped us to further de-velop the curriculum while continuingto focus on both critical reflection andthe ACRL Standards in new ways.

Appendix A

Student ID #:Please check or write in your responses below:

Age

Year in school Freshman __ Sophomore - Junior __ Senior

Current or estimated GPA:3.5-4.0 3.0-3.4 2.5-2.9 2.0-2.4 1.5 or below-1.9

Sex: M F

Have you participated in a Library Instruction session at the SWOSU Libraries duringa previous semester? __ Yes __ No

Are you a transfer student from a college or university other than SWOSU?__. Yes __ No


Appendix B

Student ID #: Instructor #:

In one or two sentences, please respond to each of the questions below about yourexperiences in today's Library Instruction class:

1. At what moment in the class today did you feel like you most understood the in-structional content?

2. At what moment in the class today did you feel most confused about the instruc-tional content?

3. What was your most rewarding experience in today's class?

4. What was your most surprising experience in today's class?

5. Was there information presented in today's class that is still unclear to you? If so,please describe.

Notes

1. "Presidential Committee on Information Literacy: Final Report," The American LibraryAssociation, January 1989. Available online from www.ala.org/ala/acrl/acrlpubs/whitepapers/presidential.htm. [Accessed 1 July 20081.

2. "Information Literacy Competency Standards for Higher Education: Standards, Outcomes,and Performance Indicators," Association of College & Research Libraries, January 2000. Availableonline from www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm. [Accessed1 July 2008].

3. Stephen Brookfield, Becoming a Critically Reflective Teacher (San Francisco: Jossey-Bass,1995).

4. David Tripp, Critical Incidents in Teaching: Developing Professional Judgment (London: Rout-ledge, 1993).

5. Peter Woods, Critical Events in Teaching and Learning (Washington, D.C.: Falmer Press,1993).

6. Stephen Brookfield, The Skillful Teacher: On Technique, Trust, and Responsiveness in the Class-room (San Francisco: Jossey-Bass, 1990).

7. L.A. Daloz, Effective Teaching and Mentoring: Realizing the Transformational Power of AdultLearning Experiences (San Francisco: Jossey-Bass, 1986).

8. Tripp, Critical Incidents in Teaching.9. Woods, Critical Events in Teaching and Learning.

10. James Henri and Ken Dillon, "Learning to Learn: Reflections upon Enquiry, InformationLiteracy and Critical Thinking," Austialian Library Journal 41 (May 1992): 103-17.

11. Tripp, Critical Incidents in Teaching.12. Ibid., 29.13. Ibid., 25.14. Woods, Critical Events in Teaching and Learning.15. Patricia J. Sikes, Lynda Measor, and Peter Woods, Teacher Careers: Crises and Continuities

(Philadelphia: Falmer Press, 1985): 230.16. Stephen Segal, "Existential Conditions of Explicitness: An Heideggerian Perspective,"

Studies in Continuing Education 21, no. 1 (1999): 73-90.17. Jack Mezirow, "Postmodem Critique of Transformation Theory: A Response to Pietrykows-

ki," Adult Education Quarterly 49, no. 1 (1998): 65-67; Jack Mezirow, "Transformation Theoryout of Context," Adult Education Quarterly 48, no. 1 (1997): 60-62; Jack Mezirow, TransformativeDimensions of Adult Learning (San Francisco: Jossey-Bass, 1991).

18. Donald Sch6n, Beyond the Stable State (New York: Random House, Inc., 1971); Donald Sch6n,

September 2008


The Reflective Practitioner: How ProfessionalsThink in Action (Averbury, U.K.: AshgatePublishing Limited, 1991).

19. Teresa Thiel, "Reflections onCritical Incidents," Prospect 14, no. 1 (Apr.1999): 44-52.

20. Stacy Miller, "Students as Agentsof Classroom Change: The Power of Cul-tivating Positive Expectations," Journal ofAdolescent & Adult Literacy 48, no. 7 (2005):540-46.

21. Donna M. Weinreich, "Service-Learning at the Edge of Chaos," Education-al Gerontology 29, no. 3 (2004): 181-95.

22. Panayiotis Angelides and MelAinscow, "Making Sense of the Role ofCulture in School Improvement," SchoolEffectiveness and School Improvement 11, no.2 (2000): 145-63.

23. Mary Jackson and Diane Wasson,"Critical Thinking Requires Critical Do-ing: An Analysis of Students' Multicul-tural Experiences within Freire's Frame-work," paper presented at the AmericanEducational Research Association, Chi-cago, Ill. (ERIC Document ReproductionService, No. ED478164, Apr. 2003).

24. Lesley S.J. Farmer, "Narrative In-quiry as Assessment Tool: A Course CaseStudy," Journal of Education for Library andInformation Science 45, no. 4 (Fall 2004):340-51.

25. Kevin J. Pugh and David A. Bergin,"The Effect of Schooling on Students'Out-of-School Experience," EducationalResearcher 34, no. 9 (2004): 15-23.

26. Graham Walton and Sarah Nettle-ton, "Reflective and Critical Thinking inUser Education Programmes," BritishJournal of Academic Librarianship 7, no. 1(1992): 31-43.

27. Donald Sch6n, The Reflective Prac-titioner.

28. Walton and Nettleton, "Reflectiveand Critical Thinking."

29. Stephen Brookfield, "CriticallyReflective Practice," Journal of ContinuingEducation in the Health Professions 18 (1998):197-205; Stephen Brookfield, "Experien-tial Pedagogy: Grounding Teaching inStudents' Learning," Journal of ExperientialEducation 19, no. 2 (1996): 62-68.

30. "Information Literacy CompetencyStandards for Higher Education."

31. Woods, Critical Events in Teachingand Learning; Tripp, Critical Incidents inTeaching; Brookfield, Becoming a CriticallyReflective Teacher.

32. Brookfield, Becoming a CriticallyReflective Teacher.

33. Tripp, Critical Incidents in Teaching,39.

34. Norman K. Denzin and Yvonne S.

VICTORIANFICTION

An Online Research Guide

Lionel Stevenson's Victorian Fiction: A Guideto Research (1964) covered eleven novelists.George H. Ford's Victorian Fiction: A SecondGuide (1978) added six more. But the lesserVictorian novelists still remained unknown.Their exclusion, as Sheila M. Smith pointed

out, "gives a distorted view of the literaryactivity of the nineteenth century."

Victorian Fiction: An Online Research Guideoffers comprehensive coverage of fifty-onemajor novelists, such as Dickens, GeorgeEliot, the Brontes, Elizabeth Gaskell, andThomas Hardy; minor novelists, such asEdward Bulwer-Lytton, Charles Reade,Samuel Butler, and George Gissing; and

underread novelists, such as George Egerton,Vernon Lee, Amy Levy, Eliza Lynn Linton,

Ouida, and John Oliver Hobbes

It includes editions of primary works andcontemporary reviews and critical works from1830 onward, and provides chronologies andinformation on biographies of novelists and

their manuscript holdings

Conduct sophisticated search of the entire35,000+ entries on books, articles, and

dissertation abstracts, each with linked &cross-referenced subject words and phrases

on areas of interest

Or narrow down your search to a limitednumber of recommended and annotated

essential entries on each novelist ornovel selected from hundreds of peer-

reviewed books and articles

* Coverage from 1830 to the present*Information on primary works &

manuscript holdings * Chronologies andeditions *Annotated entries with

subject keywords* Peer-reviewed books and articles identified

* Searching, printing & saving

of essential entries* Fast, accurate & relevant

For more information, please check

www.victorianfiction.com


Lincoln, Handbook of Qualitative Research (Thousand Oaks, Calif.: Sage Publications, 1994).35. Brookfield, The Skillful Teacher; Brookfield, Becoming a Critically Reflective Teacher.36. Clara Hoover, "We Don't Have to Learn Anything; We Just Have to Find the Answer,"

School Library Media Activities Monthly XXII, no. 2 (2005): 26-28.37. Sch6n, The Reflective Practitioner; Sch6n, Beyond the Stable State.38. Jacqueline Kracker and Peiling Wang, "Research Anxiety and Students' Perceptions of

Research: An Experiment, Part II, Content Analysis of Their Writings on Two Experiences,"Journal of the American Society for Information Science and Technology 53, no. 4 (2002): 295-307.

39. Denzin and Lincoln, Handbook of Qualitative Research.40. Ibid.41. Kathleen Dunn, "Assessing Information Literacy Skills in the California State University:

A Progress Report," Journal of Academic Librarianship 28, no. 3 (2002): 26-35.42. Patricia Davitt Maughan, "Assessing Information Literacy among Undergraduates: A

Discussion of the Literature and the University of California-Berkeley Assessment Experience,"College & Research Libraries 26, no. 1 (2001): 71-85.

43. Denzin and Lincoln, Handbook of Qualitative Research.44. Denzin and Lincoln, Handbook of Qualitative Research; Joe L. Kincheloe and Kathleen S.

Berry, Rigour and Complexity in Educational Research: Conceptualizing the Bricolage (Berkshire, U.K.:Open University Press, 2004).

45. "Information Literacy Competency Standards for Higher Education."46. Segal, "Existential Conditions of Explicitness"; Tripp, Critical Incidents in Teaching; Brook-

field, Becoming a Critically Reflective Teacher; Woods, Critical Events in Teaching.47. Tripp, Critical Incidents in Teaching; Brookfield, Becoming a Critically Reflective Teacher; Woods,

Critical Events in Teaching.48. David A. Kolb, Experiential Learning: Experience as the Source of Learning and Development

(Englewood Cliffs, N.J.: Prentice-Hall, 1984).49. William E. Doll, Jr., A Post-Modern Perspective on Curriculum (New York: Teachers College,

Columbia University, 1993).50. Dunn, "Assessing Information Literacy Skills"; Maughan, "Assessing Information Literacy

among Undergraduates."51. "Dihydrogen Monoxide Research Division," available online from www.dhmo.org/facts.

html. [Accessed 1 July 2008]; "The White House: Officious Website of George W. Bush," avail-able online from http://whitehouse.org/. [Accessed 1 July 2008]; "Martin Luther King, Jr. - A TrueHistorical Examination," Available online from www.martinlutherking.org/. [Accessed 1 July2008].

52. Tripp, Critical Incidents in Teaching.53. Brookfield, Becoming a Critically Reflective Teacher.54. Tripp, Critical Incidents in Teaching.55. Segal, "Existential Conditions of Explicitness."56. Woods, Critical Events in Teaching and Learning.57. Denzin and Lincoln, Handbook of Qualitative Research.

September 2008

COPYRIGHT INFORMATION

TITLE: Assessing Learning, Critical Reflection, and QualityEducational Outcomes: The Critical IncidentQuestionnaire

SOURCE: Coll Res Libr 69 no5 S 2008

The magazine publisher is the copyright holder of this article and itis reproduced with permission. Further reproduction of this article inviolation of the copyright is prohibited. To contact he publisher:http://www.ala.org/

assessing learning, critical reflection, and quality ......assessing learning, critical reflection,...

Documents