report of critical thinking assessment, 2015-2016critical thinking scores writing intensive courses,...

ReportonAssessmentofCriticalThinking October2016 1

Report of Critical Thinking Assessment, 2015-2016

Submitted October 2016

Prepared by Dr. Rebecca Ertel, Office of Academic Planning and Assessment for the

Senate General Education and Writing-Across-the-Curriculum Committee

General Education Critical Thinking Outcome Overview

The critical thinking component of CSU’s general education curriculum has the following learning outcome and sub-outcomes:

Outcome: Students will think critically

Students will apply the habits of inquiry and analysis to various situations to

• Accurately interpret evidence (statements, graphics, questions, etc.) • Identify relevant arguments (reasons, claims, pros and cons, etc.) • Analyze and evaluate alternative points of view • Justify key results or procedures and explain assumptions and reasons • Fair-mindedly follow where evidence and reason lead

Critical Thinking Assessment Description: Spring 2015-Spring 2016

Signature Assignments

Writing samples (Signature Assignments) were collected from 129 first-year students enrolled in English 1100 and 1101 during fall semester 2015 and from 54 senior students enrolled in upper-division writing intensive (WI) courses during spring semester 2016.

Samples were collected from four (4) sections of English 1100 offered during Fall 2015; one section did not submit samples for scoring due to late scheduling of the course. Samples were collected from five (5) sections of English 1101 offered during Fall 2015; four (4) sections did not submit samples due to instructor oversight or late scheduling.

Samples were collected from eleven (11) sections of writing intensive courses offered during Spring 2016; samples were not returned from six (6) writing intensive courses. Writing Intensive courses were not offered by fourteen (14) programs, primarily because the WI courses are offered in alternate years.

The samples were scored by the instructor of the section from which they were taken and a member of the Written Communication Faculty Learning Community. The two scores were averaged to produce a final score for the sample. In cases in


which the scores differed by more than one level on the rubric, the sample was scored by a third reader and the two closest scores were averaged to produce the final score.

The samples were assessed using a modified version of the American Association of Colleges and University’s VALUE rubric for written communication. A calibration session was held for the scorers prior to scoring the samples. The percent of agreement on the category scores was .72, above the .7 thresh hold generally accepted to establish reliability (Jonsson and Svingby, 2007).

Discussion of Scores for English 1100 and English 1101

As expected, the Fall 2015 assessment of Signature Assignments found that more than 80% of students in English 1100 and 1101 begin their first-year performing at the Benchmark (1) or Milestone 1 (2) levels on the rubric; the scores are about equally divided between 1’s and 2’s.

Critical Thinking Scores English 1100/1101, Fall 2015

Table1.

Students scored highest in the area of selecting evidence and weakest in the drawing conclusions and assessing consequences.

Comparison of English 1100 and 1101

While students in both courses scored primarily 1 or 2, more than a quarter of English 1101 students scored 3 or 4 while only about 7% of students in 1100 scored 3 or 4. Student samples from 1100 did not reflect large differences among category scores. Student samples from English 1101 showed students tended to score lower in the categories of “Explanation of Issues” and “Evidence.”


English 1100 Critical Thinking Scores, Fall 2015

Table 2.

English 1101 Critical Thinking Scores, Fall 2015

Table 3.

English 1100/1101 Score Distribution

Table 4.


Discussion of Scores for Writing Intensive Courses

The Spring 2016 assessment of Signature Assignments found that the samples submitted from senior students in upper-division writing intensive courses are evenly distributed between samples scored 1 or 2 and samples scored 3 or 4. Students scored lowest in the area of drawing conclusions and highest in the areas of “Explanation of Issues” and “Influence of Context and Assumptions.”

Critical Thinking Scores Writing Intensive Courses, Spring 2016

Table 5.

Comparison of English 1100 and 1101and Writing Intensive Courses

The data suggest most students make measurable gains in critical thinking between their first and senior years. Fifty percent (50%) of samples from senior students enrolled in upper-division writing intensive courses had a composite score of 3 or 4. In comparison, only 18.6% of samples from students enrolled in English 1100 or 1101 had composite scores of 3 or 4. Samples were scored lowest on drawing conclusions from evidence at both the first-year and senior levels.

Distribution of Scores in English 1100/1101

Table 6.


Distribution of Scores in Writing Intensive Courses

Table 7.

Recommendations of the GenEd/WAC Committee

The GenEd/WAC committee recommends the University Senate direct academic programs that offer general education courses supporting the learning outcome for Critical Thinking to identify teaching strategies and learning activities to incorporate into these courses with the goal of improving student learning in the area of analyzing evidence and drawing conclusions. All degree programs should similarly strengthen instruction in these abilities in their curricula. Programs should report their actions to the GenEd/WAC Committee by the end of March. The GenEd/WAC Committee will provide a summary report to the Senate by the end of the academic year.


References

Jonsson, A. & Gunilla, S. (2007). The use of scoring rubrics: Reliability,

validity and educational consequences. Educational Research Review,

2, 130–144

Stoddart, T., Abrams, R., Gasper, E., & Canaday, D. (2000). Concept maps

as assessment in science inquiry learning—A report of methodology.

International Journal of Science Education, 22, 1221–1246


Appendix A

CentralStateUniversity

English1100:IntroductiontoWritingandReadingforCollege(5)

CourseSyllabus

CatalogDescriptions:

ENG.1100IntroductiontoWritingandReadingforCollege(5credits,I,II):AnintensiveintroductiontowritingandreadingforCollege.Studentswillreadliteraryandnon-literarytextsandcomposeessaysthatdemonstrateproficiencyincollegelevelwritingandmechanics.Atleastonepaperwillbeareader-responseessaybasedonaliterarytext.Studentswillalsobeintroducedtothebasicprinciplesofdocumentationandwriteoneessayusingdocumentation.Lecture/discussionperiodswillfocusondiscussionsofassignedreadingsandintroducingtheconventionsofacademicprose,includingelementsofStandardEnglishgrammarandmechanics.Individualizedandsmall-groupworkonthewritingprocessincluded.RequiredofstudentswhoseperformanceontheEnglishPlacementTestindicatesaneedforintensivewritinginstruction.EquivalenttoEnglish1101forgeneraleducationrequirements.

Textbooks:

Maimon,ElaineP.andJaniceH.Peritz.WritingIntensive:EssentialsforCollegeWriters.Boston:McGrawHill,2009,2007.Print.

ConnectWriting2.0.Boston:McGrawHill,2013.Online.

LearningOutcomes

1. Recognizeelementsthatcontributetorhetoricalsituations,includingpurpose,audience,stance,andappropriateconventions.

2. Criticallyexaminepersonalviewsincomparison/contrastwiththoseexploredintexts.3. Identify,analyze,anddiscussthemesandstructuresoftexts.4. Applytheprocessofwritingthroughdrafting,revising,andediting,attimesin

collaborationwithothers.5. Applywritingprocesstoproduceeffectiveandfullydevelopedessays.6. Usecorrectgrammar,sentencestructure,andpunctuation.7. Exploreadiverserangeofauthors,cultures,andperspectives.8. Useelectronicenvironmentstodraftandpublishworkwherepossible.

Policies/Requirements:

1. Writeacombinationofessays,annotatedbibliographies,and/orshortercompositionsofformalwritingthatisroughlyequivalenttotwentypages.

2. Writeonesource-supportedessayfollowingtheMLAformat.


3. Completeallassignmentsaccordingtoscheduledduedates,includingallassignmentsfromConnectWriting2.0.TheConnectWriting2.0assignmentsshouldtotalatleast10%ofthefinalgrade.

4. Participateinallclassdiscussions,activities,andassignments.5. Takethedepartmentalfinalexaminationattheprescribedtime.6. Avoidplagiarism.Plagiarism,thesubmittingofanotherstudent’sworkormaterialfromother

printedsourcesasone’sownwork,mayresultinthegradeof"F"forthepaperorforthecourseitself.

7. Attendclassontimeregularly.Studentsareresponsibleforobtainingmaterials,assignments,andinformationgivenonmissedclassdays.

8. InstructorsmaydevelopandenforceotherpoliciesconsistentwiththecurrentCSUcoursecatalog,departmentalguidelines,andstandardacademicpractices.Veteraninstructorsmaypickanalternatesupplementaltextbook.

CriteriaforGrading:

Thegradefortheclasswillbedeterminedbythestudent’slevelofcompetenceinclassassignmentsandactivities,homework,tests,quizzes,andessays.InstructorsmaydevelopandenforceattendanceandotherpoliciesconsistentwiththecurrentCSUcoursecatalogandDepartmentalguidelines.

Eachinstructorwilldevelopacoursesectionsyllabusthatdiscussesspecificmethodsofdelivery,topics,activities,andassignments.Thefinalexamwillcountasatleast10%ofthefinalgrade.ENG1100usestheuniversity’sA-FandIgradingscale.

AmericanswithDisabilitiesCompliance:CentralStateUniversityiscommittedtoincludingstudentswithdisabilitiesasfullparticipantsinitsprograms,services,andactivitiesthroughcompliancewithSection504oftheRehabilitationActof1973andtheAmericanswithDisabilitiesAct(ADA)of1990.Ifyouareastudentwithadocumenteddisabilitypleasecall

Dr.WandaHadleyat937-376-6479todiscussyourcourseaccommodations.

Knowledge,Skills,andDispositions

Atthecompletionofthecoursethestudentwilldemonstrateknowledge,skills,anddispositionsasfollows:

Knowledge:

1. Explainthetheme,structure,andmeaningofliteraryandnon-literarytexts.2. Knowhowtodevelopathesis.3. Explaintherelationshipbetweendetailsandthemesoftexts.4. Understandwhatconstitutesevidenceandhowtodeterminethemosteffective

form.5. Developtheabilitytoproduceeditedwritingaccordingtotheconventionsof

StandardAmericanEnglish.6. Understandhowtodocumentprimaryandsecondarysourcesinessays.


7. Identify,analyzeanddiscussthemesandstructuresofliteraryandnon-literarytexts.

Skills:8.Defendthemesbyidentifyingvalidsupportingideasandevidence.

9.Writeeffectivethesisstatements.

10.Writeeffective,fullydeveloped,andorganizedessays.

11.Synthesizeideasfromreadingsintootheractivitiesandwriting.

12.Useanddocumentsecondarysources.

13.Useamoreextensive,sophisticatedvocabulary.

14.Writeforclarity,organization,anddevelopment.

15.Usecorrectgrammar,sentencestructure,andpunctuation.

Dispositions:16.Understandandappreciateavarietyofliteraryandnon-literarytexts.

17.Examinepersonalviewsincomparison/contrastwiththoseexploredintexts.

18.Exploreadiverserangeofauthors,cultures,andperspectives.

19.Learntherelationshipbetweenliteratureandlife/universaltruths.

20.Understandtheimportanceofeffectivewrittencommunication.

21.Developconfidenceinrespondingtooral/writtenprompts.

22.Understandtheimportanceofdocumentation.


CentralStateUniversity

English1101:IntroductiontoWritingforCollege(4)

CourseSyllabus

CatalogDescriptions:

ENG.1101(4credits,1,II,III)IntroductiontoWritingforCollege:Informativewritingbasedonliteraryandnon-literarytexts.Studentswillcomposeessaysthatdemonstrateproficiencyincollege-levelwritingandmechanics.Atleastonepaperwillbeareader-responseessaybasedonaliterarytext.Studentswillalsobeintroducedtothebasicprinciplesofdocumentationandwriteoneessayusingdocumentation.Individualizedandsmall-groupworkonthewritingprocessincluded.

Textbooks:


Kennedy,X.J.,BedfordGuideforCollegeWriterswithReader,NinthEdition.NewYork:Bedford/St.Martin’s,2011.Print.

LearningOutcomes

1. Recognizeelementsthatcontributetorhetoricalsituations,includingpurpose,audience,stance,andappropriateconventions.

2. Criticallyexaminepersonalviewsincomparison/contrastwiththoseexploredintexts.3. Identify,analyze,anddiscussthemesandstructuresoftexts.4. Applytheprocessofwritingthroughdrafting,revising,andediting,attimesin

collaborationwithothers.5. Applywritingprocesstoproduceeffectiveandfullydevelopedessays.6. Usecorrectgrammar,sentencestructure,andpunctuation.7. Exploreadiverserangeofauthors,cultures,andperspectives.8. Useelectronicenvironmentstodraftandpublishworkwherepossible.

Policies/Requirements:

1. Writeacombinationofessays,annotatedbibliographies,and/orshortercompositionsofformalwritingthatisroughlyequivalenttotwentypages.

2. Writeonesource-supportedessayfollowingtheMLAformat.3. Completeallassignmentsaccordingtoscheduledduedates.4. Participateinallclassdiscussions,activities,andassignments.5. Takethedepartmentalfinalexaminationattheprescribedtime.6. Avoidplagiarism.Plagiarism,thesubmittingofanotherstudent’sworkormaterialfromother

printedsourcesasone’sownwork,mayresultinthegradeof"F"forthepaperorforthecourseitself.


7. Attendclassontimeregularly.Studentsareresponsibleforobtainingmaterials,assignments,andinformationgivenonmissedclassdays.

8. InstructorsmaydevelopandenforceotherpoliciesconsistentwiththecurrentCSUcoursecatalog,departmentalguidelines,andstandardacademicpractices.Veteraninstructorsmaypickanalternatesupplementaltextbook.

CriteriaforGrading:

Thegradefortheclasswillbedeterminedbythestudent’slevelofcompetenceinclassassignmentsandactivities,homework,tests,quizzes,andessays.InstructorsmaydevelopandenforceattendanceandotherpoliciesconsistentwiththecurrentCSUcoursecatalogandDepartmentalguidelines.

Eachinstructorwilldevelopacoursesectionsyllabusthatdiscussesspecificmethodsofdelivery,topics,activities,andassignments.Thefinalexamwillcountasatleast10%ofthefinalgrade.ENG1100usestheuniversity’sA-FandIgradingscale.

AmericanswithDisabilitiesCompliance:CentralStateUniversityiscommittedtoincludingstudentswithdisabilitiesasfullparticipantsinitsprograms,services,andactivitiesthroughcompliancewithSection504oftheRehabilitationActof1973andtheAmericanswithDisabilitiesAct(ADA)of1990.Ifyouareastudentwithadocumenteddisabilitypleasecall

Dr.WandaHadleyat937-376-6479todiscussyourcourseaccommodations.


Atthecompletionofthecoursethestudentwilldemonstrateknowledge,skills,anddispositionsasfollows:

1. Knowledge:

2. Explainthetheme,structure,andmeaningofliteraryandnon-literarytexts.3. Knowhowtodevelopathesis.4. Explaintherelationshipbetweendetailsandthemesoftexts.5. Understandwhatconstitutesevidenceandhowtodeterminethemosteffectiveform.6. DeveloptheabilitytoproduceeditedwritingaccordingtotheconventionsofStandard

AmericanEnglish.7. Understandhowtodocumentprimaryandsecondarysourcesinessays.8. Identify,analyzeanddiscussthemesandstructuresofliteraryandnon-literarytexts.

Skills:

1. Defendthemesbyidentifyingvalidsupportingideasandevidence.2. Writeeffectivethesisstatements.3. Writeeffective,fullydeveloped,andorganizedessays.4. Synthesizeideasfromreadingsintootheractivitiesandwriting.5. Useanddocumentsecondarysources.6. Useamoreextensive,sophisticatedvocabulary.


7. Writeforclarity,organization,anddevelopment.8. Usecorrectgrammar,sentencestructure,andpunctuation.

Dispositions:

1. Understandandappreciateavarietyofliteraryandnon-literarytexts.2. Examinepersonalviewsincomparison/contrastwiththoseexploredintexts.3. Exploreadiverserangeofauthors,cultures,andperspectives.4. Learntherelationshipbetweenliteratureandlife/universaltruths.5. Understandtheimportanceofeffectivewrittencommunication.6. Developconfidenceinrespondingtooral/writtenprompts.7. Understandtheimportanceofdocumentation.


CentralStateUniversityEnglish1102:WritingandResearchingtheEssay

CourseSyllabus

CatalogDescription

ENG1102(4credits,I,II,III)WritingandResearchingtheEssay:Research-basedargumentativeandanalyticalwritingusinglibraryandInternetsources.Studentswillcontinuetodevelopproficiencyincollege-levelwritingandmechanics.Theywillwritebriefessaysandatleastonelongerresearchpaper,properlyformatted,usingbothprintandelectronicsources.Theresearchpaperwillbeorientedtowardthestudent’smajorfieldorareaofinterest.CourseincludesreviewfortheEnglishProficiencyexam.Prerequisite:GradeofCoraboveinENG1100or1101orequivalent.

Textbooks


Lunsford,AndreaA.AndJohnJ.Ruszkiewicz.Everything’sanArgument.Boston:Bedford/St.Martin’s,2009.Print.

LearningOutcomes

1. Analyzeargumentativestrategiesandrhetoricalappeals.2. Developappropriateargumentativestrategiesandrhetoricalappealsinessays.3. Find,evaluate,andusesourcesasrecommendedbyacademicmajororareaofinterest.4. Writeunifiedandcoherentanalyticalandpersuasiveessays.5. Applytheprocessofwritingthroughdrafting,revising,andediting,attimesincollaboration

withothers.6. Employtextualconventionsforincorporatingideasfromsources.7. Achieveproficiencyinthetoolsofresearch,includingprintdocuments,scholarlydatabases,general

databases,andinformalInternetsources.8. Usecorrectgrammar,sentencestructure,andpunctuation.

Policies/Requirements

1.ToenrollinENG1102,studentsmusthaveearnedagradeof"C"oraboveinENG1101orequivalent

2.Writeacombinationofessays,annotatedbibliographies,and/orshortercompositionsofformalwritingthatisroughlyequivalenttotwentypages.

3.Writeafullydeveloped,fullydocumentedresearchpaper,proceedingfromroughdraftstofinalform.

4.Consultwithfacultyinthemajordepartmentsforrecommendedsources.

5.Studysampleessaysfordiscussionofideasandwritingmodels.

6.Scheduleatleasttwoconferenceswiththeinstructor.Conferencescanbeindividualorinsmallgroups.

7.Attendclassontime.Studentsareresponsibleforfindingout,ontheirown,thematerialforclassesmissed.


8.Avoidplagiarism.Plagiarism,thesubmittingofanother'sworkormaterialfromotherprintedsourcesasone'sownworkmayresultinthegradeof"F"forthepaperorforthecourseitself.

9.Taketherequireddepartmentalfinalexamattheprescribedtime.

10.Achieveskillsandcompleteallworkbytheendofthesemester.

11.InstructorsmaydevelopandenforceotherpoliciesconsistentwiththecurrentCSUcatalog,departmentalguidelinesandstandardacademicpractices.Veteraninstructorsmaypickanalternatesupplementaltextbook.

CriteriaforGrading:Thefinalgradewillbedeterminedbythelevelofcompetenceinthewritinganddiscussionrequiredinthecourse,masteryofproperdocumentation(s)asrequiredbymajorarea,andclassattendanceandparticipation.Eachinstructorwilldevelopacoursesectionsyllabusthatdiscussesspecificmethodsofdelivery,topics,activities,andassignments.Thefinalexamwillcountasatleast10%ofthefinalgrade.

ENG1102usestheuniversity’sA-FandIgradingscale.

AmericanswithDisabilitiesCompliance:CentralStateUniversityiscommittedtoincludingstudentswithdisabilitiesasfullparticipantsinitsprograms,services,andactivitiesthroughcompliancewithSection504oftheRehabilitationActof1973andtheAmericanswithDisabilitiesAct(ADA)of1990.IfyouareastudentwithadocumenteddisabilitypleasecallDr.WandaHadleyat937-376-6479todiscussyourcourseaccommodations.


Atthecompletionofthiscoursethestudentwilldemonstrateknowledge,skills,anddispositionsasfollows:

Knowledge(whatyoucanexplainand/orteachtoothers)

1.Understandthevariousmodesofwriting(exposition,description,andargumentation)usedinresearchwriting.

2.Explaintherelationshipbetweenthethesisandtherestoftheresearchpaper.

3.Describethetechniquesofargumentation.

4.Refuteadversarialpositions.

5.Describeproperdocumentationasrequiredbymajorarea.

6.Communicateawarenessofissues.

Skills(tasksyoucanperform):

1. Readandthinkcritically.2. Analyzeargumentativeissues.3. Relateinformationinessaystoone’sownacademicmajororareasofinterest.4. Applythetechniquesofargumentationtoresearch-basedwriting.5. Readandthinkcritically.6. Useapproveddocumentationformsinresearch-basedwriting.


7. Useprimaryandsecondarysourcesproperly.8. Writeunifiedandcoherentanalyticalandpersuasiveessays.9. Chooseandlimitaresearchtopic.10. FindinformationinthelibraryandontheInternet.11. Formulateathesisfortheresearchpaper.12. UseMLAorAPAformattinganddocumentationconventions.

Dispositions(attitudesandpatternsofbehavior)

1. Valuecarefulreadingandeffectivewrittencommunication.2. Valuecriticalawarenessofdifferingarguments.3. Maintaininterestincurrenttopics.4. Confidenceindiscussingissues.5. Evaluateandorganizeideas.6. Achieveproficiencyinthetoolsofresearch.7. Recognizerhetoricalmodesusedinresearchwriting.


Appendix C

First-Year Signature Assignment Prompt

Name:_______________________________________ Student ID Number:___________________________

English 1100/1101

You will have the rest of the class period to write a well-developed essay in response to the following directions. Be sure to plan your ideas carefully, include specific examples and details from the attached article in your paragraphs, and leave sufficient time for editing. Directions: Read “Style Showdown: $1,000 Sweater Faces $100 Rival,” then write an essay that addresses the following questions. You do not have to answer the questions in order, but you should discuss each question in your essay. The finished essay should be at least four to six paragraphs in length. 1. In your own words, describe two similarities and two differences between the two sweaters and then discuss what is significant about the similarities and/or differences between the two sweaters. 2. In your opinion, is there anything that the author could have explained in more detail or considered more carefully? Why? 3. How would you design and manufacture a sweater given what you have learned from the reading? Feel free to combine features of both or to suggest new ideas


Appendix D


Appendix E

report of critical thinking assessment, 2015-2016critical thinking scores writing intensive courses,...

Documents