Teaching sustainable online research practices across the curriculum: The Q6C Solution
Sarah Read (English)Kate Deibel (CS / Education)
Tim Wright (History)University of Washington
Computers & Writing 2009: Ubiquitous and Sustainable Computing 2
Computers & Writing 2009
Goals: Transcend traditional boundaries
Academic disciplines K-12 and higher education Online and offline Organizers, attendees, and presenters School, work, and play
Develop a sustainable perspective on lifelong computing and communication
Computers & Writing 2009: Ubiquitous and Sustainable Computing 3
Today’s Plan
Framing Q6C Within Conference Intent A Taste of Application What is Q6C?
Development Overview
Using Q6C For the Teacher For the Students (and Teacher)
Conclusion Discussion and workshop
Computers & Writing 2009: Ubiquitous and Sustainable Computing 5
The Research Scenario
You are in a college geology course and have been assigned to brief the class on the Crandall Canyon Mine disaster. Answer the question:
Was the cause of the mine collapse geological, and what was it?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 7
Questions raised by Q6C
How do researchers know when to stop looking for sources?
What do experienced researchers in a field know about when to stop that newcomers don’t know?
How can newcomers to a field be supported to do research with the savvy of experienced researchers?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 9
Symptoms of a Serious Problem
In regards to online sources, students... …stop too soon when evaluating a source …fail to keep a skeptical frame of mind …tend to focus on surface/superficial details …trust the top search results …reject Wikipedia outright …accept Wikipedia outright …fail to transfer critical skills across domains
Computers & Writing 2009: Ubiquitous and Sustainable Computing 10
An Information Literacy Skill The Goal:
Teach students to be savvy, critical consumers of Internet sources for both academic and non-academic purposes
Challenges for Teaching What skills should be taught? How can those skills best be conveyed? What is needed to promote skill transfer
across disciplines / outside of the classroom?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 11
Today’s Plan
Framing Q6C Within Conference Intent A Taste of Application What is Q6C?
Development Overview
Using Q6C For the Teacher For the Students (and Teacher)
Conclusion Discussion and workshop
Computers & Writing 2009: Ubiquitous and Sustainable Computing 12
Previous Work and Research
Different rules and expectations for student writing exist across disciplines
Internet poses challenges and opportunities for student research
Students report IT confidence but overestimate source judging abilities
Students report difficulties and overload in conducting research
Checklists for evaluating reliability of Internet sources
McCarthy (1987)
Sorapure et al. (1998)
Kvavik (2005)
Head & Eisenberg (2009)
Many authors
Computers & Writing 2009: Ubiquitous and Sustainable Computing 13
The ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is
knowledgeable and credible? If the author's qualifications aren't listed…
Sponsorship What does the URL tell you? The URL
ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org), …
Currency How current is the site? How current are the site's links? If many
of the links no longer work, the site may be too dated for your purposes.
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Computers & Writing 2009: Ubiquitous and Sustainable Computing 14
Pitfalls of the ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is
knowledgeable and credible? If the author's qualifications aren't listed…
Sponsorship What does the URL tell you? The URL
ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org)…
Currency How current is the site? How current are the site's links? If many
of the links no longer work, the site may be too dated for your purposes.
Excerpt from Hacker’s A Pocket Manual of Style (2008)
INACCURATE: .org has never been restricted to only nonprofits
Computers & Writing 2009: Ubiquitous and Sustainable Computing 15
Pitfalls of the ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is
knowledgeable and credible? If the author's qualifications aren't listed…
Sponsorship What does the URL tell you? The URL
ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org)…
Currency How current is the site? How current are the site's links? If many
of the links no longer work, the site may be too dated for your purposes.
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Not all domains are regulated
Domains reflect only general purposes and not specific pages
Computers & Writing 2009: Ubiquitous and Sustainable Computing 16
Pitfalls of the ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is
knowledgeable and credible? If the author's qualifications aren't listed…
Sponsorship What does the URL tell you? The URL
ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org)…
Currency How current is the site? How current are the site's links? If many
of the links no longer work, the site may be too dated for your purposes.
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Ignores complexity of web authorship
Encourages the usage of titles, degrees, and symbols of authority to determine credibility
Computers & Writing 2009: Ubiquitous and Sustainable Computing 17
Pitfalls of the ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is
knowledgeable and credible? If the author's qualifications aren't listed…
Sponsorship What does the URL tell you? The URL
ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org)…
Currency How current is the site? How current are the site's links? If many
of the links no longer work, the site may be too dated for your purposes.
Excerpt from Hacker’s A Pocket Manual of Style (2008)
Suggests recent data as being more reliable
Update frequency will vary by the type of site
Computers & Writing 2009: Ubiquitous and Sustainable Computing 18
Criticisms of the Checklist
Inherent problems Emphasis on surface features over content Simplistic yes/no questions with no guidance Erroneous indicators of credibility
Students fail to develop information literacy skills and critical practices Meola (2004) Helms-Park & Stapleton (2006)
Need for better evaluative methods to develop sustained, transferable skills Sidler (2002)
Computers & Writing 2009: Ubiquitous and Sustainable Computing 19
Developing Q6C
Intent: Develop tools to support instructors in teaching web
literacy skills
Guiding Principles: Emphasize the process of source evaluation, not the
end product Recognize disciplinary differences Promote student metacognition and transfer of skills
across and outside academia
Approach: Discuss source evaluation across the curriculum Develop a working model of the evaluation process
Computers & Writing 2009: Ubiquitous and Sustainable Computing 20
Discussions about Source Evaluation
Research Literature Ayers (2006) Britt & Aglinskas (2002) Head & Eisenberg (2009) Helms-Park & Stapleton (2006) Hunt & Hunt (2006) Kvavik (2005) Lorenzo & Dziuban (2006) Meola (2004) Oblinger & Oblinger (2005) Sidler (2002) Sorapure et al. (1998) Sullivan and Porter (1997) Thompson (2003) Wineburg (1991,1991,1999)
Source Evaluation Tools Barker & Kupersmith (2009) * Beck (2009) * Cohen & Jacobson (2009) * CARS / CAFÉ – Harris (2007) * Big6 – Eisenberg & Berkowitz
(2001) Hacker (2008) * TAP – Johnson & Lamb (2007) * CSU How to… – Lederer * Rampolla (2007) * Smith (1997) *
* Checklist variant
Cross-disciplinary discussions: Sarah, Kate and Tim
Computers & Writing 2009: Ubiquitous and Sustainable Computing 21
Q6C: Modeling Source Evaluation
Question
Categorize
Critique Rhetorically
Characterize Authorship
Contextualize
Corroborate
Conclude
Computers & Writing 2009: Ubiquitous and Sustainable Computing 22
Q6C: The Start
Question
Categorize
Critique Rhetorically
Characterize Authorship
Contextualize
Corroborate
Conclude
Maintain a skeptical frame of mind
Ask questions relevant to your research, purpose, and discipline
Computers & Writing 2009: Ubiquitous and Sustainable Computing 23
Q6C: The End
Question
Categorize
Critique Rhetorically
Characterize Authorship
Contextualize
Corroborate
Conclude
Is the source credible?
Is the source useful for your research goals?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 24
Q6C: The Middle
Question
Categorize
Critique Rhetorically
Characterize Authorship
Contextualize
Corroborate
Conclude
Different dialogues to engage in with a potential source
Features and relative importance shaped by research task and discipline
Not necessary to do all components nor in any order
Computers & Writing 2009: Ubiquitous and Sustainable Computing 25
Question
Q6C: The Cloud
Categorize
Contextualize
Corroborate
Conclude
Characterize
Authorship
Critique Rhetorically
Repeat as necessary
Computers & Writing 2009: Ubiquitous and Sustainable Computing 26
Simple Instantiation of Q6C
Categorize
Series of posts
on Blogger
Characterize Authorship
Most recent post was 2001
Conclude
Too old to be useful
Question
Topic: Information / opinion on recent U.S. policy towards net neutrality
Computers & Writing 2009: Ubiquitous and Sustainable Computing 27
More Complex Instantiation of Q6C
Question
Conclude
Likely to be useful
CharacterizeAuthorship
He is a Swiss citizen
Categorize
Many posts on
ScienceBlogs
Conclude
Useful but biases exist
Corroborate
Provides links to citations
CharacterizeAuthorship
Part owner of a European
ISP
Topic: Information / opinion on recent U.S. policy towards net neutrality
CharacterizeAuthorship
Relevant posts in 2008-
09
Computers & Writing 2009: Ubiquitous and Sustainable Computing 28
Summary of Q6C
Working model of source evaluation process Definite start and end points Iterative and integrative middle
Emphasizes research as a process Only concerns one element of all of research
Not specific to particular disciplines Domain knowledge, heuristics, and criteria need to
be supplied by instructors Provides a common language for discussion
Dirty little secret… not limited to just online sourcesBut how do you use it in teaching?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 29
Classroom Applications of Q6C
Two approaches for using Q6C:1. Using Q6C to inform teaching practice and
assignment and lesson design (Sarah)
2. Infecting students to the practice and habit of source evaluation via the Q6C model (Tim)
Why is this not a best source?
Authoritative Source
Scientific SourcePartial
Answer
Wrong genre? But current: 07-08
Aha! The most credible and the most useful source!
Peer Reviewed
Directly relevant to research question
How to get a student (or a newcomer) to this “best” source?
Finally, a complete answer relevant to the research scenario
Visualizing the Q6C Process Question: Was the cause of the mine collapse geological, and what was it? Scenario: College-level geology course
1st hit: MSHA web page
Characterize Authorship
Government agency relevant to issue (.gov)
Contextualize
Not a scientific source
Conclude
Part of the story, but not citable
Corroborate
BUT: What sources does it cite?2nd hit: Berkeley
labAnnual report
CharacterizeAuthorship
.edu / Scientific lab
Categorize
Genre: annual report (not peer reviewed)
Purpose: tangential to research question
Conclude
Best source: relevant and credible
3rd hit: UT seismology
scientific report
Contextualize
Q6C as a heuristic for lesson/assignment design
1. Identify a research scenario relevant to course context
2. Identify the tacit research practices of an experienced researcher in that knowledge domain: think Q6C.
3. Construct a scaffolded lesson or assignment to teach explicitly the domain knowledge necessary for critical source evaluation.
4. Plan for reflective writing or discussion to promote metacognition about research practices.
OUTCOMES: 1. students locate more useful and credible sources2. begin to learn how to approach any new research scenario.
Other choices when planning a Q6C lesson or assignment
Prepare an archive to anticipate the research process or set students loose?
How are the research process and domain knowledge weighted in assignment evaluation?
Explicitly teach the meaning of Q6C terms, or prompt students to invent their own during a reflective process?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 39
Questions raised by Q6C
How do researchers know when to stop looking for sources?
What do experienced researchers in a field know about when to stop that newcomers don’t know?
How can newcomers to a field be supported to do research with the savvy of experienced researchers?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 50
Q6C Lessons Learned
Students already do low-level evaluation Web checklists don’t help much Thinking contextually with corroboration
and purpose in mind does Students can benefit from using Q6C
explicitly
Computers & Writing 2009: Ubiquitous and Sustainable Computing 51
Q6C for Students: Next Steps Design and re-design assignments with
Q6C Use Q6C in course design,
implicitly/explicitly Monitor and analyze results Tweak Repeat Infect others with the Q6C madness
Computers & Writing 2009: Ubiquitous and Sustainable Computing 52
Ongoing Work
Continued classroom applications Modern American Civilization From 1877 (Tim) The rhetoric of Writing in the Workplace (Sarah)
Further research and development Refinement of the Q6C working model Connect to literature on cognitive apprenticeship
and metacognition (e.g., Scardamalia & Bereiter (1983))
Dissemination and partnership building Are you interested in using Q6C in your teaching?
Talk to us.
Computers & Writing 2009: Ubiquitous and Sustainable Computing 53
Thank You!
We wish to acknowledge the following: Center for Instructional Development and
Research Practical Pedagogy Our students Suzzallo Espresso The Internet You
QUESTIONS?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 54
Extra Slides
Descriptions of each Q6C Component Suggestions for Teaching With Q6C Suggestions for Students Using Q6C
Computers & Writing 2009: Ubiquitous and Sustainable Computing 55
Q6C: Question
Maintain a skeptical frame of mind Ask questions relevant to your research
Computers & Writing 2009: Ubiquitous and Sustainable Computing 56
Q6C: Categorize
In the context of your research, is this a primary, secondary, or tertiary source?
What type of site is it (website, blog, wiki, database, etc.)?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 57
Q6C: Critique Rhetorically
What do the authors’ choice of words, tone, font, display format, images, genre, and argumentative strategies tell you about the intended audience and the credibility and reliability of this site? (‘Read’ the site.)
Computers & Writing 2009: Ubiquitous and Sustainable Computing 58
Q6C: Characterize Authorship Identify who created the content, when
they created it, and for what purpose. Single or multiple authors? Committee?
Institution? Critic? Expert? Unknown? Other?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 59
Q6C: Contextualize
Place the information collected in conversation with your existing experience and body of knowledge.
Does it fit? How?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 60
Q6C: Corroborate
Assess how the content compares to other sources.
Is the content consistent, complementary, or contradictory?
Computers & Writing 2009: Ubiquitous and Sustainable Computing 61
Q6C: Conclude
Is the source credible? Is the source useful for your research
goals? If not, find a new source, repeat Q6C.
Computers & Writing 2009: Ubiquitous and Sustainable Computing 62
Suggestions for Teaching With Q6C
Remember that subject-area experts automatically perform the Q6C process, whereas novices need to consciously perform each step when learning how to assess a source's credibility and usefulness.
Identify which components of Q6C you want to emphasize and scaffold the assignment’s research process so that students learn new skills incrementally.
Teach that research is about a process, not about a product.
Construct assignments that engage your students in authentic research practices for your target discipline.
Computers & Writing 2009: Ubiquitous and Sustainable Computing 63
Suggestions for Teaching With Q6C
Tips for designing assignments and activities:
Make explicit the outcomes of the assignment and encourage reflection to help move students to the meta-cognitive level.
Consider choosing research topics that you are not an expert in so that you can share the discovery process with the students.
Perform the assignment yourself or with a colleague prior to class in order to anticipate student responses.
Computers & Writing 2009: Ubiquitous and Sustainable Computing 64
Suggestions for Teaching With Q6C
Tips for designing assignments and activities:
Make explicit the outcomes of the assignment and encourage reflection to help move students to the meta-cognitive level.
Consider choosing research topics that you are not an expert in so that you can share the discovery process with the students.
Perform the assignment yourself or with a colleague prior to class in order to anticipate student responses.
Computers & Writing 2009: Ubiquitous and Sustainable Computing 65
Suggestions for Students Using Q6C
Q6C is a heuristic and not a checklist. You do not need to answer every question for every source. Some sources will require only asking one component of Q6C; other sources will require repeated application of Q6C.
Remember that there are two questions you should ask for every source: Is it credible? Is it useful? Answers to these questions are not absolute: a source can still be useful but still of dubious reliability.
Computers & Writing 2009: Ubiquitous and Sustainable Computing 66
Suggestions for Students Using Q6C
When you think you have found a credible source, perform one last check by applying Q6C a little further. A review of Characterize Authorship, Contextualize, and Corroborate can reveal important, overlooked details.
Practice using Q6C in other classes and daily readings. As you gain experience, you will begin to automatically use Q6C to evaluate all kinds of information.
Remember: Research is a process, not a product.