towards automated quality assurance for socially created information assets on the web david...
Post on 20-Jan-2016
216 views
TRANSCRIPT
Towards Automated Quality Towards Automated Quality Assurance for Socially Created Assurance for Socially Created Information Assets on the WebInformation Assets on the Web
David Davenport & Markus SchaalComputer Engineering Dept.,
Bilkent University, 06800 Ankara.
IntroductionIntroduction
• Quality in collaborative web-based projects • considered good • no guarantees!
Outline:• Quality & Quality Assurance• Info. Quality & User Expertise• From Actions to Assessments• A Research Testbed…
Outline:• Quality & Quality Assurance• Info. Quality & User Expertise• From Actions to Assessments• A Research Testbed…
Quality isQuality is
• something good, desirable!
• hard to definemeet or exceed user expectations
• easy to spot lack offails to meet spec.,poorly designed,difficult to use, unreliable, etc.
Quality isQuality is
an open-ended quest for
perfection
Quality AssuranceQuality Assurance
• Assuring improvements requires a process
Vision/goal
System/product/service
Make measurements & compare
Implement changes
What is Information Quality?What is Information Quality?
• Criteria: accessibility, coverage, correctness, consistency, recency, reliability, cost, etc.
• Information must “inform” / “educate”• success depends on recipient too!
• Measuring information quality• either observe user behaviour (individual & groups)• or ask their opinion… but users are not all equal.
Users are not all equalUsers are not all equal
experts
novices
& subversives!
Modulate user
opinion accordingly
But, on the web, how can you know your users?
From Actions to AssessmentsFrom Actions to Assessments
System model of user
User
users
Systemmodel
ofcontent
contents
updates
modulates
viewupdate
rate
Action
assessmentsof quality
Content & User AssessmentsContent & User Assessments
• Update content & user models using evidence• gathered from implicit observations
& by explicitly asking users, but• modulated by current assessment of user expertise/trust!
• Everything depends on users, encourage by• making it easy to contribute and rate content• providing incentives to do so, &• showing them the results!
• What evidence should be gathered & how?• What content & user models should be used?• How should the models be updated?• How should the results be reported?
Research questions
A Research Testbed…A Research Testbed…
• Target students taking CS101/2 “Java” course• Lots of Java / computing resources on the web
(some old, some too complex, some plain wrong!)• Task is to create a quality wiki-like resource.
• Module – for Moodle, Drupal, MediaWiki• Enable distributed users to collaboratively
build quality information assets on the web.
• Of note…• Log all interactions• Try out various user, content & update models• Show overall system quality & its history
To Conclude…To Conclude…
• Web is increasingly important info. resource• Improvements in Quality benefit all• Need way to evaluate quality,
identify opportunities for improvements, & track changes
• Our research aims to • produce a clear understanding
& common “quality” framework• add-on “quality” modules usable by others.
Thank you Thank you for your time & attentionfor your time & attention