annotating digital documents

Click here to load reader

Post on 23-Feb-2016




0 download

Embed Size (px)


Annotating Digital Documents. L3. Ink Annotations are different to text annotations. Ink stands out from the original It is free form – the annotator can Emphasize – underline, highlight, asterisk Question Agree Add side notes to explain, clarify - PowerPoint PPT Presentation


A Paperless Environment for Annotating and Marking Student Assignment

1Annotating Digital DocumentsL3


3Ink Annotations are different to text annotationsInk stands out from the originalIt is free form the annotator canEmphasize underline, highlight, asteriskQuestionAgreeAdd side notes to explain, clarifyThe authorship remains with the original author 34Why do we annotate?Active readingAdding annotations to a document helps the reader to process the informationCommunicate with others/selfWhen developing the document collaborativelyWhen evaluating the work (eg marking assignments)45Expressiveness and usefulnessAnnotations are very individualisticHowever they can generally be understood by others.

Interesting studies of students choosing old already annotated books and looking carefully at the style of annotationsAlso students understanding pre-annotated material more quickly than the original document56Visual message Looking at an annotated document in an instant you understand the position of the annotator. Are they:Being critical?Correcting the spelling and grammar?Adding their own explanatory notes67Example SoftwarePenmarkedAssignment markingRCACode review78Penmarked MotivationElectronic submission and marking of assignments is attractive to faculty and students alike Particularly programs assignments when a digital copy of the assignment is required for markingHowever providing meaningful feedback to students is more difficult And individualised feedback is important

89Design BriefAnnotation of the scriptScore recordingWork practice supportWho wants to waste time marking?910Implementation

1011Annotation PaneInk anywhereEraseMultiple files

A technical nightmare!Two layersLots of Windows API callsWe have more-or-less got pdf going

1112Scoring PaneMarking rubricNumeric scoresEntered in score box with pen and recognized using Microsoft recognition engine with numeric factoid feature also range checkedOr through the keyboard

1213Work Practices SupportPre-markingSet up of marking rubric, directories, file type/name filtersCollection of assignments from a dropbox or directory structureDuringAll assignments listed in student list paneCan be marked as opened and completeAutomatic unpackaging of zip filesDirect link to students folder for executing programOn completion Exporting of marks as xml fileConversion of assignments to pdf files (includes mark rubric and annotated files)Email return of assignment to student

1314Evaluation 1Iterative informal during developmentUsability testing with think aloudTalking while marking a program is challengingA few interaction problems pen/keyboard inputdo you want to save dialogueRecognition required correct formation of digitsUsing the pen to interact with non-tablet programs (the students assignments) is slow!Focus groupStudent list and workflow support invaluableAccurate writing required for score recognition 14Evaluation 2One large programming class ~ 180 studentsSix markersThree assignments increasing and size and complexityThree treatmentsPaperDatabasePenmarked Balanced treatmentEach student had one assignment marked by each processEach marker used each process Rotated so student marker different for each assignment

Marking TimeminutesNumber of AnnotationsTotal number of annotationsStudent Satisfaction process They did not like paper!Voluntary survey 42 respondents Student opinion aid to learningCorrelation between number of comments and students opinion of aid to learning.

(statistically significant)Overall PreferenceOverall preference for PenmarkedWhat else we could do with digital inkIDEs


22RCAOne of the difficulties with Penmarked is that the markers have to go to another application to compile and execute the program RCA is an annotation addin for Visual Studio2223RCA in Visual Studio

2324TechnicallyTransparent overlayAnother technical nightmare!Each annotation consists of:A linker line or circleAnnotation group text or diagramA severity indicator24CodeAnnotatorSame idea but trying to use Eclipse.Added some navigation and character recognitionPretty much the same problem!

No extensibility points from the code windows!25CodeAnnotator UI26

27Related WorkPaperless environmentsSellen and Harper (2002)Ink interactionPlimmer and Apperley (2003) Freeform - UI sketching environmentJarrett and Su (2003) AnnotationMarshall (1997)Shipman, Price et al (2003)Wolfe (2000)RecognitionTablet OSMarking softwareHeinrich and Lawn (2004)

27Browser Annotation Login and password new3 send bug reports


Browser AnnotationGoal an annotation toolbarFailed cant fire silverlight from the toolbarRevised goal copy web page and then annotate iFrameJava scriptSilverlightASP .Net MySQL

2930DiscussionWhy is it so difficult?There is a fundamental digital divide between text and imagesTrying to combine the two and keep them consistent is really difficultKnowing what to do with ink when the underlying document changes its problematicMany standard windows do not easily support non-text3031Future WorkVisual studio 2010Yeah it works!

Annotation in web browsers .

3132FinallyGoing paperless requiresProviding informal inking Reliable recognition (sometimes)and Attending to work practices which are more varied than we first imagine!32