how to write and review chi papers-chi2017 - the · pdf filehow to write and review chi papers...

HowtoWriteandReviewCHIPapersLENNARTNACKE,PHD,ASSOCIATEPROFESSOR

UNIVERSITYOFWATERLOO

IntroductionandGoalsUP-TO-DATEMATERIALS: HTTP://CHICOURSE.ACAGAMIC.COM/

SUBSCRIBETOCOURSEUPDATES: HTTP://EEPURL.COM/CNB9VD

AboutMe

GO TO WWW.HCIGAMES.COM FOR MORE INFORMATION

AssociateProfessorinGamificationandGamesUserResearch.

DirectoroftheHCIGamesGroupattheUniversityofWaterloo.

ManyyearsofexperienceservingonSIGCHIprogramandsteeringcommitteesandteachingUniversitygraduateclassesonHCIresearchmethods.

ReviewedmanypapersandhadlotsofmyownsubmissionsrejectedfromCHI

OurLearningGoalsToday

BecomeabetterwriterbylearninghowtoavoidunnecessarywordsandgiveeachsentenceastrongmeaningSimplify

UnderstandthegoalsofCHIresearchandwhatmakesastrongCHIcontributionUnderstand

Learnhowtostructureyourpaperaroundacompellingresearchnarrativetoemphasizeyourresearchproblemandsolutionasmaindrivers

Structure

WritebetterreviewsbyunderstandingwhatisimportanttohaveascontentforCHIresearchpapersReview

QuickIntroduction§ Whoareyouandwhatareyouworkingon?§ Whyareyouhere?§ Youhave15secondseach

Today’sSchedule9:30-10:50 WritingUnit

9:30-35 IntroandGoals

9:36-10:00 MicroLecture:ClarityandStructure

10:01-20 Exercise:StructuringCHIResearch

10:21-50 Exercise:WritingtheIntroduction

11:30-12:50 ReviewingUnit11:30-40 Recap

11:41-12:00 MicroLecture:OnReviewingforSIGCHI

12:01-20 Exercise:DissectingaCHIPaper

12:21-50 Exercise:WritingaHelpfulReview

OverviewofTodayUnit1:• MicroLecture:ClarityandStructure• Exercise:StructuringCHIResearch• Exercise:WritingtheIntroduction

Unit2:• Recap• MicroLecture:OnReviewingforSIGCHI• Exercise:DissectingaCHIPaper• Exercise:WritingaHelpfulReview

SurveyResponse

⟿ PICKAGOODPROBLEM⟿ KNOWTHELITERATURE⟿ STARTEARLY⟿ GETFEEDBACK⟿ DISCUSSEARLIERWORK⟿ BEBOLD

FrommyCHICourseSurveytoseniorCHImembers

ClarityandStructureUNIT1

Whatisthepurposeofyourwriting?Yourpurposeistocreatechangeinyourreader!

Tohelpthemunderstandyourresearchandact!WritingWithoutBullshit:BoostYourCareerbySayingWhatYouMean,JoshBernoff,HarperCollins,2016

WritetoConveyMeaning

𝑚𝑒𝑎𝑛𝑖𝑛𝑔𝑟𝑎𝑡𝑖𝑜 = ,-./0/123456789:6:.456789

WritingWithoutBullshit:BoostYourCareerbySayingWhatYouMean,JoshBernoff,HarperCollins,2016

Example Inovalonisaleadingtechnologycompanythatcombinesadvancedcloud-baseddataanalyticsanddata-driveninterventionplatformstoachievemeaningfulinsightandimpactinclinicalandqualityoutcomes,utilization,andfinancialperformanceacrossthehealthcarelandscape.Inovalon’suniqueachievementofvalueisdeliveredthroughtheeffectiveprogressionofTurningDataintoInsight,andInsightintoAction®.Largeproprietarydatasets,advancedintegrationtechnologies,sophisticatedpredictiveanalytics,data-driveninterventionplatforms,anddeepsubjectmatterexpertisedeliveraseamless,end-to-endcapabilitythatbringsthebenefitsofbigdataandlarge-scaleanalyticstothepointofcare.


Example(wordsthatarenotmeaningful)Inovalon isaleading technologycompanythatcombinesadvancedcloud-baseddataanalyticsanddata-driveninterventionplatformstoachievemeaningful insightandimpactinclinicalandqualityoutcomes,utilization,andfinancialperformanceacrossthehealthcarelandscape.Inovalon’s uniqueachievementofvalueisdeliveredthroughtheeffective progressionofTurningDataintoInsight,andInsightintoAction®.Largeproprietarydatasets,advancedintegrationtechnologies,sophisticatedpredictiveanalytics,data-driveninterventionplatforms, anddeepsubjectmatterexpertisedeliveraseamless,end-to-endcapability thatbringsthebenefitsofbigdata andlarge-scaleanalytics tothepointofcare.


Example(rewritefocusedonmeaning)Inovalon hasmoreinsightintohealthdatathananyoneelse.Weanalyzethatdataandapplytheknowledgetohelpyouimprovecareoptions,reducecosts,andenhancecompliance.Wehelphospitals,doctors,insurancepayers,andpatients.Weidentifygapsincare,quality,anddataintegrity,andweapplyouruniquecapabilitiestoresolvingthem.


DoaROAMAnalysisBeforeyouWrite

Readers••Whoistheaudience?

1

Objective••Howwillyouchangethereader?

2

Action••Whatdoyouwantthereadertodo?

3

Impression••Whatwillthereaderthinkofyou?

4


FourCoreResearchQuestionsWhatisthereal-worldproblem thatwearetryingtosolve?

Whyisitimportanttosolve thisproblem?

Whatisthesolution thatwecameupwithtosolveit?

Howdoweknowthatthesolutionisagoodsolutiontotheproblem?

http://chicourse.acagamic.com/chi-play-2016/interview-with-carl-gutwin

SurveyResponse

DON’TFOCUSONSTYLE,FOCUSONCORRECTNESS,SCIENTIFICVALIDITYANDONACONTRIBUTIONTHATYOUTHINKCHANGESORPROGRESSESTHEFIELDSIGNIFICANTLY.


HowtoStructureResearch1. Definearesearchquestion

2. Testyourquestionsagainsttheliterature

3. Refineyourresearchquestion

4. Deriveamethodthatanswersyourquestion

5. Planoutadetailedanalysis

6. Refineyourmethodandanalysis

seealsohttps://faculty.washington.edu/ajko/advice#goodpaper

7. Executeyourmethod

8. Executeyouranalysis(thisisyourResultsdraft)

9. Writeyourdiscussion

10. Polish!Polish!Polish!

11. Submitandwait

12. Doitalloveragain!

TheHourglassStructure1. ABSTRACT

2. Introduction

3. LiteratureReview

4. Methodology

5. Results

6. Discussion

7. Conclusion

8. Acknowledgements

9. References

Jofish’s ResearchOnPaperTitlesConferences◦ Articleswithquestionmarks(?)intitlesarelesscited◦ Colons(:)increasecitations

Journals◦ Maximizecitationswithnocolons,noquestionmarks

Transactions◦ Morequestionmarksintitleshavemorecitations

seehttp://sigchi.tumblr.com/post/104956615720/what-should-you-title-your-paper

OnceWritten,EditforClarityDoIusesimple,familiarwordsthroughout?

HaveIeliminatedalljargonandclichés,hedging?

HaveIavoidedrepeatingwords,points?

HaveIgoneintotoomuchdetailanywhere?

HaveIavoidedambiguity?

DoIavoidmakingsimplepointssoundcomplicated?

Areallmydistinctions/comparisonscrisp?

CanIcutoutanymorewords?

Areanysentencestoolong,orcontaintoomanysub-clauses?DoIusetheactivevoice?DoIavoidreversingintosentences?Aremytensesconsistent?DoIavoidwordechoes?Doesmywritingreflectclear,logicalthinking?Willareaderbeforcedtorereadanything?HaveIremovedalltheunnecessaryclutter?HaveIgottothepointimmediately?DoIhavealogicalnarrativestructure?

MoreStyleAdvice

ReduceWords

Reducewordsthatdonotcontributetothemeaningofasentence••Beclearandbebriefinwhatyousay••Deleteadverbswheneverpossible

ReduceJargon Reducejargonandcomplexwords

FindRhythm Findarhythmtoyourwriting

AvoidPassive Minimizepassivevoice

NineQuestionstoaskYourself1. Ifyoucouldaddressjustoneproblemin10years,whatwoulditbe?

2. Areyouusingyouruniquesituationandresourcestothefullest?

3. What’syourHCIresearchgenre?

4. Isyourapproachrightforyourresearchtopic?

5. Whyisyourresearchinteresting?

6. Canyoufailintryingtoanswertheresearchproblem?

7. Willyourworkopennewpossibilitiesofresearch?

8. Whydoyoubuild/prototype?

9. Inonesentence,whatisthecontributionofyourresearch?

▸ Let’sdothisexercise now!

SusanneBødker,KasperHornbæk,AnttiOulasvirta,andStuartReeves.2016.NinequestionsforHCIresearchersinthemaking.interactions23,4(June2016),58-61.DOI:10.1145/2949686

SurveyResponse

CONVINCEYOURACANDYOUHAVEACHANCE.REVIEWERSDON’TMATTERINTHEGRANDSCHEMEOFTHINGS.IFYOURPAPERISEVENCLOSETOGETTINGININNOVEMBER,ASTRONGREBUTTALANDANADVOCATEONTHEACISALLYOUNEED.


Exercise1:StructuringYourCHIResearchBuildabriefresearchplanforaCHIpublication(10minutes)◦Problemstatement◦ Indicationofyourmethodology◦Anticipatedmainfindings◦Anticipatedconclusions

Presentyourplanstothegroupwithabriefdiscussion(10minutes)ofstructuralflawsorstrengths– wewillalltrytocritiquetheplans

Exercise2:WritingtheIntroductionNow,writeyourownTitle,Abstract,andIntroduction fortheresearchplanyouhavedeveloped

Usethefourquestionstoguideyouthroughtheprocessofwritingafictional CHIpaperaboutthisresearchtopicthatyouhaveinmind:◦ Whatisthereal-worldproblemthatwearetryingtosolve?◦ Whyisitimportanttosolvethisproblem?◦ Whatisthesolutionthatwecameupwithtosolveit?◦ Howdoweknowthatthesolutionisagoodsolutiontotheproblem?

UsethesameprocessasmanyCHIauthors:◦ Sketchtheroughanswerstoeachquestionintobulletpoints◦ Gettogetheramaximumof15bulletpointsamongall4questions◦ Startwritingoutthebulletpointsintoparagraphs

Passaroundyourwrittenparagraphsanddiscussthemingroups,Iwillassist

ReviewingforSIGCHIUNIT2

CheckYourAttitudeAreyouaPaperChampion,ordoyouseekoutflaws?AreyouaPaperKiller,arovingmercenarycapableofdispatchingasubmissionwithafewchoicewords?

It’seasytopointoutflaws,butittakesskilltohelpthereviewersimprove theirpapers.“Reviews—especiallyhastilywrittenandoverly-criticalhatchetjobs—oftendofarmoreharmthangood.”

http://mobilehci.acm.org/2015/download/ExcellenceInReviewsforHCICommunity.pdf

AMantraforPapersontheEdgeWhenindoubt,trusttheliteraturetosortitout.

Forpapersneartheleadingedgeofinnovationnoteveryonewillagreetheyhavemerit.

Apaper’srealvaluecanbetosparkdiscussionandfurtherinquiryintoanewresearchdirection.

WhatMakesAGoodReview?

Agoodreviewreflectsonthecontributionsofthework

1Discussestheweaknessesandlimitationsinapositivemanner

2Clearlycallsoutthestrengthsandutilityofthework

3

AGoodReviewConsiderhowtheauthor’s◦ arguments,◦ results,and◦ demonstrationsfitintocloselyrelatedwork

Canyouraisewholenewperspectivesandanglesofcontributionthatmightbesuggestedbythework?

Canyouproposeconnectionstoareasoftheliteraturethattheauthormightnothavethoughtoforevenbeenaware?Thisisanopportunity!

Asksmartandtoughquestionstheauthorscanaddressinarevision!

The2ACReview1AC=takeschiefresponsibilityforassessingapaperfortheprogramcommittee

2AC=adoptsasupportingbutstillcriticalprogramcommitteereviewingrole

Bothare(often)responsibleforassigningexternalreviewerstopapers.

2ACreviewsarelikeexternalreviewwithextrathoughtfulandconsideratereviewoftheresearch.

Ifyoufeeluncertainaboutexpertiseorsubjectmatter,letthecommitteeknow.

Consideradjustingyourreviewinthefinalround.

TheMetaReview(1AC)Includeyourownperspectivesandacommentaryonthepaper.

Yourmustsumuptheperspectivesofallofyourreviewers(andyoushouldconsidertheaverageoftheirscoreswhenbuildingyourownscore).

Takeintoaccountthemostimportantpointsraisedthatyouwouldliketoseeaddressed.

BALANCEandFAIRNESS!Agoodmeta-reviewwillquoteorparaphrasekeycommentsfromeachreviewer.

Meta-reviewisimportanttodirecttheauthorstomostessentialcommentsforweighingthemeritsforacceptanceandtowhichtheauthorsshoulddevotethemostattentioninfuturerevisions.

FinalWordAboutReviewsTowritegoodpapers,workonlotsofreviewstogettoknowyourfield.

Youwilllearnnewtricksaboutpaperorganization,layout,researchapproaches.

Askyourself:◦ Didyouunderstandtheauthor’sgoals,results,anddiscussionscorrectly?◦ Isitpossiblethatcommunitieswithinterestsandconcernsdifferentfromyourownwouldfindtheworkinformative,eveninspiring?

◦ Doesthepapertouchonanareayoudon’tknowverywell?

Becritical,butrespectfultotheauthors

Exercise1:DissectingaCHIPaperStructureddiscussionofthefollowingpaper:◦ https://dl.acm.org/citation.cfm?id=2858558◦ MatthewKay,TaraKola,JessicaR.Hullman,andSeanA.Munson.2016.When(ish)isMyBus?:User-centeredVisualizationsofUncertaintyinEveryday,MobilePredictiveSystems.In Proceedingsofthe2016CHIConferenceonHumanFactorsinComputingSystems (CHI'16).ACM,NewYork,NY,USA,5092-5103.DOI:https://doi.org/10.1145/2858036.2858558

Readthepaper.

Whichpartsofthepaperareexcellent?Whydoyouthinktheyare?

Exercise2:WritingaHelpfulReviewReadandannotateoneofthesepapers:◦ https://dl.acm.org/citation.cfm?id=2858036.2858594◦ https://dl.acm.org/citation.cfm?id=2858361◦ https://dl.acm.org/citation.cfm?id=2858305

Writeareviewwithafocuson:◦ Reflectingonthecontributions◦ Discussingtheweaknessesandlimitationsinapositiveway◦ Callingoutthestrengthsandutilityofthework

Discuss

how to write and review chi papers-chi2017 - the · pdf filehow to write and review chi papers...

Documents