using location-aware multimodal speech interfaces to survive a zombie apocalypse

8/3/2019 Using Location- Aware Multimodal Speech Interfaces to Survive a Zombie Apocalypse

1/12

UsingLocation-AwareMultimodalSpeechInterfacesto

SurviveaZombieApocalypse

[email protected]

1.Introduction

Intheeventofalarge-scaleapocalypticassaultonhumancivilizationbyrecently

deceasedreanimatedcorpses,orzombies,apreexistinginfrastructureallowing

thereal-timemonitoringofassailantsandguidanceofarmedpersonnelwill

greatlyimprovethelikelihoodofmarginalizedhumanssurviving.Iproposethe

implementationofasystemimprovingtheconventionalEU112emergency

telephonenumberservicewithamultimodalinteractivevoiceresponse(IVR)

systemcapableofbeingdeployedattheonsetofazombieapocalypse.The

systemsinputandoutputmodalitiesincludespeechrecognition,speech

synthesis,largevisualdisplays,andglobalpositioninginformationtransmitted

viacommoncellulartelephones.

Currentemergencytelephoneservicesconnectcallerswithresponsecentersstaffedwithliveoperatorswhoreceiveinput,assesssituations,andrecommend

actionsbasedontheirtraining.Theresultisoftenthedispatchingofemergency

personneltothecallerslocationortheprovidingofinstructionsforthecallerto

follow.IntegratedGPSfunctionalityinmoderncellularphonesallowsthe

operatortodeterminethecallerslocationwithreasonableaccuracy.Non-GPS

capablephonescanbepinpointedwithalesserdegreeofaccuracyusingantenna

towertriangulation.

Shouldtheundeadrisefromtheirgravesandactivelyseekconsumptionof

humanflesh(and/orbrains),traditionalemergencycallcenterswillbeunableto

handlethesuddeninfluxofincomingrequestsforassistance.Increasingthe

numberofstafftoaccommodatetheescalateddemandwillprovedifficultdueto

decreasedpersonnelreliabilityasaresultofdeathorexodus,anddangers

imposedbyhavinglargenumbersofhumansinafixedenvironment;namely,the

attractionofzombiestobothnoiseandconcentrationsofhumanflesh.Azombie


2/12

outbreakwithinvicinityofacallcenterwouldforceevacuation,causingservice

outages.

AdecentralizedandredundantIPtelephonynetworkwouldallowfavorable

uptimeinsituationswhereitwouldbedangerousforhumanstoremain

stationaryforextendedperiodsoftime.Thelocationofsafehousesandgun

cachescanbeprovidedtocallers.Informationreceivedfromcallersaboutthe

quantityofzombiesinhisorherpresenceisaddedtoadatabasealongsideGPS

coordinatestakenfromtheusersdevice.Largepublicdisplaysstrategically

locatedinsafehousesandcentralsquaresshowmapswiththelocationsof

recentlyspottedzombiestohelpaffectedcitizensavoiddangerousareasand

trackmovementofzombiehordes.

2.SpeechRecognitionTechnologyRequirements

Speechrecognitionisanimperfectprocessanditisimportanttoaddressspecific

propertiesofthesystemsrecognizerbeforedesignanddevelopmentprocesses

begin.

Attribute Requiredvalue Rationale

Vocabularyandlanguage

Vocabularysize Small

Asmalldictionaryofimportant

actionwordsarerequiredbythe

speechrecognizer.

Grammar Phrase-based

Systemrecognizespredefined

keywords.Grammarandsysteminteractionmodelconsiderspossible

situationsanduserbase.

Extensibility Changeable

Safetyofcitizensrequiresthesystem

tosupportsomedegreeofextensibility.Additionalvocabulary

canbeaddedshouldnewfeaturesbe

desired.


3/12

Communicationstyle

Speaker Independent

Thisisapublicapplication,so

speakerandgenderindependentmodelsmustbeapplied.

Speakingstyle Continuous Word-spottingisemphasized.

Overlap Barge-in

Theabilitytointerruptsystemoutputsandoffernewcommandsis

importantinlife-threatening

situations.Thiswillalsoallowlongerandmoreinformativeoutputstobe

offered.

Usageconditions

Environment Hostile

Withthemajorityofphonecalls

takingplaceinhecticsituations,thesystemmustbepreparedtohandle

hostileenvironments.

Channelquality Low-qualitySystemoperatesovercellularphone

networksandaparticularusersreceptionmaybepoor.

2.DraftoftheSystemsSpeechInterface

Whenazombieorzombiehordeisspotted,citizensdialathree-digitemergency

numberontheirmobilephoneandusevoiceinputtoinformavirtualagentofthesituation.Speechinputisrecognizedandspeechoutputisprovidedinthe

formofdirectionstosafehousesorguncaches.Whilesupportedbythe

applicationalongsidespeechinput,traditionaltouchpadinputisimpractical

becausecallerswilloftenbefleeingonfoot,unabletodevotevisualattentiontoa

handhelddevice.Whilemodernsmartphonesarecapableofprovidingnew

interactionexperiences,includingtactileandhapticinterfaces,thesensitivityof

thisapplicationnecessitatesbroadcompatibilityacrossoldergenerationsof

mobilephoneswithalowworderrorrate(WER)thresholdoflessthan5%.


4/12

Thesystemsupportsbothsystem-initiativeandmixed-initiativedialogue

strategies.Duetothesafety-criticaldomainoftheapplication,mostuserswillbe

trainedhowtousethesystembeforecallingforthefirsttime.Expertuserswill

preferthemixed-initiativeapproach,inputtingknowncommandswithoutwaitingforguidance,whilenoviceandregularuserswillrequirethesystemto

takeinitiativeandguidetheconversation.

BasiccommandsincludeSAFEHOUSE,whenthelocationofthenearestsafe

houseisdesired,GUN,whenthelocationofthenearestguncacheissought

and,HELPwhenimmediatesupportisrequestedintheformofmedicalor

defensiveassistance.Sinceallparamedicsandsafetypersonnelwillbeheavily

armed,thereisnoneedtospecifybetweenrequestsformedicalorfirepower

support.

Word-spottingisanimportanttraitofthespeechrecognizerbutsupplementary

speechinputs;namely,screamsofvariousvolumes,lengthsandtones,shouldbe

abletobespottedalongsidestandardwords.Itispossiblethatcallerswillbe

communicatingoverlow-qualitychannelsinhostileenvironments.Contributing

totheless-than-idealsituationofthecallisthelikelihoodofauserbeing

assaultedbyazombieduringtheapplicationdialogue.Thesystemshouldbe

abletorecognizewhenacallerisscreaminginpainorfear,andimmediately

dispatchassistance.

Ifthesameemergencynumberserviceisadaptedtosupportadditional

situations,suchastraditionalfire,police,andambulancescenarios,itis

importantthesystemconfirmsthemotivationofthecall.Ifacallerengagesthe

mixed-initiativeapproachwithoutusinganyoftheabovekeywords,the

keywordZOMBIEcanbespokenatthebeginningofthecalltoindicatethe

contextofthesituation.Thisapplicationdescriptionassumesitisadedicated

zombie-onlyhotline.

Attheendofeachtransaction,callerswhoarenotintime-sensitivesituations

areaskedforspecificdetailsoftheirsituation,includingthenumberofzombies

presentattheirlocation.Thisinformationiscombinedwiththeuserslocation

coordinatestoupdatesmapsonlargepublicdisplaysincentralareasoftown.


5/12

Belowaresomeexampledialoguesofthesysteminaction.

Party Speechin/out Explanation

System: Whatisyouremergency? Callisconnectedanddialoguebegins.

Caller:THEREAREZOMBIES

EVERYWHERE!

Panickingnoviceuserengagesthe

systemsinitiativeandanswersthe

question.

System:Areyouabletosafelyflee

thesituation?

Systempreparestosuggestassistance

byclarifyingthesituation.

Caller: NO. Callerrespondstoquestion.

System:Iamdispatchingassistancetoyourlocation.

Systemacknowledgesthecallerssituationandforwardshisorher

locationcoordinatesandarecordingoftheconversationtoapplicable

personnel.

DialogueExample1



Caller: Ineedasafehouse!Callerignoresthesystemsinitiative

andstatesaknownkeyword

System:

NearestsafehouseisatKoskikatu9.Wouldyou

likedirections?

Systemrecognizessafehousekeywordandprovidesnearest

location.Takesinitiativebyoffering

directions.

Caller: Yes. Callerrespondstoquestion.

System:

TurnrightatRongankatu.

Say"next"whenyouarereadyforthenext

direction.

SystemusesthephonesGPS

functionalitytolocatethecaller.Firstdirectionisprovidedwithapromptfor

thenext.

Caller: Iamthere.Next.Callerfollowsdirectionanduses

providedkeywordtoindicateheor

sheisreadyforthenextresponse.DialogueExample2


6/12



Caller: AAAAAAAAAAAHHHHHH!Callerignoresthesystemsinitiativeandscreamsasazombiebitesintohis

orherarm.

System:

Iamdispatchingassistancetoyourlocation.Whatis

youremergency?

Systemacknowledgesthecallerssituationandforwardshisorher

locationcoordinatesandarecordingoftheconversationtoapplicable

personnel.Questionisrepeatedincase

additionaldetailscanbeprovided.

Caller:[Soundsofscreamsand

zombiegruntsinthe

background.]

Callerisbeingmauledbyzombies.

System: Whatisyouremergency?

Systemrepeatsthequestioneveryfive

secondswhilecontinuingtorecordtheconversationforlatermanualanalysis.

DialogueExample3



Caller: INEEDHELP!Callerrespondstotheapplications

inquiry.

System:Iamdispatchingassistancetoyourlocation.Howmany

zombiesarepresent?

Systemrecognizesthekeywordhelp,

acknowledgesthecallersinexperienceandforwardshisorherlocationcoordinatesandarecordingofthe

conversationtoapplicablepersonnel.

Aquestionisaskedforfurtherdetail.

Caller: ALOT!

Numbersalongwithgeneralquantitativephrasessuchasalot

mustberecognizedbythesystemforlateruse.

DialogueExample4


7/12



Caller: Guns! Callerignoresthesystemsinitiativeandstatesaknownkeyword.

System:Nearestguncacheisat

Hmeenpuisto28.Wouldyoulikedirections?

Systemrecognizesgunkeywordand

providesnearestlocation.Takesinitiativebyofferingdirections.

Caller: Yes. Calleracceptsofferfordirections.

System:[Directionsdialogue

initiated]

DialogueExample5



Caller: Twozombies.

Callerignoresthesystemsinitiativeandstatesaknownkeyword.A

numberisprovidedtoaidanalysisandupdatepublicmaps.

System:Twozombiesnoted.Doyourequireassistance?

Systemconfirmstheinputandinitiatesfurtherdialogue.

Caller: No.[Callerhangsup]Callerwasnotindangerandsimplywantedtoreportazombiesighting.

DialogueExample6


8/12



Caller: Whereisthenearestsafehouse?

Callerignoresthesystemsinitiativeandstatesaknownkeyword.

System: Nearestsafehouseisat--Systemqueriesdatabaseandbegins

providinginformationbutisinterruptedbycallersscream.

Caller: AAaGaURAAAaaHHHHHCallerbarges-inwhenazombiejumps

himorherfrombehind.

System:Iamdispatchingassistancetoyourlocation.Whatis

youremergency?

Systemrecognizesscreamandstopsgivingdirections.Personnelare

notifiedandinitialquestionisrepeatedeveryfivesecondswhile

continuingtorecordtheconversation

forlatermanualanalysis.

DialogueExample7



Caller:Iamsurroundedbyzombies!

Panickingnoviceuserengagesthesystemsinitiativeandanswersthe

question.

System:Areyouabletosafelyflee

thesituation?

Systempreparestosuggestassistance

byclarifyingthesituation.

Caller: Yes! Callerrespondstoquestion.

System:Nearestsafehouseisat

Satamakatu17.Wouldyoulikedirections?

Systemacknowledgesthecallers

abilitytorunandautomaticallyrecommendsthelocationofthe

nearestsafehouse.Directiondialogue

ensues.

DialogueExample8


9/12



Caller:IwantagunsoIcankillsomezombies.

Callerdoesnthaveanemergency,butknowshecancalltoreceivethe

locationofguncaches.

System:NearestguncacheisatHmeenpuisto28.Would

youlikedirections?

Systemrecognizesgunkeywordandprovidesnearestlocation.Takes

initiativebyofferingdirections.

Caller: Nope.Iknowwherethatis.

Callerdeniesofferfordirectionsand

hangsup.

DialogueExample9



Caller: HELPHELPHELPHELP! Callerfranticallyyellsforhelp

System:

Iamdispatchingassistancetoyourlocation.Howmany

zombiesarepresent?

Systemrecognizesthekeywordhelp,acknowledgesthecallersinexperience

andforwardshisorherlocation

coordinatesandarecordingoftheconversationtoapplicablepersonnel.

Aquestionisaskedforfurtherdetail.

Caller: FIVEORSIX.

Thesystemchoosesthelastnumber

statedtorecordinthedatabase.Thisallowscallerstocorrectthemselvesif

needbe.

DialogueExample10


10/12

3.MultimodalInteractionTechniques

Alongwiththeaforementionedspeechinput/outputinterface,location

coordinatesaretakenfromcallersmobilephonesusingGPStechnology(orcell

towertriangulationwhenGPSdataisunavailable).Thespeechinterfacewill

operateatthesametimebutindependentlyofthelocationmanagementmodule,

makingthemodalitiesconcurrent.

Thecallerscoordinatesareplottedontoamapwithapictureofazombie

indicatingthattherewasazombiesightingatthatgivenlocation.Thetimeat

whichthecallercontactedtheemergencysystemisplacedalongsidetheicon.If

thecallerprovidedaquantitativenumberofzombieslocatedathisorherpositionattimeofcall,thisnumberisalsosituatednexttothelocationicon.

Sightingsoftenormorezombiesinoneplaceconstituteahorde,andaniconof

multiplezombiesalongsideeachotherisusedinplaceofthestandardzombie

icon.Ifaqualitativefigurewasgiven,suchasalot,itisconsideredahorde.

Thesezombie-sightingmapsareshownonlargeLCDdisplaysinsafehousesand

busypublicareas.Zombiespoorvisionwillallotthemlittleinterestinthe

displaysinsituationswherenoisewoulddisturbandattracthordes.

4.ImplementationPlan

Aframe-baseddialoguesystemdesignedusingtheVoiceXMLmarkuplanguage

willmanagedialogueflow.Theopen-endednessaffordedbyframe-based

systems,opposedtotherigorousstructureoffinitestatemachines,willallow

missingpiecesofinformationtobeprovidedtotheapplicationatthewillofthe

caller.Thesystemcanbedevelopedanddeployedinacomprehensivevoice

environmentsuchasNuance.VoiceXMLsabilitytorapidlyprototype,test,

deploy,anditerateapplicationsisanimportantfeatureinconstantlychanging

safetyandrescueenvironments.

Thesystemwillbeinstalledandconfiguredonseveralserverslocatedin

geographicallydifferentplacestodecreasethelikelihoodofserveroutages

occurringsimultaneously.FollowingthesetupoftraditionalIVRsystems,thearchitectureshouldinvolvespeechrecognitionandsynthesismodulesalong


11/12

withdialogue,presentation,database,andlocationmanagementmodules.The

diagrambelowdemonstratesthecomponentsoftheserversystemarchitecture

andtheirrelationshiptooneanother.

Figure1:Serversystemarchitecture

5.EvaluationPlan

Theultimateevaluationplanwillbewitnessingwhetherthesystemiscapableof

preventingorpostponingthedemiseofhumancivilizationduringazombie

apocalypse.Untilthedaycomeswhentheundeadrisefromtheirgraves,

traditionalspeechinterfaceevaluationmethodologiescanbeusedtoquantify

thequalityofthesystem.

Worderrorrate(WER)isanimportantmetrictocontrolinthissafetycritical

applicationespeciallyconsideringtheword-spottingtechniquesmentionedin

previoussections.AWERaslowaspossible,nohigherthan5%,willensure

callersareabletoreceivetheattentiontheyneed.Concepterrorrateshouldalso

bekeptaslowaspossiblebecauseincorrectlyrecognizedconceptswillresultin


12/12

thesystemprovidingirrelevantinformation.Perplexityshouldbemonitoredto

verifythatlongerstringsofwordscanbeproperlyidentified.

Taskcompletionratesandtimesarecriticalapplicationlevelmetricsthatmust

bekepttoaminimumbecause,similartowordandconcepterrorrates,the

largerthevalueofthismetric,themorelikelyitisthatliveswillbelost.While

moneymaynotbeofconcernduringazombieapocalypse,significantcost-

savingswillbeintroducedbytheimplementationofthissystemthroughthe

reductionofliveoperatorexpenses.

Significanthuman-computerstudiesshouldbeperformedbeforethesystemis

released.Testsubjectscanbeaskedtocallthenumberandperformavarietyof

simulatedtasks.Speechrecognitioncanbecomparedagainstotherinputmodes,

suchastouchtoneinputorSMSmessaging.Testparticipantsabilitiestoquickly

obtaininformationonfictitioussafehousesandguncacheswillemulatethereal

worldexperiencesthatsomedaywillbehadbycitizensinthenot-so-distant

future.

using location-aware multimodal speech interfaces to survive a zombie apocalypse

Documents