Slide 1

2USC Institute for Creative Technologies

Eric Forbell, Nicolai Kalisch, Kelly Christoffersen,Glenn Storm, Kevin Feeley, Fabrizio Morbini

SimCoach Authoring

#1SimCoach as a PlatformSimCoach is a technology platform that enables rapidly developed virtual humans hosted in a web environment

Enables more ubiquitous and scalable virtual humans and contentBuilds upon maturing virtual human technologiesBuilt upon open web standards, promoting adoptionSupports true conversational agents that take initiative Suitable for a variety of complex domainsEnables interactivity with a vast set of existing and future web resources Direct embedding of YouTube, Web articles, Website links, other online media#22I suspect that this slide will resonate the loudest with tomorrows attendees? - kfBuilding authorable virtual humansMultimodal Perception &UnderstandingHuman UserAnimationNaturalLanguageGenerationSpeechGenerationSpeechRecognitionNonverbalBehaviorGenerationRendererNaturalLanguageUnderstandingAgentFrameworkChallenges

Virtual Humans have complex data-driven componentsTools are typically not unified and unwieldy for non-technical authorsBalance between simplicity and expressivityDifferent applications impose different requirementsTraining is more focused, but requires assessmentSupport systems like SimCoach face more uncertainty with users

#SimCoach Authoring Approach Preliminary Goals

Build a unified tool to create complete virtual charactersAuthor, test and iterate all in the same environmentMinimize barriers for getting startedTool is fully web-based, nothing to install or configureUse available technologies, invent only when necessaryEmpower the author, restrict only when necessaryConstruct a solid architectural foundation to support domain-specific authoring applications

#SimCoach Architecture What needs authoring?Human UserTextualLanguageUnderstandingWeb clientAnimationFusionSpeechAudioNon-verbalBehaviorgenerationFrameworkDialogue ManagementAnimationVideoActionRealizationMultimediaResourcesSpeechAnalysisClassifiermodelDialoguepolicyInformationstateSpeechAudioBehaviorRulesAnimationLibraryResourcesUtterances3D Character#Even less complex, more focused on end-usersYou can see the start here of what might very well a couple of these genres of VH5SimCoach Authoring Tools

Early design iterations

#Added this slide -kf6Authoring ProcessCreate projectSelect 3D character model from library

question.confidentialityconsentToPTSDQuestionnaire == trueQuestion.what-is-PTSD

Build interaction logicCompose interaction through sub-dialogue scripts activated by state variables and user actionsTrain language understandingFor all specified user dialogue actions, provide exemplar text and variantsRefine character actionsEdit character utterancesImport speech audio, render animations#

Enabling technology for web-hosted characters

SmartbodyFaceFXNVBGencoderMaya

Maya

Maya

Maya

Maya

MayaCan parallelize here!Automated Character Animation Workflow

#Again, this is probably deeper than most can grasp, but at a high-level we have a number of different technologies handling different parts of the experience how the character behaves, moves, thinks, and speaks. All/most of which have been developed in-house at the ICT (not FaceFX).8Kraken Character Animation Production SystemcreateAudioanalyzeAudiocreateBmlcreateBvh

encodeMovie

createAudiocreateAudiocreateAudioanalyzeAudioanalyzeAudiocreateBmlcreateBmlcreateBvh

createBvh

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

buildAndRender

encodeMovie

encodeMovie

New ticketProcessorTicket finishProcessorAuthoring Tools

#If this slide were to be included, Id use it simply to show that the system has been designed (innovatively) such that when an Author creates content (a vignette for lack of a better word), that task/ticket is distributed to a number of different machines to process and execute before its all pieced together and delivered to the system.9TOOLS DEMO PRE-ALPHA TECH PREVIEW

#Future WorkBuild tutorials and in-tool support for new authorsImprove overall usabilitySome changes may be very invasive, but necessarySimplify creation of common dialogue behaviorsIntroduce behavior templatesImprove author control and preview of nonverbal behaviorIn process on DCAPSEnhance testing framework to help debugProvide support for other simulation environmentsStandalone, non-web simulations#Backups#Whats this slide for?12

#Moved this to before Demo screen13

#

#

#TOOLS DEMO PRE-ALPHA TECH PREVIEW

#