voice-controlled home automation using watson, raspberry pi, … · 2017. 9. 13. · watson iot...

@k_bankoleOpenWhisk.org

Voice-controlled HomeAutomation UsingWatson, Raspberry Pi,and Openwhisk

@k_bankoleOpenWhisk.org

Voice Enabled Assistants (Adoption)

@k_bankoleOpenWhisk.org

Voice Enabled Assistants (Usage)

@k_bankoleOpenWhisk.org

Voice Enabled Assistants (Workflow)

• Initialize Voice Recording

• Record Audio

• Apply DSP algorithms

• Transcribe Audio to text

• Interpret text result

• Execute action

@k_bankoleOpenWhisk.org

Hotword

•Background process that listens for specificspeech pattern

– “Ok, google”

– “Alexa”

– “Hey Siri”

•Able to run on devices with limited resources(Arduino, Raspberry Pi)

@k_bankoleOpenWhisk.org

Hotword

Microphone Array

•Consists of multiple microphones

•Able to record all channelssimultaneously

•Separate streams are processed toincrease transcription confidence

Watson Speech to Text

Input audio via HTTP or Websocket

curl -X POST -u <username>:<password> --header "Content-Type: audio/flac” --data-binary @<path>audio-file.flac

"https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true"

{

"confidence": 0.891,

"transcript": "Turn on the light in the living room"

}

Conversation

• Intent – Action userwants to trigger

•Entity – Context foraction

•Dialog – Determinesresponse, next step

{

...

”input": [

{

”text": “turn on

the fan” }]

}

{

...

”entities": [

{

”confidence": 0.8745,

”value": “fan”

}],

”intents": [

{

”confidence": 0.9348,

”intent": “turnon”

}]

}

Conversation Service

Conversation

@k_bankoleOpenWhisk.org

Demo Architecture

@k_bankoleOpenWhisk.org

OpenWhisk provides an elegant solution

OpenWhisk is a cloud platform

that executes code

in response to events

@k_bankoleOpenWhisk.org

Why “Serverless”?

• Focus on code, not maintenance

• Less overhead

• Reusability

• Scales on demand

• Usage based pricing

@k_bankoleOpenWhisk.org

Respond to events

Database changes

Sensor value

Cognitive trends

Scheduled tasks

Data from Mobile devices

@k_bankoleOpenWhisk.org

Triggers, actions, rules (and packages)

o Services define theevents they emit astriggers.

o

o Developers associateactions to handle theevents via rules.

o Packages are used tobundle and distributesets of actions

@k_bankoleOpenWhisk.org

Triggers

A class of events that can happenT

Social events

Data changes

Device readings Location updates

User input

@k_bankoleOpenWhisk.org

Actions

Code that runs in response to an event(that is, an event-handler)

A

@k_bankoleOpenWhisk.org

Actions

Can be written in a variety of languages, such asJavaScript, Python, Java, and Swift

A

function main(msg) {return { message: 'Hello, ' + msg.name + ' from ' + msg.place };

};

@k_bankoleOpenWhisk.org

Actions

Can be composed to create sequences thatincrease flexibility and foster reuse

A

AA:= A1

+ A2+ A3

AB := A2+ A1

+ A3

AC:= A3

+ A1+ A2

@k_bankoleOpenWhisk.org

Rules

An association of a trigger to an action in a manyto many mapping.

R

R := T A

@k_bankoleOpenWhisk.org

Packages

A shared collection of triggers and actionsP

A

A read

write

T changes A translate A forecast

A post

T topic

OpenSource A myAction

T myFeed

Yours

T commit

ThirdParty

@k_bankoleOpenWhisk.org

OpenWhisk enables event driven applications

EventProviders

Cloudant

GitHub

Weather

…

Which triggers execution ofassociated OpenWhisk action

2

Slack

JS Swift Docker …

An event occurs, for example• Commit pushed to GitHub repository• Data entered in Cloudant

1 OpenWhisk

@k_bankoleOpenWhisk.org

OpenWhisk awaits events, fetches mapped code, runs it ina container

Pool of actions

Swift DockerJS

Trigger

1

Runningaction

Runningaction

Runningaction

3

OpenWhiskEngine

2

Watson IoT Platform (MQTT)

• Platform enables apps toshare and consume datacollected by authenticateddevices, sensors..

• Provides real-time andREST APIs to communicatewith your devices andconsume the data

@k_bankoleOpenWhisk.org

CitizenM

@k_bankoleOpenWhisk.org

WiFi vs RF Outlet

RF Outlet (Schematic)

IR Outlet (Schematic)