음성 ui - 봇(bot)과사물인터넷의 설계및개발 · 2019-06-27 · title: how to use the...

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted

음성 UI - 봇(Bot)과사물인터넷의설계및개발Voice User Interface & Assistant & Bots & IoT

황주필 ([email protected])유원조 ([email protected])

Sales Consultant, Infrastructure CloudAug 30, 2017

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted


User Interface의변화CLI / GUI ??

Confidential – Oracle Internal/Restricted/Highly Restricted 3


HAL 9000


Computer

HAL & Computer


User Interface


?


User Interface



User Interface – CLI


CLI - Command Line Interface


User Interface



User Interface – GUI (마우스에서손가락으로)


GUI – Graphical User Interface


Evolution of interaction



User Interface - Voice


Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 12

Mobile Voice Interface

2011 2012 2014

Speech Recognition에서 AI로의발전


Voice User Interface - Wars

스마트폰에서 Apple Siri와 Google Now같은음성인식 AI비서서비스로경쟁을했지만, Amazon이 2014년인공지능스피커 Echo를출시, 음성인식소프트웨어인 Alexa의개발자키트를외부에오픈 Skill이빠르게늘어남 (검색, 쇼핑, 뱅킹등등..) Echo 판매량증가


기업명 내용

Apple - 2010년 ‘Siri’ (음성인식소프트웨어) 인수- 2016년 VocalIQ (음성인터페이스) 인수

Amazon - 2011년 ‘Yap’ (STT : Speech-to-Text) 인수- 2014년 ‘Evi’ (지식기반 Semantic Search Engine), ‘INOVA’ (TTS : Text-to-Speech) 인수

Facebook - 2013년 ‘Jibbigo’ (모바일다국어음성번역) 인수- 2015년 ‘Wit.at’ (음성인식), ‘Two Big Ears’ (VR 오디오) 인수

Google - 2014년 ‘DeepMind technologies’ (AI 머신러닝) 인수- 2016년 API.AI (음성인식, 자연어처리) 인수, Limes Audio(음성처리) 인수




회사 이름 API / SDK

Amazon Alexa (2014) Alexa voice service sdk 제공, 한국어지원안함

Google Assistant (2016) Google assistant sdk 제공, 한국어지원

Apple Siri (2011) Siri SDK for iOS, macOS 제공, 한국어지원

SKT Nugu (2016) X

KT GigaGeenie (2017) 제휴신청필요 (법인심사 2주소요)

Naver Wave (2017) 음성인식 ‘Clova’ API 공개(Android iOS SDK), 일 3600초까지무료

Kakao Kakao mini (2017) 3분기 AI앱/스피커출시예정, 음성인식 “뉴톤” Android,iOS SDK



• Amazon – Alexa Voice Service (AVS)

• 2015 – Alexa Skills Kit (ASK) 공개, 2017년 1분기이용가능한 Skills 개수가 10,000 개 (Uber호출, Amazon물건구입, 스타벅스커피주문등..)



나만의음성인식비서를만들어보자



DIY Alexa !


집에 Raspberry Pi 하나쯤은있잖아요 !


DIY Alexa !


Step 1 … 10 따라하기


DIY Alexa !

Alexa-avs-sample-app

http://github.com/alexa/alexa-avs-sample-app

Alexa 개발자 – AVS 활성화 – 기기등록

Alexa! How is the weather today??


http://github.com/alexa/alexa-avs-sample-app


Google Assistant 는?



• google-oauthlib-tool –client-serect/home/pi/client_id.json –scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless

• googlesampeles-assistant-hotword


DIY Google Home !

• Raspberry Pi – Audio Enable

• Audio Lib 설치

• Python Module 설치 with pip– pip install –upgrade google-assistant-

sdk

– pip install –upgrade google-auth-oauthlib

• Google APIs > credentials > client id 등록및 client-secret json다운로드후 oAuth tool을통해서credentials.json 생성


한번보시죠Demo



Home Automation with Google Assistant



AI Speaker - 아키텍처

AI. Speaker Cloud

How’s the weather?

기본기능시간,날씨,메모,음악

Intent 파악

응답 Speech 생성

인터넷검색


AI Speaker - 아키텍처

AI. Speaker Cloud

How’s the weather?

Intent 파악

종료

Custom 기능Alexa Skill, Google Action

Other 시스템

“Ask my skill” “Talk to my Home”

Webhook


Home Automation - 아키텍처

음성인식,출력 Case에따른 Device 호출 Device 제어

자연어인식

의도(intent) 파악

Custom 기능 Action 호출

기기(TV, 에어컨, 조명)

명령 (On, Off)

기기제어 API 호출

기기제어 API

적외선(IR) 호출

Python Simple Http

Server

AI Speaker SDK


Home Automation - 아키텍처


Google Assistant SDK

Google Actions

Google Api.AI

Python Simple Http

Server

BroadLinkRM3 Mini

Python API

ngrokWebhook

https

HomeNetwork

Server

TCP

IR (적외선)

IRCustomAction

Talk to“My Action”


ngrok 이란?

이미지출처 https://ngrok.com/


TV, 에어컨제어 - IR (적외선)


직접만들수있음Receiver, Sender, LIRC단점: 공구비용, 단방향

BroadLink RM3 Mini


TV, 에어컨제어 - IR (적외선)


TV

Audio

Aircon

사용자추가기기

192.168.1.111192.168.1.100

Send SIGNAL Code

Learn SIGNAL

IR Signal

TV ON Code : 26001399401923….Audio ON Code : 14454512345451….Aircon ON Code : 6234973491-384….

TCP


TV, 에어컨제어 - Code


pi@raspberrypi:~/rm3/python-broadlink/cli $ ls

README.md aircon.turnoff broadlink_discovery tv.channel1 tv.channel5 tv.channel9 tv.turnon

aircon.downtemp aircon.turnon MyHomeServer.py tv.channel2 tv.channel6 tv.channeldown tv.volumedown

aircon.morewind aircon.uptemp device.info tv.channel3 tv.channel7 tv.channelup tv.volumeup

aircon.smallwind broadlink_cli tv.channel0 tv.channel4 tv.channel8 tv.turnoff

pi@raspberrypi:~/rm3/python-broadlink/cli $ ./broadlink_cli -h

usage: broadlink_cli [--device DEVICE] [--type TYPE] [--host HOST] [--send SEND] [--sensors] [--learn] [--learnfile

LEARNFILE]

--send SEND send command

--learnfile LEARNFILE learn command and save to specified file

pi@raspberrypi:~/rm3/python-broadlink/cli $ cat tv.turnon

2600139940192398109823874098123098120398120974812738129739128673129038012983012938216498123213

BroadlinkRM3 Mini

Python API

01340891801…….

34923043123……

tv.channelup

tv.turnon

broadlink_cli

https://github.com/mjg59/python-broadlink

http://device.info/


TV, 에어컨제어 - Code


if command == 'channel':

if level != '':

if level == '11':

exeCmd = './broadlink_cli --send @' + device + '.' + command + "1"

subprocess.call(shlex.split(exeCmd))

exeCmd = './broadlink_cli --send @' + device + '.' + command + "1"


elif command == 'volumedown' or command == 'volumeup':

for i in range(1,numlevel):

exeCmd = './broadlink_cli --send @' + device + '.' + command


...

...

def run(server_class=HTTPServer, handler_class=S, port=9876):

server_address = ('', port)

httpd = server_class(server_address, handler_class)

print("Starting httpd... 9876")

httpd.serve_forever()

Python Simple Http

Server

BroadLinkRM3 Mini

Python API


Oracle Bot Integration with Alexa



Live Demo - Pizza 주문


Oracle's Intelligent Bot

피자주문

크러스트

사이즈

피자종류

사이드

User결제수단

기존카드정보확인

접속요청


Oracle Bot Integration with Alexa - 아키텍처


AmazonAVS SDK

Alexa Voice Service

Alexa Skill(Oracle Pizza)

ask“Oracle Pizza”

Oracle Intelligent Bot

Channel

Channel Configurator

DialogFlow

Execution

CustomComponents

AI EngineWelcome to

oracle pizza

Cloud


Alexa Skill 설정

{"intents": [

{"slots": [

{"name": "command","type": "LITERAL"

}],"intent": "CommandBot"

},{

"intent": "AMAZON.StopIntent"}

]}

Intent schemaSkill invocation name


Oracle Intelligent Bot 의 Alexa Channel 설정


Oracle Intelligent Bot 동작방식

“I want to order a pizza” “Can I order large thin cheese pizza?”

메시지수신

1

의미파악

2

처리를위한특정정보기억

3

답변수행

4

Intent

Utterance

Entity

Dialog Flow


Oracle Intelligent Bot 의 Intent, Entity 설정


Web기반음성인식 with Oracle Bot



Web기반음성인식 with Oracle Intelligent Bot



Web기반음성인식 with Oracle Intelligent Bot - Code


if (!('webkitSpeechRecognition' in window)) {

upgrade();

} else {

start_button.style.display = 'inline-block';

recognition = new webkitSpeechRecognition();

...

recognition.onresult = function(event) {

var interim_transcript = '';

for (var i = event.resultIndex; i < event.results.length; ++i) {

if (event.results[i].isFinal) {

var recData = event.results[i][0].transcript;

sendToBot(recData);

...

var u = new SpeechSynthesisUtterance();

speechSynthesis.speak(u);

Ajax로 Oracle Bot 호출

음성인식객체

음성출력


Summary


• Machine Learning 의발달로점점완벽한음성인식가능

• AI Speaker 시장의확대

•다양한음성인식플랫폼의전성시대

• ChatBot 음성인식 Bot으로의전환

•음성인식시스템의 SDK의오픈개발자들의다양한기회


Let’s go home to Code!!



Safe Harbor Statement

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


음성 ui - 봇(bot)과사물인터넷의 설계및개발 · 2019-06-27 · title: how to use the...

Documents