dress.code #3 - interactive technologies
TRANSCRIPT
presented by
What We Do & Who We Are With Roots In Design, Development, And Film, We Meandered The Crowded Landscape Of Web And App Creation, Eventually
Casting A Wider Net With More Innovative And Imaginative Solutions For Our Diverse Clientele.
3 1 50+ ∞Offices Promise Awards Bad Jokes
July 2015
A Digital Production CompanyEqual Parts Craft & Curiosity
Development Content Creation Experiential
July 2015
THIS IS DRESS.CODEA Community Where Like-Minded Individuals Can Come Together To Learn To Subvert Traditional Approaches To
Technology In Media.
CREATIVE TECHNOLOGY IS OUR FORTE
July 2015
Jean-Michel KristofferJustin
July 2015
THE MAKING OF THE MOLSON’S GLOBAL BEER FRIDGE (ARDUINO, ANDROID, GOOGLE APIS)
HUNGER GAMES MOCKINGJAY FACE RECOGNITION ACTIVATION (KINECT, FACE TRACKING)
July 2015
INTERACTIVE TECHNOLOGIES
July 2015
Molson’s Global Beer FridgeCASE STUDY
Objective
Locally translate multiple languages from the original spoken word to English.
Determine if the phrase “I am Canadian” was spoken.
Arduino Android
July 2015
Google APIs
July 2015
Problems
- Language / speech is complex
- Very large amount of processing power is required
- Series of very broad corpora (bodies of preprocessed text and speech patterns)
- The availability of such corpora was limited at best (ex: restricted to Universities / selection of languages)
July 2015
✓ Most advanced
✓ Largest number of languages
✓ Best API
๏ Not a commercially available product
๏ Development purpose only
PROS CONS
Google Speech Recognition API
July 2015
Process
1. Record phrase and convert to sound file (.flac)
2. Transmit to Google Speech Recognition API with selected languages to compare against with
3. Receive transcribed speech or a failure message
4. Compare transcribed speech to known phrase in that language
5. Confirm that the required phrase was spoken
Google Speech Recognition API
July 2015
Limitation
Google Speech Recognition API
๏ 50 requests per day
๏ Limits access to predefined IP addresses
๏ New configuration would be necessary on every location or IP address change
July 2015
✓ Chromium and Chromium OS
✓ An open source web browser for Linux that emulates Chrome
✓ Includes a Speech API key which can be used by developers within reason
✓ Daily requests and the IP limitation was overcome
Chromium Project
July 2015
Web Speech API
(Demo)
July 2015
Android app
✓ Main controller
✓ Voice recognition
✓ Display
✓ Audio visualizer
July 2015
Hardware
July 2015
Arduino
✓ Open-source computer hardware and software
July 2015
Arduino IDE
✓ Open-source software
✓ Windows, Mac, Linux
✓ Written in Java
✓ Code in C/C++
July 2015
HARDWARE / APIs
Arduino
Android app
Microphone
Bluetooth
Serial Data
Google Speech Recognition API6 RGB Addressable LEDs
Magnetic Lock
Relay
Button
July 2015
Arduino Box
✓ Arduino Uno
✓ Relay Shield
✓ Bluetooth
March 2015
Hunger Games MockingjayCASE STUDY
Hunger Games MockingjayCASE STUDY
Objective
Kinect Camera Face Tracking Website (share / gallery)iPad (share)
July 2015
Take a selfie and a branded face-paint graphic will be applied on their face
Share their poster
July 2015
✓ Points (1347), Triangles (2340)
✓ Hair Color, Skin Color
✓ Face properties (glasses, mouth, eyes)
๏ Output quality (1080p resolution video camera)
PROS CONS
FACETRACKING - R&D
HD Face Tracking SDK (Kinect)
(Example)
July 2015
✓ Performance
✓ Open source๏ Tracking
PROS CONS
FACETRACKING - R&D
ofxFaceTracker (openFrameworks)
(Example)
July 2015
✓ Performance
✓ Tracking
✓Cross-platform [ C++ (Android / iOS) - AS3 - JS ]
๏ $$$ (around $2500 / project)
PROS CONS
FACETRACKING - R&D
Beyond Reality Face Nxt
(Example)
July 2015
✓ SDK
๏ Liveview (via Wi-FI)
DSLR camera(Canon)
Mirrorless Camera (Sony)
CAMERA - R&D
Camera
๏ Shut down after 2h (with liveview) ๏ API (via Wi-FI)
✓ Liveview
July 2015
Kinect
(Example)
✓ Skeletal tracking
✓ Face tracking
✓ 1080p camera
✓ Speech Recognition.
July 2015
HARDWARE
TV
Kinect
Camera
Laptop
iPad
July 2015
TECHNOLOGIES
Create a poster
Kinect
Camera Video stream (as Webcam)
Kinect API (via ANE)
Custom filters (liveview)
Generate picture Upload to Server(http://localhost:4000/)
Face Tracking
July 2015
TECHNOLOGIES
Share a posterServer iPad app (Web App)
fs.watch
Send email (via SendGrid)
Upload pictures
www
REST API
Refresh gallery
Form completed
Show pictures
Show gallery
April 2015
Challenges
✓ UI / Kinect Gesture
✓ Lights setup
✓ Focus
THANK YOU
Jean-Michel KristofferJustin
July 2015