portland cyber theatre marek perkowski graduate seminar, friday, may 7, 2004 part 1
Post on 19-Dec-2015
221 views
TRANSCRIPT
Portland Cyber Portland Cyber TheatreTheatreMarek Perkowski
Graduate Seminar, Friday, May 7, 2004
Part 1Part 1
Portland Cyber Portland Cyber TheatreTheatreFunded by Korea Institute of Science and Technology,
Daejeon, Republic of Korea and Intel Corporation, Hillsboro
Martin Lukac, Jacob Biamonte, Stefan Gebauer, Robert Klug, Myron Machado, Normen Giesecke, Atsumu Iseno, Tsutomu Sasao, Michele Folgheraiter, Uland Wong, Mikhail Pivtoraiko, Jeff Morris, Chris Motch, Nicola Sian Rees, Ping Hang Cheung, Randall Borck, Ray Schmelzer, Steve Mbah,Yu Xia, Akashdeep Aulakh, Tyler Nguyen, Jeff Allen, Kevin Xue, Kevin Chen, Ben Xue
ContributorsContributors
A robot with many bodies, A robot with many bodies, many personalities, quantum, many personalities, quantum, fuzzy and logic brains fuzzy and logic brains oror how how to Use of Machine Learning to Use of Machine Learning
based on Constructive based on Constructive Induction in Dialogs with Induction in Dialogs with
Robotic HeadsRobotic Heads
Or more scientifically
Talking RobotsTalking Robots• Many talking toys exist,
but they are still very primitive
• Actors for robot theatre, agents for advertisement, education and entertainment.
• Designing inexpensive natural size humanoid caricature and realistic robot heads
We concentrate on Machine Learning techniques used to teach robots behaviors, natural language dialogs and facial gestures.
Dog.com from Japan
Work in progress
Robot with a Personality?Robot with a Personality?• Future robots will interact
closely with non-sophisticated users, children and elderly, so the question arises, how they should look like?
• If human face for a robot, then what kind of a face?
• Handsome or average, realistic or simplified, normal size or enlarged?
• Why is Kismet so successful? •We believe that a robot that will interact with humans should have some kind of “personality” and Kismet so far is the only robot with “personality”.
•The famous example of a robot head is Kismet from MIT.
Professor Perky – the robot Professor Perky – the robot built by us in Japanbuilt by us in Japan
A robot of Martin Lukac: A robot of Martin Lukac: Cynthi-A singsCynthi-A sings
Robot face should be friendly and funny
The Muppets of Jim Henson are hard to match examples of puppet artistry and animation perfection.
We are interested in robot’s personality as expressed by its:– behavior,
– facial gestures,
– emotions,
– learned speech patterns.
Behavior, Dialog Behavior, Dialog and Learningand Learning
• Robot activity as a mapping of the sensed environment and internal states to behaviors and new internal states (emotions, energy levels, etc).
• Our goal is to uniformly integrate verbal and non-verbal robot behaviors.
Words communicate only about 35 % of the information transmitted from a sender to a receiver in a human-to-human communication.
The remaining information is included in para-language.
Emotions, thoughts, decision and intentions of a speaker can be recognized earlier than they are verbalized.
Fig. 1. Learning Behaviors as Mappings fromFig. 1. Learning Behaviors as Mappings fromenvironment’s features to interaction proceduresenvironment’s features to interaction procedures
AutomaticAutomaticsoftwaresoftwareconstructionconstructionfrom examplesfrom examples
(decision tree, (decision tree, bibi--decomposition,decomposition,AshenhurstAshenhurst, DNF), DNF)
Speech frommicrophones
Image featuresfrom cameras
Sonars and othersensors
Emotions andknowledge memory
probability Verbal responsegeneration (textresponse and TTS).Stored sounds
Headmovementsand facialemotionsgeneration
Neck and shouldersmovement generation
Robot Head ConstructionRobot Head Construction
Furby head with new controlFurby head with new control JonasJonas
We animate various kinds of humanoid heads with from 4 to 20 DOF, looking for comical and entertaining values.
Robot Head ConstructionRobot Head Construction
SkeletonSkeleton Alien
We use inexpensive servos from Hitec and Futaba, plastic, playwood and aluminum.
The robots are either PC-interfaced, use simple micro-controllers such as Basic Stamp, or are radio controlled from a PC or by the user.
AdamAdamMarvin the Crazy RobotMarvin the Crazy Robot
Technical Construction Technical Construction DetailsDetails
MaxMax
Image processing and pattern recognition uses software developed at PSU, CMU and Intel (public domain software available on WWW). Software is in Visual C++, Visual Basic, Lisp and Prolog.
BUG (Big Ugly Robot)BUG (Big Ugly Robot)
Visual Feedback and Learning based on Visual Feedback and Learning based on Constructive InductionConstructive Induction
Mister ButcherMister Butcher
4 degree of freedom neck
Latex skin from Hollywood
Virginia WoolfVirginia Woolf
heads equipped with microphones, USB cameras, sonars heads equipped with microphones, USB cameras, sonars and CDS light sensorsand CDS light sensors
Professor Perky Professor Perky
1 dollar latex skin 1 dollar latex skin from Chinafrom China
• We compared several commercial speech systems from Microsoft, Sensory and Fonix. •Based on experiences in highly noisy environments and with a variety of speakers, we selected Fonix for both ASR and TTS for Professor Perky and Maria robots.
• We use microphone array from Andrea Electronics.
Professor Perky with automated Professor Perky with automated speech recognition (ASR) and speech recognition (ASR) and text-to-speech (TTS) capabilitiestext-to-speech (TTS) capabilities
MariaMaria
20 DOF
Construction Construction details of Mariadetails of Maria
location of location of controlling controlling rodsrods
location location of head of head servosservos
location location of remote of remote servosservosCustom
designed skin
skull
Animation of eyes and eyelidsAnimation of eyes and eyelids
Frown of eyes and eyelidsFrown of eyes and eyelids
Smile and Jaw ActionSmile and Jaw Action
High School students help us to build High School students help us to build robot headsrobot heads
The newest robot of Jake Biamonte and Jeff The newest robot of Jake Biamonte and Jeff Allen has many sensors on the faceAllen has many sensors on the face
The back of the new robot head
The hands do not move. The new, movelable hands are under The hands do not move. The new, movelable hands are under design by Michele Folgheraiter, our collaborator from Italy.design by Michele Folgheraiter, our collaborator from Italy.
The new The new robot is robot is mobile.mobile.
The new robot is controlled by a
micro-controlled and a laptop. It
will move randomly and talk
about PSU greatness. See
demo soon.
Laptop and its screen
I am I am friendly friendly and and beautifulbeautiful
I can turn I can turn my body my body left and left and right right towards towards the the speaker speaker or away or away from himfrom him
Figure 2: Advanced construction of the robotic head, with servos inside the head, added are eyes and jaws. Also can be seen wires for future movements of cheeks, lips, eyebrows, chins and front.
Figure 1: Empty skull of the robot
Figure 3: Opened back head of the robot showing implacement of servos controlling the lower jaw, eyes, eye brows, cheeks and the chin.
Figure 4: Latex skin on the robotic head
New neck mechanism constructed by Jake and two Jeffs.
New neck mechanism constructed by Jake and two Jeffs.
On a grant from Korea Institute of
Advanced Studies we are
building a theatre of traditional
korean masks
Hahoe Hahoe RobotsRobots
Hahoe RobotsHahoe Robots
Hahoe RobotsHahoe Robots
The Hahoe Theatre has 12 robots, automatically controlled stage, lights and smoke generators. Help needed to design more heads and program scripts.
The Hahoe robots will sing and
dance.
A long neck and springs.
Mechanisms of Braitenberg’s Vehicles for Quantum vehicles explained next
Please wait for change of slides file to Part 2