tele-robotics demonstration (rover tele-operation) justsap-pisces symposium 2008 november 11, 2008
TRANSCRIPT
Tele-Robotics DemonstrationTele-Robotics Demonstration(Rover Tele-operation)(Rover Tele-operation)
JUSTSAP-PISCES Symposium 2008JUSTSAP-PISCES Symposium 2008
November 11, 2008November 11, 2008
2
AgendaAgenda
Demonstration overview
Technology and Products of remote control
platform
Let’s enjoy “Rover Tele-operation Time Trial”
3
Demonstration overview Demonstration overview
Let's experience the effect of "latency cancLet's experience the effect of "latency cancel technology" for teleoperation.el technology" for teleoperation.
Let's experience the effect of "latency cancLet's experience the effect of "latency cancel technology" for teleoperation.el technology" for teleoperation.
RoverWireless LANAccess pointInternet simulate PC
Operation PC
PISCES
Simulate data transmission delay
in Internet
InternetInternetInternetInternet
Japan/USA
4
Technology and ProductsTechnology and Products
You can develop You can develop advancedadvanced remote control remote control system system easilyeasily and at a and at a low costlow cost..
TTeleelePPresence over resence over IPIP TTeleelePPresence over resence over IPIP
Platform for the remote control Platform for the remote control system system which requires the digital which requires the digital video video transmitted by wireless transmitted by wireless
LAN and internet.LAN and internet.
5
Technology and ProductsTechnology and Products
Advanced remote control system Operation from everywhere of Internet available places
High operativity using Motion JPEG video
Easily and low cost I/F for using COTS equipments (WirelessLAN, Internet, PC, game con
troller, etc.)
Software Library for easy system development by users
All-in-one platform for video-based remote control system
6
Remote control system using TPIPRemote control system using TPIP
Video & Sound data、 Sensor dataVideo & Sound data、 Sensor data
Control command data、 Sound dataControl command data、 Sound data
internetinternetinternetinternet
PC
controller
Target SystemTarget System
Wireless LAN card
WireLess LANAccess point
Bidirectional WireLess LANIEEE802.11a/b/g
NTSCCamer
a
SensorMicrophon
e
Speaker
Motorcontrol
input
output
TPIP UnitTPIP Unit
TPIP LibraryTPIP Library
7
Minimization of “Video transmission delay”Minimization of “Video transmission delay”
NTSC video capture
JPEGEncoder
JPEG compression
Wireless LANIEEE802.11g20Mbsp 15Kbyte(1/60)
Decoder&
rendering
JPEGdecode & rendering
PC monitorRefresh Rate:60Hz
VGA/QVGA 30fps
NTSC Camera
+ Internet
33ms
Processing timeProcessing time
20ms
6ms
20ms
17ms
TPIPTPIP Total delay time
96ms96ms≒100ms≒100msWeb Camera Total delay time
500ms500ms
Network Camera
VGA 15fpsQVGA 30fps
8
High-speedHigh-speed image compressionimage compression
30 frames/sec. (NTSC -> Motion JPEG)
No re-transmissionNo re-transmissionCommunicationCommunication UDP/IP
High-speedHigh-speedimage renderingimage rendering
High-speed Motion JPEG decoding, image processing middleware
Resolution VGA(640 x 480),QVGA(320 x 240)
Frame rate 30 frame/sec
Video transmission delay Less than 100ms
Performance
Minimization of “Video transmission delay”Minimization of “Video transmission delay”
9
Removal of “Transmission delay influence”Removal of “Transmission delay influence”
1sec delay1sec delay
moves during delay time
4km/h4km/h (( ex. walkex. walk ))
10km/h10km/h (( ex. model carex. model car ))
HawaiiHawaii JapanJapan
6sec? delay6sec? delay EarthEarth MoonMoon
HawaiiHawaii
2.8m
MoonMoon
HawaiiHawaii
MoonMoon
5700km5700km
384400km384400km
16.8m
6.6m
1.1m
10
received imagereceived image captured at “captured at “ T T ””
corrected imagecorrected image view at view at T+ΔTT+ΔT
Steering angleSteering angle
speedspeed
Image transmissionImage transmissiondelaydelay
ΔTΔT :: Image transmission delayImage transmission delay
specify a red frame partspecify a red frame partby three data by three data
PointPoint TT
T+ΔTT+ΔT
Removal of “Transmission delay influence”Removal of “Transmission delay influence”
11
Correction of “Transmission delay fluctuation”Correction of “Transmission delay fluctuation”
Remove the video frame intervals fluctuation which dRemove the video frame intervals fluctuation which degrades operativity without image bufferingegrades operativity without image buffering
Delay time : t1
T1 T2 T3
T4
Time Stamp
( t1=“Time now” -T1 )
SendSendimageimage
T1
T2
T3 T4
T1
T2
T3
T4
ReceiveReceiveimageimage
DisplayDisplayimageimage
A fluctuation by the transmission delay
occurs
display images at same interval by time stamps
12
Let’s enjoy “Rover Tele-operation Time Let’s enjoy “Rover Tele-operation Time Trial”Trial”
Start and GoalStart and Goal
First First GateGate
Second Second GateGate
Latency cancel technology
1st lap OFF
2nd lap ON
13
An Introduction “TPIP Unit”
14
Image processing boardImage processing boardImage processing boardImage processing board
Control boardControl boardControl boardControl board
The feature of TPIP unitThe feature of TPIP unit
Value remote control platformValue remote control platform
All-in-one platform for video-based remote control systemAll-in-one platform for video-based remote control system((Image processing board, control board, software libraryImage processing board, control board, software library))
Various input/output for remote control systemVarious input/output for remote control system
TPIP UnitTPIP UnitTPIP UnitTPIP Unit
Input image : NTSC
Output image : MotionJPEG (VGA/QVGA 30fps)
Sound input/output : Audio
Wireless Communication : IEEE802.11a/b/g
Digital I/O RC servo I/F
DC motor I/F
Pulse input
Analog input CAN I/F
RS232 I/F
15
TPIP Unit
Hardware
16
TPIP Unit TPIP Unit SpecificationsSpecifications
Vout
V1
V2
V3
pw
RS232
SH250MHz
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
WirelwssLAN(11a,11g)
Servo
NTSCCAMERA
Voltage
Encoder
V in
RS232
pw
Ethernet
WirelessLANCard
DC
6~
16
V SensorAnalog Input
JPEGchip
Motor SH4240MHz
Image processing boardImage processing boardTarget control boardTarget control board
P1
P2
A1
RS232
CAN I/F
RS232
Other unit
Other unit
5V
A3
A2
A4
A5
A7
A6
A8
I1I3I2I4
O1
O3
O2
O4
O5
O7 O
6O
8
SensorDigital INPUT
Switch
Audio
Mikespeaker
Vout NTSCdisplay
Item Specifications / performance
Communication
LANWireless : IEEE802.11a,g,b (PC card) × 1Cable : 10baseT / 100baseTX × 1
Image Image transmit
VGA/QVGA 30fps, delay : Less than 100ms( Remove IP network delay time )
Camera I/F Max 3( change signal )
SoundSound
transmitMonaural recording bidirectional transmit( Sampling Rate:16/8KHz,44.1KHz)
Board size 105×84×60mm
Item Specifications / performance
I/O RC servo I/F Max 10ch
DC motor I/F 1ch
Pulse input Encoder Input 2ch
Analog input 8ch
Digital I/O Input 4ch / output 8ch (TTL)
CAN I/F 1ch
RS232 I/F 2ch
17
Compatibility with the radio control system Compatibility with the radio control system
Image processingboard
R/C receiver
Servo
Servo Motor
amp
Motor
amp
Radio control system for hobby
Servo
Servo Motor
Motor
amp
Target controlboard
TPIP unitTPIP unit
DC MotorDC MotorOutputOutput
RC servoRC servoOutputOutput
11a
CameraCameraInputInput
SensorSensorInputInput
RC partsRC partsusableusable
RC servoRC servoOutputOutput
replaceablereplaceable
Two-wayTwo-wayDigital control commandDigital control commandVideo & Sensor dataVideo & Sensor data
One-wayOne-wayAnalog control Analog control command onlycommand only
DC MotorDC MotorOutputOutput
18
TPIP Unit
Software
19
SoundSoundcontrolcontrol
TPIP libraryTPIP library
OS : LinuxOS : Linux
JPEG compression boardJPEG compression board
Motor control boardMotor control board
JPEG device driverJPEG device driver
Real-Time Image transmitReal-Time Image transmitSoftwareSoftware
UDP / IPUDP / IP
S/W
H/W
Internet
TPIP UnitTPIP UnitTPIP UnitTPIP Unit
ControllerControllerSoftwareSoftware
PC/AT compatible machinePC/AT compatible machine
OS : Windows XPOS : Windows XPUDP / IPUDP / IP
JPEG decodeJPEG decodeSoftwareSoftware
Real-TimeReal-TimeImage transmitImage transmit
SoftwareSoftware
High-speedHigh-speedrenderingrendering
User programUser program
DLL DLL (Dynamic Link Library)(Dynamic Link Library)
TPIP LibraryTPIP LibraryTPIP LibraryTPIP Library
20
TPIP libraryTPIP library
USBUSB
DisplayDisplay
WirelessWirelessLANLAN
RS232RS232OtherOther
controllercontroller
Controller inputController inputfunctionfunction
Image control / transmitImage control / transmitfunctionfunction
SIO CommunicationSIO Communicationfunctionfunction
Graphic controlGraphic controlfunctionfunction
UDP functionUDP function
UDP-SIO communicationUDP-SIO communicationfunctionfunction
Base libraryBase library apply libraryapply library
TPIP libraryTPIP library
TPIP UnitTPIP Unit
21
UniversityUniversityTechnicalTechnicaljunior collegejunior college
ResearchResearchlaboratorieslaboratories
SanritzAutomationSanritzAutomation
TechnicalTechnical informationinformation
TPIP Library TPIP Library source codesource code
CommunityCommunity
ProductProductinformationinformation
Research &Research &developmentdevelopment
resultsresults
offer & technical supportoffer & technical support
archivearchive
Research and development(school) Research and development(school) :: 26 members26 members
Research and development(Enterprise):Research and development(Enterprise):
11 members11 members
RRC Student member : 23 membersRRC Student member : 23 members
Student member : 9 membersStudent member : 9 members
Activities themeActivities theme・・ Robot ControlRobot Control・・ OpenCV Image data processinOpenCV Image data processingg etc…etc…
TPIP User CommunityTPIP User Community
22
And…And…・・ TPIP Platform customizeTPIP Platform customize・・ TPIP Systems construction supportTPIP Systems construction support
Image processing boardImage processing board(( compression & transmitcompression & transmit ))
\140,000\140,000
TPIP Platform PACKAGETPIP Platform PACKAGE\200,000\200,000
academic price \100,000academic price \100,000
Price in JapanPrice in Japan
Tele-Operation Rover PACKAGETele-Operation Rover PACKAGE\2,500,000\2,500,000
academic price \1,500,000academic price \1,500,000Remote controlRemote control
TargetTarget(Rover)(Rover)
TPIP UnitTPIP Unit
TPIP libraryTPIP library
TPIP UnitTPIP Unit
TPIP libraryTPIP library
Image processing boardImage processing board
TPIP libraryTPIP library
You can choose TPIP PlatformYou can choose TPIP Platform