ece477 project minotaur software narrative
DESCRIPTION
ECE477 Project Minotaur Software Narrative. Jon Roose. Project Overview. Home Security Robot Interaction through a W ebserver Robot driven manually through house and spots intruders Relays video to web interface using kinnect. Hardware Hierarchy. Software Hierarchy. Benefits of Model. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/1.jpg)
ECE477 Project MinotaurSoftware Narrative
Jon Roose
![Page 2: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/2.jpg)
Project Overview
• Home Security Robot• Interaction through a Webserver• Robot driven manually through house and
spots intruders• Relays video to web interface using kinnect
![Page 3: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/3.jpg)
Hardware Hierarchy
![Page 4: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/4.jpg)
Software Hierarchy
![Page 5: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/5.jpg)
Benefits of Model
• Reliance upon open source projects• Ease of access, available to HTML5 browsers• Emphasis on x86 architecture• Modularity• Scalability
![Page 6: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/6.jpg)
Software Hierarchy - Web Interface
![Page 7: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/7.jpg)
Software Hierarchy - Web Interface
• Apache Web Server– Serves interface to user’s web browser
• HTML5 and JavaScript WebSockets– Opens efficient communication with C&C Server– Real time commands, removes HTTP reliance
• RDP Video Stream– Implemented and configured through FFServer– Connects via an embedded video object
![Page 8: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/8.jpg)
Software Hierarchy - C&C Server
![Page 9: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/9.jpg)
Software Hierarchy - C&C Gateway
• C&C Server (Written in Python)– Mediator for command and control signals– Communicates on internal LAN with robot– Simple websocket interface with User’s browser– Leverages TCP/IP LAN interface to communicate
with robot via WiFi• FFServer (Open source application)– Accepts video from Atom board and serves to
user, manages RDP video stream
![Page 10: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/10.jpg)
Software Hierarchy - Atom Board
![Page 11: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/11.jpg)
Software Hierarchy – Atom Board
• Intelligent Drive System (C++)– Navigates room and avoids obstacles– Accepts commands on a TCP/IP socket from C&C– Interprets sensor data from Kinect and PIC24– Converts video into YUV encoding for FFMpeg
• FFMpeg (Open source application)– Accepts data via stdin from the Intelligent Drive
system and packages it in an MPEG video format
![Page 12: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/12.jpg)
IDS Software Architecture
• Thread 1 - 30 FPS event loop– Transcodes and Outputs video to FFMpeg– Polls Kinect Video and Depth Sensor– Polls network socket for C&C Commands
• Thread 2 - Queue driven event loop– Implements room mapping and human detection– Timers add standard events to queue– Flexible queue design allows adjusting task frequency– Communicates with PIC24 over RS232
![Page 13: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/13.jpg)
Software Hierarchy - Microcontroller
![Page 14: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/14.jpg)
Software Hierarchy - MicroController
• Low Level Peripheral Controller (C)– Implements fail-safe obstacle avoidance– Accepts commands via 5 byte protocol over RS232– Controls motors and polls IR sensors– Flag driven event loop
• Libfreenect & OpenNI (Open source)– Library for communicating with Kinect– Retrieves depth sensor and video data– Recognizes human form
![Page 15: ECE477 Project Minotaur Software Narrative](https://reader035.vdocument.in/reader035/viewer/2022062411/56816932550346895de0856f/html5/thumbnails/15.jpg)
Debugging
• Hardware/MicroController– Standard Reset pushbutton and debug LED– RJ11 allows for in-circuit debugging via ICD3– Headers for port pins broken out– Printing strings over SCI
• Atom Board– SSH directly into board– Sending metrics and debugging data to C&C– Sending bitmaps of room map