![Page 1: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/1.jpg)
Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrians and Vehicles
Dr J. Barthélemy, Dr N. Verstaevel, Dr H. Forehead, Senior Prof. P. Perez
![Page 2: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/2.jpg)
Edge Computing with Jetson TX2 for Monitoring Flows of Pedestrian and Vehicles
At SMART, we believe that People with good
information and good tools will make good
Decisions and change our world
2
![Page 3: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/3.jpg)
Technology and community: DLL
3
![Page 4: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/4.jpg)
Solving problems: The case of Liverpool
CBD is growing fast: new UoW campus, airport development,…
What does it mean for the city and its community? What are the problems?
Smart Cities and Suburbs Program: How can we use IoT to solve the problems?
4
![Page 5: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/5.jpg)
Which problems?
Let’s ask the people! Pedestrians- Where are they going?- What are the most popular routes?- What are the most congested locations?- Impact of city activity?
Cyclists- Which route are they taking?- How can we improve bike usage?
Cars- Live traffic?
5
![Page 6: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/6.jpg)
Sensors locations
20 sensors
Image credit: OpenStreetMap
6
![Page 7: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/7.jpg)
How about using CCTV?
7
Privacy!
![Page 8: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/8.jpg)
Sensor requirements
• Mobile units + leveraging CCTV infrastructure
• Privacy is important!
– On board video analytics
– Only indicators transmitted (no raw data!)
• Real-time image processing
• LoRaWAN network
– Long range, low bandwidth (200 bytes/message)
– Free to use by the community8
![Page 10: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/10.jpg)
History of the prototypes
10
![Page 11: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/11.jpg)
The sensorAn edge computing device
Components
— NVIDIA Jetson TX2 for onboard processing
— Pycom LoPy 4 for data transmission on The Things Network
— Camera (USB webcam / existing CCTV)
11
![Page 12: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/12.jpg)
Architecture of the solution
Fixed counters
Air quality x20
Noise level x20
x5
x15
Mobile counters
Sensors IoT Core ApplicationsTransport
+Private and Public APIs Traffic
modelling
Analytics
Dashboard
Citizens app
12
![Page 13: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/13.jpg)
From the input to the dashboard
Image Acquisition
• CCTV/Webcam (OpenCV)
Detection
• Deep Convolutional Neural Network
Tracking
• Kalman Filtering
Data transmission
• LoRaWAN/OneM2M
Dashboard/Database
Image credit: NVIDIA Corporation
Image credit: Pycom
13
![Page 14: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/14.jpg)
Detection: YOLO v3
Inference: Jetson TX2 @ 10 fpsTraining: Titan Xp
• Fully convolutional DNN• 106 hidden layers• Detections at 3 scales• 3 classes: person, bicycle, vehicles• Pascal VOC and COCO datasets
cuDNN
FP16+ +
14
NVIDIA GPU Grant
![Page 15: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/15.jpg)
Detection: YOLO v3• Detecting locations of pedestrians and vehicles
• Number of objects of each type
15
VIRAT Video Dataset
![Page 16: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/16.jpg)
Tracking: Kalman Filtering• Associating IDs with the detections
• Trajectories
16
VIRAT Video Dataset
![Page 17: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/17.jpg)
Final output
No image Privacy OK!
17
![Page 18: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/18.jpg)
Dashboard
18
![Page 19: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/19.jpg)
DashboardHeatmap of the maximum number of detections
19
![Page 20: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/20.jpg)
DashboardTrajectories of the detections
20
![Page 21: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/21.jpg)
DashboardTrajectories of the detections (inside a building)
21
![Page 22: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/22.jpg)
DashboardTrajectories of the detections (inside a building)
22
![Page 23: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/23.jpg)
Dashboard
23
![Page 24: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/24.jpg)
Next step: inferring network dynamics
Image credit: OpenStreetMap Image credit: Google Maps
24
![Page 25: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/25.jpg)
ConclusionsIt’s only the beginning!
• Jetson TX2 for real time object detection and tracking
• Privacy compliant… but meaningful information
• Open data for people centric approach
– citizen applications
– city and traffic planners
• Scalability and interoperability
• Framework can integrates other sensors
– air quality, noise
Traffic modelling
AnalyticsDashboardCitizens app
25
![Page 26: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/26.jpg)
An ecosystem around the Jetson TX2
26
![Page 27: Edge Computing with Jetson TX2 for Monitoring Flows of ...on-demand.gputechconf.com/gtc-au/pdf/UOW-nvidia_ai... · Sensors Transport IoT Core Applications + Private and Public APIs](https://reader035.vdocument.in/reader035/viewer/2022081523/5fd2f379d3e7295a3f35802e/html5/thumbnails/27.jpg)
Keep In Touch
linkedin.com/company/smart-
infrastructure-facility-university-
of-wollongong
@SMART_Facility
smart.uow.edu.au
uowblogs.com/smartinfrastructureSMART Infrastructure Facility
27