FAIR-SPACE and Future Autonomy
Professor Yang Gao FIET FRAeS, Professor of Space Autonomous Systems
Head of Space Technology for Autonomous & Robotic systems Laboratory (STAR-LAB)
Associate Dean (International), Faculty of Engineering & Physical Sciences
University of Surrey
@Liverpool University, London campus, UK, January 2020
@IEEE-UK RAS2020, Manchester, UK, January 2020
• Underpinning industry-defined challenges, and meeting technological roadmap of international space community.
• Addressing UK priorities in orbital manipulation, planetary surface/subsurface exploration and robot-astronaut coworking.
2
R&D Programme Overview
RESEARCH THEME 1: Sensing & Perception
RESEARCH THEME 2: Mobility & Mechanisms
RESEARCH THEME 3: Autonomy & AI
RESEARCH THEME 4: Human Robot Interaction
RESEARCH THEME 5: System Engineering
USE C
ASES: O
rbital O
peratio
n
USE C
ASES: P
lane
tary Explo
ration
USE C
ASES: R
ob
ot-A
stron
aut In
teraction
Yang Gao, et. al., UK-RAS Network White Paper on Space Robotics & Autonomous Systems, 2018. https://www.ukras.org/wp-content/uploads/2018/10/UK_RAS_wp_Space_080518.pdf
Landscape in “Orbital” Scenario
Credit: ESA
Non-
cooperati
ve target
Credit: NASA
Cooperative target
Free-flying platform
Credit: Tohuku Uni
Pseudo-fixed platform
Credit: NASA
• Many on-orbit applications requiring advanced AI robotics capabilities, in 2025-2035 timeframe.
• UK to position itself now to develop and demonstrate relevant capabilities required.
• Mission focuses envisaged:
o Clean Space (debris removal) o Satellite servicing o On-orbit assembly / deployment o Constellations o Physical / topographical reconfiguration of satellites
for lifetime extension / mission adaptability o Reduction of satellite costs
• Yang Gao, Steven Chien, Review on space robotics: Toward top-level science through space exploration. Science Robotics, 2, eaan5074 (2017). http://robotics.sciencemag.org/content/2/7/eaan5074.full
• Angadh Nanjangud, Peter C. Blacker, Saptarshi Bandyopadhyay, and Yang Gao, "Robotics and AI enabled On-Orbit Operations with Future Generation of Small Satellites" Proceedings of the IEEE, 106 (3), pp. 429-439, 2018, 10.1109/JPROC.2018.2794829.
Orbital formation flying
• Funded by InnovateUK KTP and ESA
• Industry-academia collaboration between NUK (now part of MDA UK) and Surrey-STAR LAB
• Laser based Fine Lateral and Longitudinal System (FLLS) onboard ESA PROBA-3 mission, for in-space demonstration of precision formation flying of two spacecraft 250 m apart at high accuracy of 300 µm
• Lateral system – tracks beam movement
• Longitudinal system – measures laser signal phase
• M. J. Bradshaw, Y. Gao, K. Homewood, “Interpolation methods for tracking spacecraft in ultra-tight formation”, Journal of Astronomical Telescopes, Instruments, and Systems, 5(2), 028003, 2019, doi: 10.1117/1.JATIS.5.2.028003.
• M. J. Bradshaw, Y. Gao, K. Homewood, Fine Lateral and Longitudinal Sensor (FLLS) on-board ESA’s PROBA-3 mission, 68th International Astronautical Congress, Adelaide, 2017
Credit: ESA
Credit: NUK & Surrey-STAR LAB
Key Enabling Technologies
Autonomous orbital rendezvous
• Monocular vision based post estimation technique: low computation, fast speed, high accuracy and adaptation using deep learning.Current TRL 4-5.
• Validated using photorealistic orbit simulator (developed in house, software license available for academic/industry).
• Performance highly ranked in ESA-Stanford Competition: 2nd on real mission dataset and 3rd on synthetic dataset.
• To advance TRL for in-orbit validation (potentially on-board D-Orbit mission launch towards end of 2020).
• Pedro F. Proença & Yang Gao, Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering, Proc. IEEE ICRA 2020, https://arxiv.org/abs/1907.04298• Pedro F. Proença, Yang Gao, "Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty”, Robotics and Autonomous Systems, 10 March
2018, doi.org/10.1016/j.robot.2018.02.018.
Credit: Surrey-STAR LAB
Key Enabling Technologies
Autonomous orbital manipulation & grasping
• Sparse point-cloud sensing and visual guided GNC -low computation, fast speed, noise and uncertainty resilient.
• Validated in orbital testbeds (digital & physical). TRL 4.
• Extending UK industry capability in collaboration with Shadow Robotics. Exhibition demo with Shadow Hand in Innovation Zoom at UKSC 2019.
• Academic collaborations with Liverpool/Warwick (V&V/Security for the GNC algorithms).
Credit: Surrey-STAR LAB
• Nikos Mavrakis and Yang Gao, Visually Guided Robot Grasping of a Spacecraft's Apogee Kick Motor, Proc. ESA ASTRA Conference, Noordwijk, Netherlands, 2019• Zhou Hao, Nikos Mavrakis, Pedro Proenca, Richard Gillham Darnley, Saber Fallah, and Yang Gao, Ground-Based High-DOF AI And Robotics Demonstrator for In-Orbit Space
Optical Telescope Assembly, IAC-19-C2.3.11, Proc. International Astronautical Conference, Washington DC, USA, October 2019.
Key Enabling Technologies
Landscape in “Planetary” Scenario
Manufacturing, assembling (2030+):Heterogenous robotic elements building infrastructure.
Sample Return (2020+)Sample fetch rover exploring an unstructured terrain and collecting subsurface samples.
In-Situ Resource Utilization, Moon Village, Commercial Lunar Exploration (2030+)Human-robot eco-system around south pole of the Moon
Credit: NASA
Credit: ESA Credit: EU
• Yang Gao, Steven Chien, Review on space robotics: Toward top-level science through space exploration. Science Robotics, 2, eaan5074 (2017). http://robotics.sciencemag.org/content/2/7/eaan5074.full
• Yang Gao, (Ed.) Contemporary Planetary Robotics – An Approach to Autonomous Systems, pp. 1-450, Berlin: Wiley-VCH, ISBN-10: 3527413251, ISBN-13: 978-3527413256, August 2016.
Autonomous robotic visual GNC for sample manipulation
• Funded by UKSA NSTP-2 programme
• Industry-academia collaboration between Airbus-UK GNC team and Surrey-STAR LAB
• Visual saliency based approach to address robust feature tracking on homogenous scenes in space, such as rocky terrain.
• The AI algorithm needs to perform good trade-offs between computation load and detection accuracy/precision.
• Applicable to sample return missions (such as MSR, PSR) for high-precision & accuracy placement of robotic arm end-effector for sample collection.
• Anton Donchev, Calum Murray, Yang Gao, Affan Shuakat, Wissam Albukhanajer, Daisy Lachat, “Vision-Based Accurate Planetary Robotic Arm Placement”, Proceedings of 14th Symposium on Advanced Space Technologies in Robotics and Automation (ASTRA), ESA/ESTEC, Noordwijk, the Netherlands, 2017.
• Affan Shaukat, Said Al-Milli, Abhinav Bajpai, Conrad Spiteri, Guy Burroughes, Yang Gao, Daisy Lachat, and Matthias Winter, "Next-Generation Rover GNC Architectures," Proceedings of 13th Symposium on Advanced Space Technologies in Robotics and Automation (ASTRA), ESA/ESTEC, Noordwijk, the Netherlands, 11-13 May 2015.
Credit: Surrey-STAR LAB
Key Enabling Technologies
Autonomous big planetary data analytics
• Funded by Royal Academy of Engineering
• International collaboration between Chinese Academy of Sciences–National Observatory and Surrey-STAR LAB
• Chang’E3 Yutu rover PanCam data deep analytics to assess lunar soil properties and to complement for lack of in-situ soil measurement.
• Photoclinometry (Shape from Shading) to determine the Digital Terrain Model (DTM), depth of the Yutu rover tracks or extracted using visual ques, combined with terramechanics models and machine learning to estimate soil properties.
• Enabled comparison with Apollo and Luna landing sites.
• Yang Gao, Conrad Spiteri, Chun-Lai Li and Yong-Chun Zheng, “Lunar Soil Strength Estimation based on Chang’E3 Images”, Advances in Space Research, 2016, doi: 10.1016/j.asr.2016.07.017.• Conrad Spiteri, Yang Gao, Said Al-Milli, and Aridane Sarrionandia de León, Real-time Visual Sinkage Detection for Planetary Rovers, Robotics and Autonomous Systems, Vol 72, pp. 307–317,
2015.
Credit: Surrey-STAR LAB
Resulting DTM from SfS of selected area
0.998
1.161
3.076
0 0.5 1 1.5 2 2.5 3 3.5
Luna 17
CE-3
Apollo 15
Stiffness Modulus k in N/cm3.19
Soil stiffness at various Lunar sites
Terramechanics:BG model: ρ=kzn
Small wheel model: ρ=kznDm
Key Enabling Technologies
Credit: Surrey-STAR LAB
Autonomous surface mobility
▪ Active suspension design to allow crawling and climbing behaviours.
▪ 4- wheeled active rover chassis to improve the crossing capabilities over rough terrain and loose soil with a minimal amount of actuation (or comparable to rocker-bogie design with passive suspension).
▪ Physical prototype constructed. Initial validation is performed under digital simulations. TRL 4.
▪ Deep reinforcement learning enabled GNC algorithms that can learn traversability features and automatically tune the GNC parameters.
▪ Industry collaboration on lunar ISRU
Key Enabling Technologies
• Mohamed Alkalla, Yang Gao, Arthur Bouton Customizable and Optimized Drill Bits Bio–inspired from Wood–Wasp Ovipositor Morphology for Extra-terrestrial Surfaces, to appear, Proc. IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Hong Kong, July 2019. Finalist of Best Paper Award.
• Craig Pitcher, Norbert Kömle, Otto Leibniz, Odalys Morales-Calderon, Yang Gao, and Lutz Richter. "Investigation of the properties of icy lunar polar regolith simulants." Advances in Space Research, Volume 57, Issue 5, Pages 1197–1208, 2016, doi:10.1016/j.asr.2015.12.030.
• Craig Pitcher and Yang Gao, First implementation of burrowing motions in dual-reciprocating drilling using an integrated actuation mechanism, Advances in Space Research, Volume 59, Issue 5, p. 1368-1380, 2017, doi:10.1016/j.asr.2016.12.017
Credit: Surrey STAR LAB
Robotic subsurface mobility
▪ 3rd generation “wasp drill” and sampler based on bio-inspired drilling mechanism – light weight and low power for micro-gravity environment.
▪ Built on international collaboration with OHB and Austrian Academy of Science through ESA funded work on lunar simulants and Lunar Generic Regolith Acquisition/ Sampling Paw (L-GRASP)
▪ Physical prototypes are under development. TRL 4.
▪ Commercial collaboration with British Telecom (BT) to support Fibre to the Premises (FTTP) programme.
▪ Supporting ESA Sample Analogue Curation Facility at ECSAT.
Key Enabling Technologies
In the near-medium term, to address key challenges imposed by space environments and spacecraft design constraints:• Low-computation, high-accuracy 3D mapping &
perception• Energy-optimized locomotion mechanisms & control• Resource-aware computation, & data assimilation for
parameter tuning • Hardware/software reconfiguration & self-verification in
real time
In the long term, to achieve long-lived, robust mobility & autonomy for next-generation spacecraft.
Looking into the future:
AI Robotics enabling sustainable in-space operations
Deliberative
Executive
Functional
Planner `̀
Environment
High Level Goals
Plan Exceptions
Plan
Execution Commands
Exceptions, Status
Actions (Actuators)
Observations(Sensors)
Autonomous System Architecture
Hierarchical Task Network
Planning
Problem Description
(PDDL)
Knowledge Base(Methods)
Domain model(Rover or S/C
models)
Planner( Continuous planning)
Status Information
Plan Library
* Reconfigurable autonomy, including domain-independent generic and reusable autonomous software architecture based on rational agents for complex space systems, such as multi-satellite and multi-rover scenario.
* Reliable machine learning and planning software agents.
* Ontology based modelling.
• Guy Burroughes and Yang Gao, "Ontology-Based Self-Reconfiguring Guidance, Navigation, and Control for Planetary Rovers". AIAA Journal of Aerospace Information Systems, Vol. 13, No. 8, pp. 316-328, 2016, doi: 10.2514/1.I010378.
• Affan Shaukat, Guy Burroughes, and Yang Gao, "Self-Reconfigurable Robotics Architecture Utilizing Fuzzy and Deliberative Reasoning," Proceedings of SAI Intelligent Systems Conference, London, UK, 10-11 November 2015.
For example: Moving from “Weak AI” to “Strong AI”
Thank You
Web: https://www.surrey.ac.uk/surrey-space-centre/research-groups/star-lab
Youtube: https://www.youtube.com/user/SpaceAutonomy/