reinforcement learning based adpncr.mae.ufl.edu/aa/files/2019kickoff/nfc_rt2.pdf · 2019. 5....
TRANSCRIPT
-
Reinforcement Learning Based ADP:
Computational reductions, faster learning, and more complex problems
– An Application
-
Space is Becoming Congested
“We will be using proliferated LEO,” Lt. Gen. David Thompson told a New America Foundation conference today in response to my question. “It is simply a matter of for what missions.”
Excerpted from “LEO and MEO broadband constellations mega source of consternation,” a 2018 SpaceNews article by Caleb Henry
https://breakingdefense.com/2019/04/thompson-afspc-will-use-mega-constellations/
https://breakingdefense.com/2019/04/thompson-afspc-will-use-mega-constellations/
-
Challenges
• Threats• Cyber • Physical (i.e., debris)
• Sources of space debris• Abandoned upper stages and satellites• Collisions, ASAT demos, fragmentation
• Space debris population • Baseball size & larger (≥ 10 cm) ~23K• Marble size (≥ 1 cm) ~500K• Dot size mall (≥ 1 mm) ~100M
• Population growth projected• Mega constellations significant SSA/space traffic management challenge
• OneWeb: > 600 satellites (w/ Airbus)• SpaceX Starlink: > 4,000 satellites• Samsung: > 4,000 satellites• Others
• Kessler Syndrome additional debris created from collisions with debris
Reference: http://orbitaldebris.jsc.nasa.gov
Small (mm-to-cm) sized debris dominate mission ending threat
http://orbitaldebris.jsc.nasa.gov/
-
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
12000
13000
14000
15000
16000
1956
1958
1960
1962
1964
1966
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
Nu
mb
er o
f O
bje
cts
Year
Monthly Number of Objects in Earth Orbit by Object Type (US Satellite Catalog)
Total Objects
Fragmentation Debris
Spacecraft
Mission-related Debris
Rocket Bodies Iridium-Cosmos
The Debris Problem
FY-1C ASAT Test
0
100
200
300
400
500
600
700
800
200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000
Eff
ec
tive
Nu
mb
er o
f O
bje
cts
pe
r 2
0 k
m A
lt B
in
Altitude (km)
1-Apr-10 SSN catalog
1-Jan-07 SSN catalog
Chinese ASAT resulted in ~120% increase in the region below 1000 km
-
The Debris Problem
• Observed and predicted A/M distribution of Cosmos 2251 fragments matches well
0
0.05
0.1
0.15
0.2
0.25
-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
Nu
mb
er
(No
rmal
ize
d)
Log10(A/M m2/kg)
A/M Distribution of Cosmos 2251 Fragments
NASA Breakup Model Prediction
SATCAT Data (17 September 2010)
• Observed A/M distribution of Iridium 33 fragments is higher than prediction; ~3 times higher)
• Possible source of difference: lightweight materials (e.g., MLI, CFRP) used in the newer vehicle 0
20
40
60
80
100
120
-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
Nu
mb
er
Log10(A/M m2/kg)
A/M Distribution of Iridium 33 Fragments
NASA Breakup Model Prediction
SATCAT Data (17 September 2010)
Iridium 33(560 kg)
Cosmos 2251(900 kg)
0
20
40
60
80
100
120
-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2
Nu
mb
er
Log10(A/M m2/kg)
A/M Distribution of Iridium 33 Fragments
NASA Breakup Model Prediction
SATCAT Data, A/M/3 (17 September 2010)
-
DebriSat Laboratory HVI Test
MIRO310(14886)(MirrorCorrection).avi
-
Results
• Preliminary outcomes of the HVI test• Developed a unique database with ~200K fragments (> 250K projected – x3?)• Imaging systems developed to characterize fragments• Models developed to compute fragments’ (i) characteristic length (ii) average cross-
sectional area, (iii) volume
➢Task: Utilize center’s modelling tools to develop models to better characterize the debris environment and facilitate future space traffic management / SSA capabilities in congested and/or contested environments (e.g., COLA maneuvers)
Current models predicted 85K fragments
Recorded Collected
-
0
10000
20000
30000
40000
50000
60000
70000
1950 1970 1990 2010 2030 2050 2070 2090 2110 2130 2150 2170 2190 2210
Eff
ec
tive
Nu
mb
er o
f O
bje
cts
(>
10
cm
)
Year
Non-Mitigation Projection (averages and 1-σ from 100 MC runs)
LEO (200-2000 km alt)
MEO (2000-35,586 km alt)
GEO (35,586-35,986 km alt)
The Debris Problem
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
1950 1970 1990 2010 2030 2050 2070 2090 2110 2130 2150 2170 2190 2210
Eff
ec
tive
Nu
mb
er o
f O
bje
cts
(>
10
cm
)
Year
LEO Environment Projection (averages of 100 LEGEND MC runs)
Reg Launches + 90% PMD
Reg Launches + 90% PMD + ADR2020/02
Reg Launches + 90% PMD + ADR2020/05
• PMD scenario predicts the LEO populations would increase by ~75% in 200 years
• LEO environment can be stabilized with PMD and a removal rate of ~5 objects/year
Average collisions in the next 200 years(non-mitigation scenario)
Cat. Non-cat. Total
LEO 83 95 178
MEO 0.5 1.5 2
GEO 1.5 1.5 3
-
ADR Approaches
Necropolis Concept (European)• A two spacecraft system• A “Hunter” that hunts, collects, and transports spent
GEO spacecraft and a “Terminus” for final storage
RemoveDEBRIS Mission• On-orbit demo by Surrey Space Centre for active
debris removal (ADR)• Successfully demonstrated (i) on-orbit debris
identification via vision-based navigation (ii) “debris” capture via tethered net and harpoon, and (iii) deorbiting via drag sail
Others:spacecraft• GLiDeR – electrostatic tug (Hanspeter Schaub)• Tether-Nets for ADR (Botta, Sharf, Misra)• ESA’s Clean Space Initiative (E.DeOrbit mission)
http://www.esa.int/Our_Activities/Space_Safety/Clean_Space/ESA_s_e.Deorbit_debris_removal_mission_reborn_as_servicing_vehicle
http://www.esa.int/Our_Activities/Space_Safety/Clean_Space/ESA_s_e.Deorbit_debris_removal_mission_reborn_as_servicing_vehicle
-
ADR Approaches
Active Debris Removal (ADR)
Contact Noncontact
Cooperative Non-cooperative
• Orbital Express (DARPA)
• SUMO/FREND (DARPA ARD)
• RemoveDEBRIS
• Risky (nets, harpoons) • Needs investigation➢ Game-based
(minimax, Stackelberg, Nash)
• GLiDeR (and others)• Necropolis Hunter• Herding approaches➢ Game-based
(minimax, Stackelberg, Nash)
Task: Design strategies to address interactions
(contact/noncontact) with non-cooperative RSOs
-
Two-Person Differential Games
Two-person differential game
• Minimax: zero sum, cooperative
• Stackelberg: non-zero sum, leader-follower
• Nash: non-zero sum, non-cooperative game
Multiple and/or no solution
(e.g., Prisoner’s Dilemma)
-
Summary
Debris abatement includes removal for disposal (or repair) – need to:• Minimize interactions between vehicles (space tug and
disabled RSO) • Navigate congested environment while minimizing
likelihood of collision with other RSO (herding)
Minimax, Stackelberg, Nash strategies
-
• The National Space Traffic Management Policy (SPD-3), includes the following:
– Space Situational Awareness (SSA) shall mean the knowledge and characterization of space objects and their operational environment to support safe, stable, and sustainable space activities.
– Space Traffic Management (STM) shall mean the planning, coordination, and on-orbit synchronization of activities to enhance the safety, stability, and sustainability of operations in the space environment.
– Orbital debris (OD), or space debris, shall mean any human-made space object orbiting Earth that no longer serves any useful purpose.
– Advance SSA and STM Science and Technology. The United States should continue to engage in and enable S&T research and development to support the practical applications of SSA and STM. These activities include improving fundamental knowledge of the space environment, such as the characterization of small debris, advancing the S&T of critical SSA inputs such as observational data, algorithms, and models necessary to improve SSA capabilities, and developing new hardware and software to support data processing and observations.
Conclusion
https://www.whitehouse.gov/presidential-actions/space-policy-directive-3-national-space-traffic-management-policy/
https://www.whitehouse.gov/presidential-actions/space-policy-directive-3-national-space-traffic-management-policy/