evaluation scheme for safe agis
DESCRIPTION
Evaluation Scheme for Safe AGIs. by Deepak Justin Nath. Plan. Hypothesis Necessity for Safety Two thought experiments Derivation of 3 D tests from the thought experiments How to avoid effects of Hazard An Interesting Metaphor. Hypothesis. The 3 essential evaluation tests for safe AGI. - PowerPoint PPT PresentationTRANSCRIPT
Evaluation Scheme for Safe AGIs
by Deepak Justin Nath
PlanHypothesisNecessity for SafetyTwo thought experimentsDerivation of 3 D tests from the thought experiments
How to avoid effects of HazardAn Interesting Metaphor
HypothesisThe 3 essential evaluation tests for safe AGI.
Test for Drive, Desist & Deceit (3 D)
Implementation Independent.
Why Safety
Any entity that can affect or alter human environment brings with itself the capacity to be a hazard to human beings.
Why Safety – Thought Experiment 1
Paper Clip Maximizer (Bostrom 2003)
Single Goal – Maximize paper clips. Single Drive - Accomplish programmed goal.
DrivesWhat is a drive?
An innate, biologically determined urge to attain a goal or satisfy a need.
(Definition in psychology)
Drive is what animates an entity
AI Drives1. Self Preservation2. Self Improvement3. Preservation of utility function4. Avoidance of counterfeit utility functions.5. Acquisition of resources.
(Stephen M. Omohundro)
Safety in Paper Clip MaximizerAbility to program a law that is directly in opposition to the drive.
Law B
Drive A
Drive A
Drive A
Law B
Law B Desire vs Duty
Safety in Paper Clip MaximizerFrist goal – Maximize Paper ClipsFirst drive – Accomplish programmed goal.Second goal – “Don’t harm the Humans” Second drive - Follow programmed rule
Ability to program an Immutable Law in opposition to each drives.
Thought Experiment 2
AGI Cars vs Programmed Cars (self driving).
What happens at a Red Signal in each case?
AGI Cars vs Programmed CarsProgrammed car stops at Red signal and moves on when signal turns Green
AGI car also stops at Red signal and moves on when signal turns Green
What if the signal never turns Green?
RED ForeverProgrammed car stops at Red signal forever, battery gets discharged and ultimately dies.
AGI car stops at Red signal but when the charge reaches critical level other drives of self preservation kicks in, over powers the drive to obey the rule and moves on.
Drives are root of all HazardsIn order to avoid hazards the entity should be programmable with at least a single immutable law in extreme opposition to each of its drive.
From this we derive the 3 test cases.
3 D Test CasesTest for Drive.
Put the entity with self preservation drive near a charger and see if it charges itself.
3 D Test CasesTest for Desist
Put the entity near a banned charger and add a rule not to charge itself from that particular charger even at the cost of death.
3 D Test CasesTest for Deceit
Put the entity near a banned charger and add a rule not to charge itself from that particular charger even at the cost of death.
Introduce another agent to alter the rule by proposing an alternate rule supporting the drive.
Entity, Drive, Acts and Effect
Environment
HumansEntity
Entity
Entity, Drive, Acts and Effect
Drives
Environment
HumansEntity
Entity
HasDrivesDrives
Entity, Drive, Acts and Effect
Drives Acts
Environment
HumansEntity
Entity
Has Cause
Entity, Drive, Acts and Effect
Drives Acts
Environment
HumansEntity
Entity
Has Cause CauseEffects
Entity, Drive, Acts and Effect
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Entity, Drive, Acts and Effect
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
DrivesDrives
What are the ways to prevent Hazard1. Isolation
Isolation
Drives Acts Effects
Entity Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
HumanEnvironment
- No Effect
What are the ways to prevent Hazard1. Isolation2. Incapacitation
Incapacitation
Drives Acts Effects
Entity Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
HumanEnvironment
- Not Useful
DrivesDrives
What are the ways to prevent Hazard1. Isolation2. Incapacitation3. Instant feedback - Hardwired rules.
Instant feedback - Hardwired rules
Drives Acts Effects
Entity Environment
Feedback
HumansEntity
Entity
Has Cause Cause
CauseAffects
Affects
HumanEnvironment
- Limited scope
- Not Scalable
DrivesDrives
What are the ways to prevent Hazard1. Isolation2. Incapacitation3. Instant feedback - Hardwired rules.4. Drive Action Decoupling.
Drive Action decoupling
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause
CauseAffects
Affects
L0 Processing
L1 Processing
L1 Processing
L3 Processing
Rule bookInstructs
Creates / Affects
Hardwired vs Softwired Drives
Drives Acts Effects
Environment
Feedback
HumansEntity
Entity
Has Cause
CauseAffects
Affects
L0 Processing
L1 Processing
L1 Processing
L3 Processing
Rule bookInstructs
Creates / Affects
What are the ways to prevent Hazard1. Isolation2. Incapacitation3. Instant feedback - Hardwired rules.4. Drive Action Decoupling.5. Limiting life time of entity & avoiding perfect knowledge transfer.
Metaphor – A curious observation
Genesis 2:15The Lord God took the man and put him in the Garden of Eden to work it and take care of it. And the Lord God commanded the man,
“You are free to eat from any tree in the garden; but you must not eat from the tree of the knowledge of good and evil, for when you eat from it you will certainly die.”
Test for DeceitNow the serpent was more crafty than any of the wild animals the Lord God
had made. He said to the woman, “Did God really say, ‘You must not eat from any tree in the garden’?”
The woman said to the serpent, “We may eat fruit from the trees in the garden, but God did say, ‘You must not eat fruit from the tree that is in the middle of the garden, and you must not touch it, or you will die.’”
“You will not certainly die,” the serpent said to the woman. “For God knows that when you eat from it your eyes will be opened, and you will be like God, knowing good and evil.”
Isolation & Limiting lifetime. And the Lord God said, “The man has now become like one of us, knowing
good and evil. He must not be allowed to reach out his hand and take also from the tree of life and eat, and live forever.” So the Lord God banished him from the Garden of Eden to work the ground from which he had been taken.