immune1.ppt
TRANSCRIPT
![Page 1: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/1.jpg)
Artificial Immune Systems
Andrew Watkins
![Page 2: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/2.jpg)
Why the Immune System?
• Recognition– Anomaly detection– Noise tolerance
• Robustness• Feature extraction• Diversity• Reinforcement learning• Memory• Distributed• Multi-layered• Adaptive
![Page 3: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/3.jpg)
Definition
AIS are adaptive systems inspired by theoretical immunology and observed
immune functions, principles and models, which are applied to complex problem
domains
(de Castro and Timmis)
![Page 4: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/4.jpg)
Some History
• Developed from the field of theoretical immunology in the mid 1980’s.– Suggested we ‘might look’ at the IS
• 1990 – Bersini first use of immune algos to solve problems
• Forrest et al – Computer Security mid 1990’s
• Hunt et al, mid 1990’s – Machine learning
![Page 5: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/5.jpg)
How does it work?
![Page 6: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/6.jpg)
Immune Pattern Recognition
• The immune recognition is based on the complementarity between the binding region of the receptor and a portion of the antigen called epitope.
• Antibodies present a single type of receptor, antigens might present several epitopes.– This means that different antibodies can recognize a single
antigen
![Page 7: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/7.jpg)
Immune Responses
Antigen Ag1 Antigens Ag1, Ag2
Primary Response Secondary Response
Lag
Response to Ag1
Ant
ibod
y C
once
ntra
tion
Time
Lag
Response to Ag2
Response to Ag1
...
...
Cross-Reactive Response
...
...
Antigen Ag1 + Ag3
Response to Ag1 + Ag3
Lag
![Page 8: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/8.jpg)
Clonal Selection
![Page 9: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/9.jpg)
Immune Network Theory
• Idiotypic network (Jerne, 1974)
• B cells co-stimulate each other– Treat each other a bit like antigens
• Creates an immunological memory
![Page 10: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/10.jpg)
Shape Space Formalism
• Repertoire of the immune system is complete (Perelson, 1989)
• Extensive regions of complementarity
• Some threshold of recognition
V
V
V
V
![Page 11: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/11.jpg)
Self/Non-Self Recognition
• Immune system needs to be able to differentiate between self and non-self cells
• Antigenic encounters may result in cell death, therefore– Some kind of positive selection– Some element of negative selection
![Page 12: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/12.jpg)
General Framework for AIS
Application Domain
Representation
Affinity Measures
Immune Algorithms
Solution
![Page 13: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/13.jpg)
Representation – Shape Space
• Describe the general shape of a molecule
•Describe interactions between molecules
•Degree of binding between molecules
•Complement threshold
![Page 14: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/14.jpg)
Define their Interaction
• Define the term Affinity• Affinity is related to distance
– Euclidian
L
iii AgAbD
1
2)(
• Other distance measures such as Hamming, Manhattan etc. etc.
• Affinity Threshold
![Page 15: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/15.jpg)
Basic Immune Models and Algorithms
• Bone Marrow Models
• Negative Selection Algorithms
• Clonal Selection Algorithm
• Somatic Hypermutation
• Immune Network Models
![Page 16: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/16.jpg)
Bone Marrow Models• Gene libraries are used to create antibodies from the
bone marrow• Use this idea to generate attribute strings that represent
receptors• Antibody production through a random concatenation
from gene libraries
![Page 17: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/17.jpg)
Negative Selection Algorithms
• Forrest 1994: Idea taken from the negative selection of T-cells in the thymus
• Applied initially to computer security• Split into two parts:
– Censoring– Monitoring
![Page 18: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/18.jpg)
Clonal Selection Algorithm (de Castro & von Zuben, 2001)
Randomly initialise a population (P)For each pattern in Ag
Determine affinity to each Ab in PSelect n highest affinity from P
Clone and mutate prop. to affinity with Ag
Add new mutants to P endForSelect highest affinity Ab in P to form part of MReplace n number of random new ones
Until stopping criteria
![Page 19: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/19.jpg)
Immune Network Models (Timmis & Neal, 2001)
Initialise the immune network (P)
For each pattern in Ag
Determine affinity to each Ab in P
Calculate network interaction
Allocate resources to the strongest members of P
Remove weakest Ab in P
EndFor
If termination condition met
exit
else
Clone and mutate each Ab in P (based on a given probability)
Integrate new mutants into P based on affinity
Repeat
![Page 20: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/20.jpg)
Somatic Hypermutation
• Mutation rate in proportion to affinity• Very controlled mutation in the natural immune
system• The greater the antibody affinity the smaller its
mutation rate • Classic trade-off between exploration and
exploitation
![Page 21: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/21.jpg)
How do AIS Compare?
• Basic Components:– AIS B-cell in shape space (e.g. attribute
strings)• Stimulation level
– ANN Neuron• Activation function
– GA chromosome• fitness
![Page 22: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/22.jpg)
Comparing
• Structure (Architecture)– AIS and GA fixed or variable sized
populations, not connected in population based AIS
– ANN and AIS• Do have network based AIS
• ANN typically fixed structure (not always)
• Learning takes place in weights in ANN
![Page 23: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/23.jpg)
Comparing
• Memory– AIS in B-cells
• Network models in connections
– ANN In weights of connections– GA individual chromosome
![Page 24: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/24.jpg)
Comparing
• Adaptation
• Dynamics
• Metadynamics
• Interactions
• Generalisation capabilities
• Etc. many more.
![Page 25: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/25.jpg)
Where are they used?
• Dependable systems
• Scheduling
• Robotics
• Security
• Anomaly detection
• Learning systems
![Page 26: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/26.jpg)
Artificial Immune Recognition System (AIRS):
An Immune-Inspired Supervised Learning Algorithm
![Page 27: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/27.jpg)
AIRS: Immune Principles Employed
• Clonal Selection
• Based initially on immune networks, though found this did not work
• Somatic hypermutation – Eventually
• Recognition regions within shape space
• Antibody/antigen binding
![Page 28: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/28.jpg)
AIRS: Mapping from IS to AIS
• Antibody Feature Vector• Recognition Combination of feature Ball
(RB) vector and vector class• Antigens Training Data• Immune Memory Memory cells—set of
mutated Artificial RBs
![Page 29: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/29.jpg)
Classification• Stimulation of an ARB is based not only on its
affinity to an antigen but also on its class when compared to the class of an antigen
• Allocation of resources to the ARBs also takes into account the ARBs’ classifications when compared to the class of the antigen
• Memory cell hyper-mutation and replacement is based primarily on classification and secondarily on affinity
![Page 30: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/30.jpg)
AIRS Algorithm• Data normalization and initialization• Memory cell identification and ARB
generation• Competition for resources in the
development of a candidate memory cell• Potential introduction of the candidate
memory cell into the set of established memory cells
![Page 31: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/31.jpg)
Memory Cell IdentificationMemory Cell PoolA
ARB Pool
![Page 32: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/32.jpg)
MCmatch FoundMemory Cell PoolA 1
ARB Pool
MCmatch
![Page 33: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/33.jpg)
ARB GenerationMemory Cell PoolA 1
ARB Pool
2
MCmatch
Mutated Offspring
![Page 34: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/34.jpg)
Exposure of ARBs to AntigenMemory Cell PoolA 1
ARB Pool
2
MCmatch
Mutated Offspring3
![Page 35: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/35.jpg)
Development of a Candidate Memory Cell
Memory Cell PoolA 1
ARB Pool
2
MCmatch
Mutated Offspring3
![Page 36: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/36.jpg)
Comparison of MCcandidate and MCmatch
Memory Cell PoolA 1
ARB Pool
2
MCmatch
Mutated Offspring3
MC candidate
4 A
![Page 37: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/37.jpg)
Memory Cell IntroductionMemory Cell PoolA 1
ARB Pool
2
MCmatch
Mutated Offspring3
MCcandidate
45
A
![Page 38: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/38.jpg)
Memory Cells and Antigens
![Page 39: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/39.jpg)
Memory Cells and Antigens
![Page 40: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/40.jpg)
Fisher’s Iris Data SetPima Indians Diabetes
Data Set
Ionosphere Data Set Sonar Data Set
AIRS: Performance Evaluation
![Page 41: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/41.jpg)
Iris Ionosphere Diabetes Sonar
1 Grobian (rough)
100% 3-NN + simplex
98.7% Logdisc 77.7% TAP MFT Bayesian
92.3%
2 SSV 98.0% 3-NN 96.7% IncNet 77.6% Naïve MFT Bayesian 90.4%
3 C-MLP2LN 98.0% IB3 96.7% DIPOL92 77.6% SVM 90.4%
4 PVM 2 rules 98.0% MLP + BP 96.0% Linear Discr. Anal. 77.5%-77.2%
Best 2-layer MLP + BP, 12 hidden
90.4%
5 PVM 1 rule 97.3% AIRS 94.9 SMART 76.8% MLP+BP, 12 hidden 84.7%
6 AIRS 96.7 C4.5 94.9% GTO DT (5xCV) 76.8% MLP+BP, 24 hidden 84.5%
7 FuNe-I 96.7% RIAC 94.6% ASI 76.6% 1-NN, Manhatten 84.2%
8 NEFCLASS 96.7% SVM 93.2% Fischer discr. anal 76.5% AIRS 84.0 9 CART 96.0% Non-linear
perceptron 92.0% MLP+BP 76.4% MLP+BP, 6
hidden 83.5%
10 FUNN 95.7% FSM + rotation
92.8% LVQ 75.8% FSM - methodology?
83.6%
11 1-NN 92.1% LFC 75.8% 1-NN Euclidean 82.2%
12 DB-CART 91.3% RBF 75.7% DB-CART, 10xCV 81.8%
13 Linear perceptron
90.7% NB 75.5-73.8%
CART, 10xCV 67.9%
14 OC1 DT 89.5% kNN, k=22, Manh 75.5%
15 CART 88.9% MML 75.5%
… . . .
22 AIRS 74.1
23 C4.5 73.0%
11 others reported with lower scores, including Bayes, Kohonen, kNN, ID3 …
![Page 42: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/42.jpg)
AIRS: Observations
• ARB Pool formulation was over complicated – Crude visualization– Memory only needs to be maintained in the
Memory Cell Pool
• Mutation Routine– Difference in Quality– Some redundancy
![Page 43: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/43.jpg)
AIRS: Revisions
• Memory Cell Evolution– Only Memory Cell Pool has different classes– ARB Pool only concerned with evolving
memory cells
• Somatic Hypermutation– Cell’s stimulation value indicates range of
mutation possibilities– No longer need to mutate class
![Page 44: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/44.jpg)
Comparisons: Classification Accuracy
• Important to maintain accuracy AIRS1: Accuracy AIRS2: Accuracy
Iris 96.7 96.0
Ionosphere 94.9 95.6
Diabetes 74.1 74.2
Sonar 84.0 84.9
• Why bother?
![Page 45: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/45.jpg)
Comparisons: Data Reduction
• Increase data reduction—increased efficiency
Training Set Size AIRS1: Memory Cells AIRS2: Memory Cells
Iris 120 42.1 / 65% 30.9 / 74%
Ionosphere 200 140.7 / 30% 96.3 / 52%
Diabetes 691 470.4 / 32% 273.4 / 60%
Sonar 192 144.6 / 25% 177.7 / 7%
![Page 46: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/46.jpg)
Features of AIRS
• No need to know best architecture to get good results
• Default settings within a few percent of the best it can get
• User-adjustable parameters optimize performance for a given problem set
• Generalization and data reduction
![Page 47: immune1.ppt](https://reader035.vdocument.in/reader035/viewer/2022062514/558e186a1a28aba0788b4781/html5/thumbnails/47.jpg)
More Information
• http://www.cs.ukc.ac.uk/people/rpg/abw5
• http://www.cs.ukc.ac.uk/people/staff/jt6
• http://www.cs.ukc.ac.uk/aisbook