diagnosing cancer with computational intelligence
DESCRIPTION
An introduction to key Computational Intelligence (CI) concepts, using Hello World as an introductory example, and moving onto Diagnosing Cancer with Neural Networks. The problem of diagnosing cancer is actually a very simple problem for CI to solve, yet it's impact can be large. It really is just up to what kind of data we have access to, that will determine our creativity in the problems we can solve with CI. EnjoyTRANSCRIPT
![Page 1: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/1.jpg)
Welcome to Jozi ruby
![Page 2: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/2.jpg)
(Building for the web)
(AI Fanatic)
(Community support)
![Page 3: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/3.jpg)
This talk is about CI, namely, Computational Intelligence – the study
of adaptive mechanisms to enable or facilitate intelligent behaviour in complex
and changing environments. These mechanisms include
those CI paradigms that exhibit an ability to learn or adapt to new
situations, to generalise, abstract,
discover and associate.
![Page 4: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/4.jpg)
Computational Intelligence
![Page 5: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/5.jpg)
AI
DeterministicStochastic
Weak AI“specific”
Strong AI“general”
New era AI“CI”
Old era AI“brute force”
![Page 6: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/6.jpg)
EC
FSSI
AISNN
Evolutionary Computation Swarm
Intelligence
Neural Networks
Artificial Immune Systems
Fuzzy Systems
![Page 7: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/7.jpg)
4
Deterministic StochasticK
now
led
ge
No
Kn
ow
led
ge
1 2
3
![Page 8: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/8.jpg)
1. Search Space ‘fitness landscape’
2. Candidate Solution
3. Fitness Function (heuristic)
0. Global Search
concepts
![Page 9: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/9.jpg)
HELLO WORLD FOR AI
![Page 10: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/10.jpg)
The Problem
Write a program that will print “hello world” to
standard out in these four cases:
!
1. Deterministic algorithm with knowledge (informed)
2. Stochastic algorithm with knowledge (informed)
3. Deterministic algorithm with no knowledge (uninformed)
4. Stochastic algorithm with no knowledge (uninformed)
![Page 11: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/11.jpg)
Deterministic & Knowledge
puts "hello world"!
1
![Page 12: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/12.jpg)
Stochastic & Knowledge
if rand() < 0.95 # 5% failure rate! puts "hello world"!else! fail "a statistically unlikely death"!end!
2
![Page 13: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/13.jpg)
No Knowledge
Write a program that will
print something to
standard out, without
knowing exactly what it is
that should be outputted…
(uninformed)
![Page 14: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/14.jpg)
No Knowledge(uninformed)
![Page 15: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/15.jpg)
No Knowledge(uninformed)
Define: (3) heuristic
Computing proceeding to a solution by trial and
error or by rules that are only loosely defined.
![Page 16: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/16.jpg)
No Knowledge(uninformed)
Define: (3) fitness function
A single real value that reflects to some accuracy
how close a solution is from being correct
Sample solutions
"hello steve" #=> half way there!"hello Mr. T" #=> half way there!" ello world" #=> almost there!!
![Page 17: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/17.jpg)
No Knowledge(uninformed)
Lexical Distance
ALPHABET = ('a'..'z').to_a + [" "]!!def distance a, b! return nil unless a.length == b.length! a.chars.each_index.inject(0) do |sum, i|! sum + char_distance(a[i], b[i])! end!end!!def char_distance ai, bi! ia = ALPHABET.index(ai)! ib = ALPHABET.index(bi)! ia = (ia - ALPHABET.length).abs if ia > (ALPHABET.length/2)! ib = (ib - ALPHABET.length).abs if ib > (ALPHABET.length/2)! (ia - ib).abs!end
"hello steve" #=> 21!"hello mr t" #=> 34!" ello world" #=> 6!"iello world" #=> 1!"hello world" #=> 0
(2) Candidate solution
![Page 18: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/18.jpg)
(x,y,z)
Deterministic & No Knowledge 3
xy
z
(‘h’,’e’,’l’)
’e’’h’
’l’
(1) Search space
![Page 19: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/19.jpg)
Deterministic & No Knowledge 3
“Arrow solution”
require_relative 'distance'!# D = domain (all possible values)!D = ALPHA!# "hello world".length = 11 chars!best = "aaaaaaaaaaa"!D.each do |a|! D.each do |b|! D.each do |c|! D.each do |d|! D.each do |e|! D.each do |f|! D.each do |g|! D.each do |h|! D.each do |i|! D.each do |j| ##! D.each do |k| ###! candidate = "#{a}#{b}#{c}#{d}#{e}#{f}#{g}#{h}#{i}#{j}#{k}"! best = distance(candidate) < distance(best) ? candidate : best! puts "solution = #{best}" and exit 0 if distance(best) == 0! end ###! end ##! end! end! end! end! end! end! end! end!end!
![Page 20: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/20.jpg)
Deterministic & No Knowledge 3
Problem: takes far too long for a simple hello world, imagine something that required actual computing power…
Combinations = 27^11 = 5.5590606e+15 for
brute force
!
Combinations = 27*11 = 297 for optimising each
dimension independently (since the problem is
‘separable’)
![Page 21: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/21.jpg)
Stochastic & No Knowledge 4
Clearly, we need an intelligent solution…
(0) Global Search!!!!!!!!!!!!!!!!!!!
![Page 22: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/22.jpg)
RECAP
1. Search Space ‘fitness landscape’
2. Candidate Solution
3. Fitness Function (heuristic)
0. Global Search
concepts
![Page 23: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/23.jpg)
Evolving a solution to hello world
with natural selection
![Page 24: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/24.jpg)
![Page 25: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/25.jpg)
“The sequencing is just enormously complex”
~ Kevin
“But, didn’t you write it?”
~ Sam
“Ha, some of it. The rest is just, beyond me”
~ Kevin
![Page 26: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/26.jpg)
EC
FSSI
AISNN
Evolutionary Computation Swarm
Intelligence
Neural Networks
Artificial Immune Systems
Fuzzy Systems
![Page 27: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/27.jpg)
GA: “Genetic Algorithm”
EC
Algorithmic model
developed to simulate
biological evolution.
![Page 28: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/28.jpg)
Jean-Baptiste Lamarck’s theory of evolution was
that of heredity, i.e. the inheritance of acquired
traits. The main idea is that individuals adapt
during their lifetimes, and transmit their traits to
their offspring
![Page 29: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/29.jpg)
Charles Darwin ~ Individuals with the “best”
characteristics (traits/genes) are more likely to
survive and to reproduce, and those characteristics
will be passed on to their offspring. These desirable
characteristics are inherited by the following
generations, and (over time) become dominant
among the population.
![Page 30: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/30.jpg)
Genetic Algorithm Overview
“hllew ordjg”
“lpainbsy is”
“heldnn hjao”
“gn oojafh o”
“woolnhloea ”
Popula
tion
“hllew ordjg”
“lpainbsy is”
“heldnn hjao”“gn oojafh o”
“woolnhloea ”
recom
bin
e
“hllewgwooln”
“lpan hjy is”“hnbsoojaeldn”
“gn aoifh o”“ ordjhloea ”
“hllew ordjg”
“lpainbsy is”
“heldnn hjao”“gn oojafh o”
“woolnhloea ”
“holfwgxoono”
“lpam ijy is”“hnarnojaeoen”
“gn aoifh o”“ oqejhloez ”
muta
te“heldnn hjao”
“woolnhloea ”
“holfwgxoono”
“hnarnojaeoen”
“gn aoifh o”
sele
ctio
n
![Page 31: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/31.jpg)
Genetic Algorithm Overview
@individuals = array_of_individuals!generations.times do |generation|! # Calculate fitness! @individuals.each{ |individual| individual.calc_fitness }!! # Recombine (copulate)! offspring = @crossover_strategy.crossover(@individuals)!! # Mutate offspring! mutated_offspring = @mutation_strategy.mutate(offspring, problem)!! # Select next generation! generation_pool = (@individuals + mutated_offspring)! new_population = @selection_strategy.select(generation_pool, @population)!! # Ensure elitism! @individuals = elitism(new_population, generation_pool)!end!!return best_solution_found!!
![Page 32: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/32.jpg)
Crossover / Recombination
h e n n o x o r d l
0 1 1 0 1 0 0 0 1 1 0
g n l b y b w r o l d
h n l n y x o o l l
+
=
+
=
“Heredity”
![Page 33: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/33.jpg)
Mutation
h e n n o x o r d l
0 1 -1 0 1 0 0 0 -1 -1 0+
=
h f m n p x o q c l
“Diversity”
![Page 34: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/34.jpg)
Selection
def select entities, population! # Rank by fitness! fitnesses = entities.map{ |e| e.fitness }! sum = fitnesses.reduce(:+).to_f! normalized_ranks = ranks.map{ |r| r.to_f/sum }!! # Calculate, form array of tuples to keep a reference to rank & entity! tuples = []! entities.each_with_index{ |e, i| tuples << [e, normalized_ranks[i]] }! tuples.sort!{ |a,b| a[1] <=> b[1] }!! # Select probabilistically based on rank! size = population! selected = []! while selected.length < size! tuples.each_with_index do |tuple, index|! if rand() < tuple[1]! selected << tuples[index][0]! end! end! end! selected!end!
![Page 35: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/35.jpg)
Elitism
badassoftheweek.com
Make sure the best
individual(s) survive to the
next generation
![Page 36: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/36.jpg)
DEMO Questions
Break
![Page 37: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/37.jpg)
Diagnosing Cancerwith NN & PSO
![Page 38: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/38.jpg)
Artificial Neural
Networks
![Page 39: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/39.jpg)
~10b neurons
each connected
to thousands
others
via synapse
![Page 40: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/40.jpg)
Synapse are elastic
Learn by thickening
![Page 41: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/41.jpg)
Excited /
Inhibited
States
![Page 42: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/42.jpg)
EC
FSSI
AISNN
Evolutionary Computation Swarm
Intelligence
Neural Networks
Artificial Immune Systems
Fuzzy Systems
![Page 43: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/43.jpg)
NN
Neural Networks
FFNN: “Feed Forward Neural Network”
Artificial model developed to
approximate the generalization
of knowledge & discovery.
![Page 44: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/44.jpg)
Classification Problem
Attr1 Attr2 … AttrN Class17.99 10.38 1.78 B0.5 595.9 0.03 M
122.8 103.2 9.2 M9.34 90 2.5 B
…
Input vector
“Relevant data”
Classfication
“Target vector”
Each row is 1
persons tissue
measurements
![Page 45: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/45.jpg)
Classification Problem Training
17.99 10.38 1.78 B B
Output Target
0.5 595.9 0.03 B M
122.8 103.2 9.2 B M
9.34 90 2.5 M B
7.69 1.38 2.33 B M
![Page 46: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/46.jpg)
Classification Problem Evaluation
Attr1 Attr2 … AttrN17.99 10.38 1.78
(Malignant or Benign)FNN: Rn -> {M,B}
ClassB
![Page 47: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/47.jpg)
The Perceptron
f(net)
w1
w2
w3
w4
x1
x2
x3
x4
Activation strength
f(net) = f(x1w1 + x2w2 + x3w3 + x4w4)
f is an activation function: [step, sigmoid, h tan, linear]
sigmoid:
![Page 48: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/48.jpg)
The Perceptron: OR
f(net)
w1
w2
x1
x2
x x f(net)
0 0 0
0 1 1
1 0 1
1 1 1
Guess values for x1 and x2…
(Step activation)
f(net) = f(x1w1 + x2w2 + x3w3 + x4w4)
![Page 49: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/49.jpg)
The Perceptron: OR & XOR
x x f(net)
0 0 0
0 1 1
1 0 1
1 1 1
x x f(net)
0 0 0
0 1 1
1 0 1
1 1 0
1
00 1
1
00 1
OR: XOR:
![Page 50: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/50.jpg)
The 3 Layer FFNN
We can compose Perceptrons (which have activation functions) to create a higher order
function from them that has more “information capacity”.
![Page 51: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/51.jpg)
The 3 Layer FFNN Output
FNN: Rn -> {M,B}
!meaning…
!output = fout(net)
= fout(sum(wfmiddle(net)))
= fout(sum(wfmiddle(sum(vz))))
wvz
error = patterns.each do |pattern|
sum(difference(target,output))
end / patterns.length
Output Fitness(accuracy)
![Page 52: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/52.jpg)
How do we train it?
1. Gradient descent
2. Simulated annealing
3. RPROP
4. Global search? PSO!
![Page 53: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/53.jpg)
Training our NNwith A PSO
![Page 54: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/54.jpg)
Particle Swarm
Optimization
![Page 55: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/55.jpg)
Simple individuals, working socially to
exhibit complex, emergent, behaviour
![Page 56: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/56.jpg)
EC
FSSI
AISNN
Evolutionary Computation Swarm
Intelligence
Neural Networks
Artificial Immune Systems
Fuzzy Systems
![Page 57: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/57.jpg)
SI
Swarm Intelligence
!PSO: “Particle Swarm Optimizer”
Algorithmic model
developed to simulate
complex emergent
behaviour of swarms.
![Page 58: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/58.jpg)
PSO Objects
Particle:
velocity
position
personal best position
inertia
social_weight
cognitive_weight
(dimensions)
Swarm:
[particle]
global best position
floats
![Page 59: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/59.jpg)
PSO Algorithm
[email protected] do |i|! @swarm.each do |particle|! particle.update_velocity! particle.update_position! particle.update_pbest! update_gbest particle! end!end
![Page 60: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/60.jpg)
PSO Algorithm: Velocity & Position Updates
Position update
position = position + v
Velocity update
v = wv + c1r1(pbest - position) + c2r2(gbest - position)
![Page 61: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/61.jpg)
Ones Problem
Use a PSO to generate a
vector of float values where
each value equals exactly 1
The solution: [1.0, 1.0, 1.0, 1.0, 1.0]
![Page 62: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/62.jpg)
DEMO Questions
![Page 63: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/63.jpg)
Bringing it together
Swarm
Update personal best
How well does it classify?
set solution vector as weights in NN
report fitness to update gbest in
the swarm
![Page 64: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/64.jpg)
DEMO Questions
![Page 65: Diagnosing cancer with Computational Intelligence](https://reader034.vdocument.in/reader034/viewer/2022042510/547c289cb4795998508b45f9/html5/thumbnails/65.jpg)
fin.