genetic algorithms more applications - sjsukhuri/aalto_2017/aalto_more... · –fuzzy sets –logic...

1©2017 Sami Khuri

Aalto UniversitySummer 2017

Genetic AlgorithmsMore Applications

Sami KhuriComputer Science Department

San José State University©2017 Sami Khuri

Soft Computing Techniques

Evolutionaryalgorithms

Neuro--computing

Roughsets

Uncertainvariables

Probabilistictechniques

Softcomputing

Fuzzylogic

©2017 Sami Khuripritisajja.info/Lecture%202.ppsm

Additional Topics

• Craniofacial Superimposition in Forensic Science–References

• Rough Sets–Attribute Reduction in Rough Sets–References

©2017 Sami Khuri

Craniofacial Superimposition• Craniofacial Superimposition in Forensic

Identification is a process that aims to identify a person by overlaying a photograph and a model of the skull.

• Photographic supra-projection is a forensic process where photographs or video shots of a missing person are compared with the skull that is found.

• By projecting both photographs on top of each other (or matching a scanned three-dimensional skull model against the face photo/video shot), the forensic anthropologist can try to establish whether that is the same person.

7th ANNUAL (2010) “HUMIES” AWARDS Sergio Damas

©2012 Sami Khuri

Human Identification• Human identification (of living or dead

people) is one of the outstanding research areas in forensic medicine.

• If anthropologists get enough information, DNA-based techniques, such as sequencing, fingerprinting, and performing autopsies might be applied.

• What do we do in the absence of this kind of information?


©2012 Sami Khuri

Forensic AnthropologyA solution in the absence of information: Perform Skeleton-Based Human Identification (Forensic Anthropology)

Craniofacial superimpositionis a forensic process where photographs or video shots of a missing person are compared with “a model” of a skull that is found

Projecting one above theother (skull-face overlay)the anthropologist cantry to determine whetherthat is the same person


©2012 Sami Khuri

2©2017 Sami Khuri


Manual Craniofacial Superimposition


©2012 Sami Khuri

• The skull-face overlay is formulated as a 3D/2D image registration problem that aims to determine the best 3D/2D geometric transformation projecting the 3D skull into the 2D photograph

3D/2D Image Registration


• It is determined by 12 parameters that translate, rotate, scale, and project the 3D skull landmarks to reach the location of the 2D landmarks in the photograph

©2012 Sami Khuri

Image Registration Optimization Problem


IR methods usually require thefollowing 4 components:– Two input 2D or 3D images: Scene and Model. Both are vectors of points.– A Registration transformation f, a parametric function relating both images.– A Similarity metric function F, used to measure a qualitative value of closeness or “degree of fitting” between the scene and the model images.– An Optimizer which looks for the optimal transformation f within the defined solution search space.

The key idea of the IR process is to achieve the transformation that places different 2D/3D images in a common coordinate system.

©2012 Sami Khuri

Image Registration Optimization Problem

Manual Genetic Algorithm

7th ANNUAL (2010) “HUMIES” AWARDS Sergio Damas©2012 Sami Khuri

References• L. Ballerini, O. Cordon, S. Damas, J. Santamaria, I.

Aleman, M. Botella, Craniofacial Superimposition in Forensic Identification using Genetic Algorithms. Proceedings of the Third International Symposium on Information Assurance and Security, Pages 429-434, 2007.

• Nickerson, B. A., Fitzhorn, P. A., Koch, S. K., Charney, M. A methodology for near-optimal computational superimposition of two-dimensional digital facial photographs and three-dimensional cranial surface meshes. Journal of Forensic Sciences 36, 2 (March), Pages 480-500, 1991.

©2012 Sami Khuri

Rough Sets

Attribute Reduction in Rough Sets

Notes adapted from Stefan Enroth from LCB, Sweden ©2012 Sami Khuri

3©2017 Sami Khuri


Dealing with Uncertainty• There are quite a few fields that deal with uncertainty:

– Probability theory– Information theory– Evidence theory– Fuzzy sets– Logic– Neural networks

… and also– Rough sets

©2012 Sami Khuri

Main Points of Rough Sets• Information systems• Indiscernibility relation• Decision systems• Rough set approximations• Reducts• Decision rules• Discretization

©2012 Sami Khuri


©2012 Sami Khuri

Information SystemConsider the following Information System, I:I=(U, A), where:

• Universe:– U={x1, x2,…, xn}– xi∈ U are called objects

• Conditional Attributes:– A={a1, a2,…, ap}– a : U à Va– Va : the value of a

©2012 Sami Khuri

Synthesizing RNA

©2012 Sami Khuri

Conditional Attributes

©2012 Sami Khuri

4©2017 Sami Khuri


Main Points of Rough Sets• Information systems• Indiscernibility relation• Decision systems• Rough set approximations• Reducts • Decision rules• Discretization

©2012 Sami Khuri

Equivalence Relation

• Equivalence relationA binary relation which is

– reflexive (xRx for any object x) ,– symmetric (if xRy then yRx), and– transitive (if xRy and yRz then xRz).

• The equivalence class of an element consists of all objects

such that xRy.

XXR ´Í

XxÎ XyÎRx][

©2012 Sami Khuri

The Indiscernibility Relation (I)

• Let B ⊆ A be a subset of attributes• Recall:

– A={a1, a2,…, ap} is the set of attributes– a : U à Va where Va is the value of attribute a

• The indiscernibility relation:INDI(B) = {(x, x´) ∈ U2 | ∀a∈B: a(x)=a(x´)}

• The indescernibility relation induces a partition of the universe.

©2012 Sami Khuri

The Indiscernibility Relation (II)

• In other words, if I = (U, A) is an information system, and , then there is an associated equivalence relation:

where is called the B-indiscernibilityrelation.

• If then objects x and x’ are indiscernible from each other by attributes from B.

• The equivalence classes of the B-indiscernibilityrelation are denoted by

ABÍ

)}'()(,|)',{()( 2 xaxaBaUxxBINDI =Î"Î=)(BINDI

),()',( BINDxx IÎ

.][ Bx©2012 Sami Khuri

Indescernibility: Examples (I)The partition induced byINDI ({Gene1}) = {{x1, x6, x7, x8, x10, x15, x16 },

{x2 , x3, x4, x5, x9, x11, x12, x13, x14, x17, x18 }}

The partition induced byINDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18 },

{x5, x11, x12, x17 }, {x7, x8, x15 }, {x9 }, {x10, x16 }, {x14 }}

©2012 Sami Khuri

Indescernibility: Examples (II)The partition induced byINDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18 },

{x5, x11, x12, x17 }, {x7, x8, x15 }, {x9 }, {x10, x16 }, {x14 }}

And

The partition induced byINDI ({Gene1, Gene2, Gene3, Smoking}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18 },

{x5, x11, x12, x17 }, {x7, x8, x15 }, {x9 }, {x10, x16 }, {x14 }}

©2012 Sami Khuri

5©2017 Sami Khuri


Indescernibility: Examples (III)

Note:

INDI ({Gene1, Gene2, Gene3}) = INDI ({Gene1, Gene2, Gene3,

Smoking})

Since the two indescernibility relations induce identical partitions, the attribute Smoking does not give any additional information and could therefore be discarded.

©2012 Sami Khuri

Indescernibility: Examples (IV)

DISCARD

Note:

INDI ({Gene1, Gene2, Gene3}) = INDI ({Gene1, Gene2, Gene3,

Smoking})

Since the two indescernibility relations induce identical partitions, the attribute Smoking does not give any additional information and could therefore be discarded.

©2012 Sami Khuri

Partitioning of the Universe• Equivalence classes [INDI ({Gene1, Gene2, Gene3})]

– are the basic blocks from which concepts can be built.– can be represented by a single member from the partition.

©2012 Sami Khuri


©2012 Sami Khuri

Conditional Attributes Decision Attribute Cancer Origin

©2012 Sami Khuri

Decision System

D = ( U, A, d ) d: decision attribute

• Example:if Gene1 is ↓ andGene2 is ↑ and Gene3 is 0 and Smoking is No then site of origin ofcancer is Colon

Conditional Attributes Decision Attribute

©2012 Sami Khuri

6©2017 Sami Khuri


Decision SystemRecall, INDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18 },

{x5, x11, x12, x17 },{x7, x8, x15 }, {x9 }, {x10, x16 }, {x14 }}

Note that {x3, x13, x18 } has different decision attributes (outcomes).And {x10, x16 } also has differentdecision attributes (outcomes).


©2012 Sami Khuri

Generalized DecisionAssociate a set of decision attribute values with each

equivalence class (partition) via δδA(x) = {i | y ∈ [x] and d(y) = i}

©2012 Sami Khuri


©2012 Sami Khuri

Set ApproximationI=(U, A), B ⊆ ALet X be a decision class:

- We want to approximate X by using information contained in the set of attributes, B- So we construct B-lower and B-upper

B-lower approximation of X: BX = {x | [x]B⊆ X}B-upper approximation of X: BX = {x | [x]B ∩ X ≠ ∅}

©2012 Sami Khuri

Definition of a Rough Set• B-boundary region of

X is given by: BNB = BX – BX

• A set X is said to be a rough set if BNB ≠ ∅

• Recall:B-lower approximation of X: BX = {x | [x]B⊆ X}B-upper approximation of X: BX = {x | [x]B ∩ X ≠ ∅}

©2012 Sami Khuri

Rough Set: An Example (I)• Back to the previous example where X = Lung• So we want to approximate the set of patients for

which the origin of cancer is the lung, using the three conditional attributes, Gene1, Gene2, and Gene3.

• Recall,If B = {Gene1, Gene2, Gene3}

thenINDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18 }, {x5, x11, x12, x17},

{x7, x8, x15 }, {x9 }, {x10, x16 }, {x14 }}©2012 Sami Khuri

7©2017 Sami Khuri


Rough Set: An Example (II)INDI ({Gene1, Gene2, Gene3})

= {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18 }, {x5, x11, x12, x17}, {x7, x8, x15 }, {x9 }, {x10, x16 }, {x14 }}

So,


©2012 Sami Khuri

Rough Set: An Example (III)Conditional Attributes Decision

Attribute

©2012 Sami Khuri

Rough Set: An Example (IV)

©2012 Sami Khuri

yes

yes/no

no

[x1] [x2] [x5]

[x3] [x10]

[x7] [x9] [x14]

Rough Set: An Example (V)

©2012 Sami Khuri

Rough Membership

• The rough membership function quantifies the degree of relative overlap between the rough set X and the equivalence class to which x belongs.

©2012 Sami Khuri

Rough Membership: ExampleConditional Attributes Decision

AttributeRecall: INDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18}, {x5, x11, x12, x17}, {x7, x8, x15}, {x9}, {x10, x16 }, {x14 }}

©2012 Sami Khuri

8©2017 Sami Khuri



©2012 Sami Khuri

Reducts (I)• Keep only those attributes that preserve

the indiscernibility relation and, consequently, set approximation.

• There are usually several such subsets of attributes and those which are minimal are called reducts.

©2012 Sami Khuri

Reducs (II)• A reduct is a minimal set of attributes B ⊆ A

such that INDI (B) = INDI (A)• The set {Gene1,Gene2,Gene3} is a reduct

because INDI ({Gene1,Gene2,Gene3}) = INDI (A) and it is minimal, i.e. the equality is no longer true if one of the attributes is removed

• If we know the values for all attributes in a reduct, we also know the values for all other attributes.

©2012 Sami Khuri

Reducs (III)• Finding reducts is a way of finding

dependencies in the data.• Boolean Algebra can be used to find reducts.

Recall: INDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18}, {x5, x11, x12, x17}, {x7, x8, x15}, {x9}, {x10, x16 }, {x14 }}

©2012 Sami Khuri

Cancer Origin Unreduced Table

INDI ({Gene1, Gene2, Gene3}) = {{x1, x6 ,}, {x2, x4 }, {x3, x13, x18}, {x5, x11, x12, x17}, {x7, x8, x15}, {x9}, {x10, x16 }, {x14 }}

So the cancer origin unreduced table is:

©2012 Sami Khuri

Discernibility Matrix• Let I=(U, A) be an Information System• A discernibility matrix of I is a symmetric

nxn matrix (n: number of partitions) with entriescij = {a ∈ A | a(xi) ≠ a(xj)} for i, j = 1,…,n

©2012 Sami Khuri

9©2017 Sami Khuri


Discernibility Matrices: An Example

cij = {a ∈ A | a(xi) ≠ a(xj)}for i, j = 1,…,n

©2012 Sami Khuri

Discernibility Matrices: Example Attributes for which the first two equivalence classes differ

©2012 Sami Khuri

Discernibility Functions (I)• A discernibility function fI for an

information system I is a Boolean function of m Boolean variables a*1,… , a*m (corresponding to attributes a1,… , am):

fI (a*1,… , a*m) = ∧{∨ c*ij | 1 ≤ j ≤ i ≤ n, c*ij ≠∅}where c*ij = {a* | a ∈ cij}

©2012 Sami Khuri

Discernibility Functions (II)• An implicant of a boolean function f is any

conjunction of literals such that if the values of these literals are true under an arbitrary evaluation v of variables, then the value of the function f under v is also true.

• A prime implicant is a minimal implicant.• The set of all prime implicants of fI determines

the set of all reducts of I.

©2012 Sami Khuri

Discernibility Functions: Example

The discernibility function of the cancer origin problem is:fI(G1,G2, G3, S) =

(G1 ∨ G2) (G1 ∨ G3 ∨ S) (G1) (G2 ∨ S) (G1 ∨ G2) (G3 ∨ S) (G1 ∨ G2 ∨ G3 ∨ S)(G2 ∨ G3 ∨ S) (G2) (G1 ∨ G2 ∨ S) (G2) (G1 ∨ G2 ∨ G3 ∨ S) (G2 ∨ G3 ∨ S)(G3 ∨ S) (G1 ∨ G2 ∨ G3) (G2 ∨ G3 ∨ S) (G1) (G2)(G1 ∨ G2 ∨ S) (G2) (G1 ∨ G3 ∨ S) (G2 ∨ G3 ∨ S)(G1 ∨ S) (G2 ∨ G3) (G1 ∨ G3)(G1 ∨ G2 ∨ G3 ∨ S) (G3 ∨ S) (G1 ∨ G2).

The discernibility function can be simplified to:fI(G1,G2, G3, S) = (G1∧G2∧G3)∨(G1∧G2∧S).

©2012 Sami Khuri

K-relative Reducts (I)

Example: Cancer origin: consider one complete column in the discernibility matrix

fI [x2](G1, G2, G3, S) = (G1∨G2) (G2∨G3∨S) (G2) (G1∨G2∨S) (G2) (G1∨G2∨G3∨S)(G2 ∨ G3 ∨ S) = G2 (after a few steps of simplification)

©2012 Sami Khuri

10©2017 Sami Khuri


K-relative Reducts (II)

Since fI[x2](G1, G2, G3, S) = G2 the equivalence class of x2 can be discerned from the other classes by considering only the Gene2 attribute.

©2012 Sami Khuri

K-relative Reducts (III)• A Boolean function restricted to the conjunction

running only over column k in the discernibility matrix (instead of over all columns) gives the k-relative discernibility function.

• The set of all prime implicants of this function determines the set of all k-relative reducts of A.

• k-relative reducts reveal the minimum amount of information needed to discern xk∈ U (or, more precisely, [xk] ⊆ U) from all other objects.

©2012 Sami Khuri

The Classification Problem• D=( U, A, d ) is a decision system, and

M(I) = (cij) its discernibility matrix.• Example: Cancer origin• Given a training set (i.e., a decision sub-table U’⊂U)

find an approximation of the decision attribute (site of origin).

©2012 Sami Khuri

Decision-Relative Discernibility (I)

• D=( U, A, d ) is a decision system, and M(I) = (cij) its discernibility matrix.

• Construct the decision-relative discernibility matrix of D: Md(D) = (cdij) assuming

cdij = ∅ if d(xi) = d(xj), andcdij = cij -{d} otherwise.

©2012 Sami Khuri

Decision-Relative Discernibility (II)

©2012 Sami Khuri

Decision-Relative Discernibility (III)

©2012 Sami Khuri

11©2017 Sami Khuri


Reduced Decision-Relative Discernibility Function

Reduced decision-relative discernibility function:fdM(D) = (G1 ∨ G3 ∨ S) ∧ (G3 ∨ S) …

= (G2∧S) ∨ (G2∧G3)

Decision-relative reducts: {Gene2,Gene3} and {Gene2, Smoking} uniquely defines to which decision class an object belongs.

©2012 Sami Khuri


©2012 Sami Khuri

Decision Rules (I)Reduct: {Gene2, Smoking} fdM(D) = (G2∧S) ∨ (G2∧G3)Decision Rules:

IF Gene2 = ↓ AND Smoking = Yes THEN Site of origin = LungIF Gene2 = 0 AND Smoking = Yes THEN Site of origin = LungIF Gene2 = ↓ AND Smoking = No THEN Site of origin = Lung

OR Site of origin = ColonIF Gene2 = ↑ AND Smoking = No THEN Site of origin = ColonIF Gene2 = ↑ AND Smoking = Yes THEN Site of origin = Colon

©2012 Sami Khuri

Decision Rules (II)Reduct: {Gene2, Gene3} fdM(D) = (G2∧S) ∨ (G2∧G3)Decision Rules:

IF Gene2 = ↓ AND Gene3 = 0 THEN origin = LungIF Gene2 = 0 AND Gene3 = 0 THEN origin = LungIF Gene2 = ↓ AND Gene3 = ↑ THEN origin = Lung

OR origin = ColonIF Gene2 = ↑ AND Gene3 = 0 THEN origin = ColonIF Gene2 = ↑ AND Gene3 = ↑ THEN origin = Colon

©2012 Sami Khuri

Quality Measures for Rules (I)

Let r be ϕ ⇒ d = v be a rule induced from a decision table D = (U, A, d).

©2012 Sami Khuri

Quality Measures for Rules (II)Recall:r is ϕ ⇒ d = v a rule induced from a decision table

D = (U, A, d).

©2012 Sami Khuri

12©2017 Sami Khuri


Quality Measures: Example (I)

• Recall that by using {Gene2, Gene3} we obtain the following decision rules:

– IF Gene2 = ↓ AND Gene3 = 0 THEN origin = Lung– IF Gene2 = 0 AND Gene3 = 0 THEN origin = Lung– IF Gene2 = ↓ AND Gene3 = ↑ THEN origin = Lung

OR origin = Colon– IF Gene2 = ↑ AND Gene3 = 0 THEN origin = Colon– IF Gene2 = ↑ AND Gene3 = ↑ THEN origin = Colon

©2012 Sami Khuri

Quality Measures: Example (II)

©2012 Sami Khuri


©2012 Sami Khuri

Real World Data• A real world data set always contains mixed

types of data such as continuous valued, symbolic data, etc.

• When it comes to analyzing attributes with real values, they must undergo a process called discretization, which divides the attribute’s values into intervals.

• There is a lack of unified approach to discretization problems so far, and the choice of method depends heavily on data considered.

©2012 Sami Khuri

Discretization• Human reasoning with many factors is often qualitative• Experimental data are mostly quantitative• So, very often we discretize the data by going from

qualitative to quantitative data.• Example:

©2012 Sami Khuri

Genetic Algorithms to the Rescue

• Recall, a reduct is a reduced set of attributes, that essentially provides the same amount of information about a data set as a complete set of attributes.

• Computing the reduct set can be very time consuming particularly when the decision table has too many attributes or different attribute values.

• Genetic algorithms can be used to optimize the number of reducts.

©2012 Sami Khuri

13©2017 Sami Khuri


Attribute reduction via GA• Attribute reduction is the process of identifying

and removing redundant and irrelevant attributes from huge data sets, reducing its volume. The reduced data set can be much more effectively analyzed. Attribute reduction in decision-theoretic rough set models through region preservation is an optimization problem, thus Genetic Algorithms are used to achieve this optimization.

©2012 Sami Khuri

Attribute reduction in decision-theoretic rough set models using genetic algorithmby S. Chebrolu and S. Sanjeevi, 2011.

References

• Zdzisław Pawlak. Rough set theory and its applications. Journal of Telecommunications and Information Technology, Vol. 3, 2002.

• S. Vinterbo and A. Øhrn. Minimal approximate hitting sets and rule templates. International Journal of Approximate Reasoning, 25(2):123–143, 2000.

• J. Wroblewski. Finding minimal reducts using genetic algorithms. In Proc. Second International Joint Conference on Information Sciences, pages 186–189, Sept. 1995.

©2012 Sami Khuri

genetic algorithms more applications - sjsukhuri/aalto_2017/aalto_more... · –fuzzy sets –logic...

Documents