genetic algorithms and hybrid systems will genetic algorithms work for my problem?

Genetic algorithms and hybrid systemsGenetic algorithms and hybrid systems Will genetic algorithms work for my Will genetic algorithms work for my

problem?problem?The travelling salesman problemThe travelling salesman problem

Will a hybrid system work for my problem?Will a hybrid system work for my problem?Neuro-fuzzy decision-support systemsNeuro-fuzzy decision-support systems

Time-series predictionTime-series prediction

SummarySummary

Lecture 15Lecture 15

Knowledge engineering:Knowledge engineering:

Optimisation Optimisation is the process of finding a better is the process of finding a better solution to a problem.solution to a problem.

A genetic algorithm generates a population of A genetic algorithm generates a population of competing candidate solutions and then causes competing candidate solutions and then causes them to evolve through the process of natural them to evolve through the process of natural selection – poor solutions tend to die out, while selection – poor solutions tend to die out, while better solutions survive and reproduce. By better solutions survive and reproduce. By repeating this process over and over again, the repeating this process over and over again, the genetic algorithm breeds an optimal solution.genetic algorithm breeds an optimal solution.

Will genetic algorithms work for my Will genetic algorithms work for my problem?problem?

The travelling salesman problemThe travelling salesman problem

Suppose, we are going to travel by car in Western Suppose, we are going to travel by car in Western and Central Europe. We want to produce an optimal and Central Europe. We want to produce an optimal itinerary for visiting all major cities and returning itinerary for visiting all major cities and returning home. home.

This problem is well known as the This problem is well known as the travelling travelling salesman problemsalesman problem. Given a finite number of cities, . Given a finite number of cities, NN, and the cost of travel (or the distance) between , and the cost of travel (or the distance) between each pair of cities, we need to find the cheapest way each pair of cities, we need to find the cheapest way (or the shortest route) for visiting each city exactly (or the shortest route) for visiting each city exactly once and returning to the starting point.once and returning to the starting point.

Case study 7:Case study 7:

arranging routes for school buses to pick up arranging routes for school buses to pick up children in a school district,children in a school district,

delivering meals to home-bound people,delivering meals to home-bound people,

scheduling stacker cranes in a warehouse,scheduling stacker cranes in a warehouse,

planning truck routes to pick up parcel post and planning truck routes to pick up parcel post and many others. many others.

The TSP is represented in numerous The TSP is represented in numerous transportation and logistics applications such astransportation and logistics applications such as

A classic example of the TSP is the scheduling A classic example of the TSP is the scheduling of a machine to drill holes in a circuit board. of a machine to drill holes in a circuit board.

Suppose we have nine cities named from 1 to 9. In Suppose we have nine cities named from 1 to 9. In a chromosome, the order of the integers represents a chromosome, the order of the integers represents the order in which the cities will be visited by the the order in which the cities will be visited by the salesman.salesman.

How does a genetic algorithm solve the How does a genetic algorithm solve the travelling salesman problem?travelling salesman problem? First, we need to decide how to represent a route ofFirst, we need to decide how to represent a route of the salesman. The most natural way of representingthe salesman. The most natural way of representing a route is the a route is the path representationpath representation. . Each city isEach city is given an alphabetic or numerical name, the routegiven an alphabetic or numerical name, the route through the cities is represented as a chromosome,through the cities is represented as a chromosome, and appropriate genetic operators are used to createand appropriate genetic operators are used to create new routes. new routes.

An example of the salesman’s routeAn example of the salesman’s route

1

2

3

4

6 5

7

8

9

1 56 3 2 48 9 7

The crossover operator in its classical form cannot be The crossover operator in its classical form cannot be directly applied to the TSP. A simple exchange of parts directly applied to the TSP. A simple exchange of parts between two parents would produce illegal routes between two parents would produce illegal routes containing duplicates and omissions – some cities would containing duplicates and omissions – some cities would be visited twice while some others would not be be visited twice while some others would not be visited at all:visited at all:

How does the crossover operator works?How does the crossover operator works?

1 56 3 2 48 9 7Parent1: 3 67 1 9 84 2 5Parent2:

1 56 3 9 84 2 5Child1: 3 67 1 2 48 9 7Child2:

Crossover operators for the TSPCrossover operators for the TSP

Step1

Parent 2: 3 67 2 51 9 84Parent 1: 9 71 56 3 2 8 4

Step2

* 1 9 84Child 1: 3 2 48Child 2:** ** ***

Step3

9 84Child 1: 3 2 48Child 2:7 * *67 56 2 7 9

Parent 1: 9 71 56 3 2 8 4 Parent 2: 3 67 2 51 9 84

135 7 ** 16 5

The The inversion operator inversion operator selects two random selects two random points along the chromosome string and points along the chromosome string and reverses the order of the cities between these reverses the order of the cities between these points. points.

How does the mutation operator works?How does the mutation operator works?

There are two types of mutation operators: There are two types of mutation operators: reciprocal exchange reciprocal exchange and and inversioninversion..

The The reciprocal exchange operator reciprocal exchange operator simply simply swaps two randomly selected cities in the swaps two randomly selected cities in the chromosome.chromosome.

Mutation operators for the TSPMutation operators for the TSP

9 71 56 3 2 8 4(a) originalchromosomes

Reciprocalexchange Inversion

9 71 56 43 2 8

9 71 86 3 2 5 4(b) mutatedchromosomes

9 71 32 45 6 8

The fitness of each individual chromosome is The fitness of each individual chromosome is determined as the reciprocal of the route determined as the reciprocal of the route length.length.

In other words, the shorter the route, the fitter In other words, the shorter the route, the fitter the chromosome.the chromosome.

How do we define a fitness function in the How do we define a fitness function in the TSP? TSP?

Performance graph and the best salesman’s route created Performance graph and the best salesman’s route created in a population of 20 chromosomes after 100 generations in a population of 20 chromosomes after 100 generations

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1The total distance: 5.124

4

1

2

5

67

8

9

10

11

12

13

14

16

17

18

19

20

3

15

0 10 20 30 40 50 60 70 80 90 1004

5

6

7

8

9

10

11

Generations

N = 20, pc = 0.7, pm = 0.001

BestAverage

Dis

tance

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


4

1

2

5

67

8

9

10

11

12

13

14

16

17

18

19

20

3

15

0 10 20 30 40 50 60 70 80 90 1004

5

6

8

7

9

10

11

Generations

N = 200, pc = 0.7, pm = 0.001

BestAverage

Performance graphs and the best routes created in a Performance graphs and the best routes created in a population of 200 chromosomes: mutation rate is 0.001population of 200 chromosomes: mutation rate is 0.001

Performance graphs and the best routes created in a Performance graphs and the best routes created in a population of 200 population of 200 chromosomes: mutation rate is 0.01chromosomes: mutation rate is 0.01

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


4

1

2

5

67

8

9

10

11

12

13

14

16

17

18

19

20

3

15

0 10 20 30 40 50 60 70 80 90 1004

5

6

7

8

9

10

11

Generations

BestAverage

N = 200, pc = 0.7, pm = 0.01

Solving complex real-world problems requires Solving complex real-world problems requires an application of complex intelligent systems an application of complex intelligent systems that combine the advantages of expert systems, that combine the advantages of expert systems, fuzzy logic, neural networks and evolutionary fuzzy logic, neural networks and evolutionary computation. Such systems can integrate computation. Such systems can integrate human-like expertise in a specific domain with human-like expertise in a specific domain with abilities to learn and adapt to a rapidly changing abilities to learn and adapt to a rapidly changing environment.environment.

Will a hybrid system work for my Will a hybrid system work for my problem?problem?

As an example, we will develop an intelligent As an example, we will develop an intelligent system for diagnosing myocardial perfusion system for diagnosing myocardial perfusion from cardiac images. Suppose, we have a set from cardiac images. Suppose, we have a set of cardiac images as well as the clinical notes of cardiac images as well as the clinical notes and physician’s interpretation.and physician’s interpretation.

Case study 8Case study 8: : Neuro-fuzzy decision-support systemsNeuro-fuzzy decision-support systems

Diagnosis in modern cardiac medicine is based on Diagnosis in modern cardiac medicine is based on the analysis of SPECT (Single Proton Emission the analysis of SPECT (Single Proton Emission Computed Tomography) images.Computed Tomography) images.

By injecting a patient with radioactive tracer, two By injecting a patient with radioactive tracer, two sets of SPECT images are obtained: one is taken sets of SPECT images are obtained: one is taken 10 – 15 minutes after the injection when the 10 – 15 minutes after the injection when the stress is greatest (stress images), and the other stress is greatest (stress images), and the other is taken 2 – 5 hours after the injection (rest is taken 2 – 5 hours after the injection (rest images). Distribution of the radioactive tracer in images). Distribution of the radioactive tracer in the cardiac muscle is proportional to the muscle’s the cardiac muscle is proportional to the muscle’s perfusion.perfusion.

A cardiologist detects abnormalities in the heart A cardiologist detects abnormalities in the heart function by comparing stress and rest images.function by comparing stress and rest images.

What are SPECT images?What are SPECT images?

The SPECT images are usually represented by The SPECT images are usually represented by high resolution two-dimensional black-and-white high resolution two-dimensional black-and-white pictures with up to 256 shades of grey. Brighter pictures with up to 256 shades of grey. Brighter patches on the image correspond to well-perfused patches on the image correspond to well-perfused areas of the myocardium, while darker patches areas of the myocardium, while darker patches may indicate the presence of an ischemia.may indicate the presence of an ischemia.

Unfortunately a visual inspection of the SPECT Unfortunately a visual inspection of the SPECT images is highly subjective; physicians’ images is highly subjective; physicians’ interpretations are therefore often inconsistent interpretations are therefore often inconsistent and susceptible to errors.and susceptible to errors.

For this study, we use 267 cardiac diagnostic For this study, we use 267 cardiac diagnostic cases. Each case is accompanied by two SPECT cases. Each case is accompanied by two SPECT images (the stress image and the rest image), and images (the stress image and the rest image), and each image is divided into 22 regions.each image is divided into 22 regions.

The region’s brightness, which in turn reflects The region’s brightness, which in turn reflects perfusion inside this region, is expressed by an perfusion inside this region, is expressed by an integer number between 0 and 100.integer number between 0 and 100.

Thus, each cardiac diagnostic case is represented Thus, each cardiac diagnostic case is represented by 44 continuous features by 44 continuous features and one binary feature that assigns an overall and one binary feature that assigns an overall diagnosis – normal or abnormal.`diagnosis – normal or abnormal.`

The entire SPECT data set consists of 55 cases The entire SPECT data set consists of 55 cases classified as normal (positive examples) and 212 classified as normal (positive examples) and 212 cases classified as abnormal (negative examples).cases classified as abnormal (negative examples).

The entire set is divided into training and test The entire set is divided into training and test sets.sets.

The training set has 40 positive and 40 negative The training set has 40 positive and 40 negative examples. The test set is represented by 15 examples. The test set is represented by 15 positive and 172 negative examples.positive and 172 negative examples.

The SPECT data set The SPECT data set

The number of neurons in the input layer is The number of neurons in the input layer is determined by the total number of regions in the determined by the total number of regions in the stress and rest images. In this example, each stress and rest images. In this example, each image is divided into 22 regions, so we need 44 image is divided into 22 regions, so we need 44 input neurons.input neurons.

Since SPECT images are to be classified as either Since SPECT images are to be classified as either normal or abnormal, we use two output neurons.normal or abnormal, we use two output neurons.

A good generalisation is obtained with as little as A good generalisation is obtained with as little as 5 to 7 neurons in the hidden layer. 5 to 7 neurons in the hidden layer.

Can we train a back-propagation neural Can we train a back-propagation neural network to classify the SPECT images into network to classify the SPECT images into normal and abnormal?normal and abnormal?

However, when we test the network on the test However, when we test the network on the test set, we find that the network’s performance is set, we find that the network’s performance is rather poor – about 25 percent of normal cardiac rather poor – about 25 percent of normal cardiac diagnostic cases are misclassified as abnormal diagnostic cases are misclassified as abnormal and over 35 percent of abnormal cases are and over 35 percent of abnormal cases are misclassified as normal; the overall diagnostic misclassified as normal; the overall diagnostic error exceeds 33 percent.error exceeds 33 percent.

This indicates that the training set may lack some This indicates that the training set may lack some important examples (a neural network is only as important examples (a neural network is only as good as the examples used to train it).good as the examples used to train it).

First, we need to redefine the problem. To train the First, we need to redefine the problem. To train the network, we used the same number of positive and network, we used the same number of positive and negative examples. Although in real clinical trials, negative examples. Although in real clinical trials, the ratio between normal and abnormal SPECT the ratio between normal and abnormal SPECT images is very different, the misclassification of an images is very different, the misclassification of an abnormal cardiac case could lead to infinitely more abnormal cardiac case could lead to infinitely more serious consequences than the misclassification of a serious consequences than the misclassification of a normal case. Therefore, in order to achieve a small normal case. Therefore, in order to achieve a small classification error for abnormal SPECT images, we classification error for abnormal SPECT images, we might agree to have a relatively large error for might agree to have a relatively large error for normal images.normal images.

Can we improve the accuracy of the Can we improve the accuracy of the diagnosis?diagnosis?

The neural network produces two outputs. `The The neural network produces two outputs. `The first output corresponds to the possibility that the first output corresponds to the possibility that the SPECT image belongs to the class SPECT image belongs to the class normalnormal, and , and the second to the possibility that the image the second to the possibility that the image belongs to the class belongs to the class abnormalabnormal. If, for example, . If, for example, the first (the first (normalnormal) output is 0.92 and the second ) output is 0.92 and the second ((abnormalabnormal) is 0.16, the SPECT image is ) is 0.16, the SPECT image is classified as normal, and we can conclude that the classified as normal, and we can conclude that the risk of a heart attack for this case is low.risk of a heart attack for this case is low.

On the other hand, if the On the other hand, if the normal normal output is low, output is low, say 0.17, and the say 0.17, and the abnormal abnormal output is much output is much higher, say 0.51, the SPECT image is classified higher, say 0.51, the SPECT image is classified as abnormal, and we can infer that the risk of a as abnormal, and we can infer that the risk of a heart attack in this case is rather high. However, heart attack in this case is rather high. However, if the two outputs are close – say the if the two outputs are close – say the normal normal output is 0.51 and the output is 0.51 and the abnormal abnormal 0.49 – we cannot 0.49 – we cannot confidently classify the image.confidently classify the image.

To build a fuzzy system, we first need to To build a fuzzy system, we first need to determine input and output variables, define determine input and output variables, define fuzzy sets and construct fuzzy rules.fuzzy sets and construct fuzzy rules.

For our problem, there are two inputs (For our problem, there are two inputs (NN NN output 1 output 1 and and NN output 2NN output 2) and one output (the ) and one output (the risk risk of a heart attack).of a heart attack).

The inputs are normalised to be within the The inputs are normalised to be within the range of [0, 1], and the output can vary range of [0, 1], and the output can vary between 0 and 100 percent.between 0 and 100 percent.

Can we use fuzzy logic for decision-making Can we use fuzzy logic for decision-making in medical diagnosis?in medical diagnosis?

Fuzzy sets of the neural network output Fuzzy sets of the neural network output normalnormal

0.10

1.0

0.0

0.2

0.4

0.6

0.20 .3 0.40 .5 0.60 .7 0.80 .9 1NN output 1 (normalised)

0.8HighLow Medium

Deg

ree

of m

embe

rshi

p

Fuzzy sets of the neural network output Fuzzy sets of the neural network output abnormalabnormal

0.10

1.0

0.0

0.2

0.4

0.6

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1NN output 2 (normalised)

0.8HighLow Medium

Deg

ree

of m

embe

rshi

p

Fuzzy sets of the linguistic variable Fuzzy sets of the linguistic variable RiskRisk

1000.0

20 30 40 50 60 70 80 90 100Risk, %

1.0

0.2

0.4

0.6

0.8ModerateLow HighVery low Veryhigh

Deg

ree

of m

embe

rshi

p

1. If (NN_output1isLow)and(NN_output2isLow)then(Risk isModerate)2. If (NN_output1isLow)and(NN_output2isMedium) then(Risk isHigh)3. If (NN_output1isLow)and(NN_output2isHigh) then(Risk isVery_high)4. If (NN_output1isMedium) and(NN_output2isLow)then(Risk isLow)5. If (NN_output1isMedium) and(NN_output2isMedium) then(Risk isModerate)6. If (NN_output1isMedium) and(NN_output2isHigh) then(Risk isHigh)7. If (NN_output1isHigh) and(NN_output2isLow)then(Risk isVery_low)8. If (NN_output1isHigh) and(NN_output2isMedium) then(Risk isLow)9. If (NN_output1isHigh) and(NN_output2isHigh) then(Risk isModerate)

Fuzzy rules for assessing the risk of a heart Fuzzy rules for assessing the risk of a heart decease decease

Structure of the neuro-fuzzy systemStructure of the neuro-fuzzy system

Fuzzy Rule Base

Risk

NNoutput1 NNoutput2

4321 765

21

31 2 4442 43

Three-dimensional plot for the fuzzy rule baseThree-dimensional plot for the fuzzy rule base

00.2

0.40.6

0.81 0

0.20.4

0.60.8

1

20

40

60

80

NNoutput2NNoutput1

Ris

k, %

The system’s output is a crisp number that The system’s output is a crisp number that represents a patient’s risk of a heart attack.represents a patient’s risk of a heart attack.

Based on this number, a cardiologist can now Based on this number, a cardiologist can now classify cardiac cases with greater certainty – classify cardiac cases with greater certainty – when the risk is quantified, a decision-maker when the risk is quantified, a decision-maker has a much better chance of making the right has a much better chance of making the right decision. For instance, if the risk is low, say, decision. For instance, if the risk is low, say, smaller than 30 percent, the cardiac case can smaller than 30 percent, the cardiac case can be classified as be classified as normalnormal, but if the risk is , but if the risk is high, say, greater than 50 percent, the case is high, say, greater than 50 percent, the case is classified as classified as abnormalabnormal..

However, cardiac cases with the risk However, cardiac cases with the risk between 30 and 50 percent cannot be between 30 and 50 percent cannot be classified as either classified as either normal normal or or abormal abormal – – rather, such cases are rather, such cases are uncertainuncertain..

Can we classify some of the uncertain cases Can we classify some of the uncertain cases using the knowledge of a cardiologist?using the knowledge of a cardiologist?

A cardiologist knows that, in normal heart A cardiologist knows that, in normal heart muscle, perfusion at maximum stress is usually muscle, perfusion at maximum stress is usually higher than perfusion at rest:higher than perfusion at rest:

If perfusion inside region If perfusion inside region i i at stress is higher than at stress is higher than perfusion inside the same region at rest, then the perfusion inside the same region at rest, then the risk of a heart attack should be decreased.risk of a heart attack should be decreased.

If perfusion inside region If perfusion inside region i i at stress is not higher at stress is not higher than perfusion inside the same region at rest, then than perfusion inside the same region at rest, then the risk of a heart attack should be increased.the risk of a heart attack should be increased.

TheseThese heuristics can be implemented in the heuristics can be implemented in the diagnostic system as follows:diagnostic system as follows:

Step 1Step 1: : Present the neuro-fuzzy system with a Present the neuro-fuzzy system with a cardiac case. cardiac case.

Step 2Step 2: : If the system’s output is less than 30, If the system’s output is less than 30, classify the presented case as classify the presented case as normal normal and and

then stop. If the output is greater than 50, then stop. If the output is greater than 50, classify the case as classify the case as abnormal abnormal and and stop. stop. Otherwise, go to Step 3. Otherwise, go to Step 3.

Step 3Step 3: : For region 1, subtract perfusion at rest from For region 1, subtract perfusion at rest from perfusion at stress. If the result is perfusion at stress. If the result is positive, positive, decrease the current risk by decrease the current risk by multiplying multiplying its value by 0.99. its value by 0.99. Otherwise, increase the Otherwise, increase the risk by multiplying risk by multiplying its value by 1.01. Repeat its value by 1.01. Repeat this procedure for all this procedure for all 22 regions and then go 22 regions and then go to Step 4. to Step 4.

Step 4Step 4: : If the new risk value is less than 30, classify If the new risk value is less than 30, classify the case as the case as normalnormal; if the risk is greater than ; if the risk is greater than

50, classify the case as 50, classify the case as abnormalabnormal; ; otherwiseotherwise – classify the case as – classify the case as uncertainuncertain..

The accuracy of diagnosis has dramatically The accuracy of diagnosis has dramatically improved – the overall diagnostic error does not improved – the overall diagnostic error does not exceed 5 percent, while only 3 percent of exceed 5 percent, while only 3 percent of abnormal cardiac cases are misclassified as abnormal cardiac cases are misclassified as normal.normal.

Although we have not improved the system’s Although we have not improved the system’s performance on normal cases (over 30 percent of performance on normal cases (over 30 percent of normal cases are still misclassified as abnormal), normal cases are still misclassified as abnormal), and up to 20 percent of the total number of cases and up to 20 percent of the total number of cases are classified as uncertain, the neuro-fuzzy system are classified as uncertain, the neuro-fuzzy system can actually achieve even better results in can actually achieve even better results in classifying SPECT images than a cardiologist can.classifying SPECT images than a cardiologist can.

In this example, the neuro-fuzzy system has a In this example, the neuro-fuzzy system has a heterogeneous heterogeneous structure – the neural network andstructure – the neural network and

fuzzy system work as independent fuzzy system work as independent components (although they cooperate in solving the components (although they cooperate in solving the problem). When a new case is presented to the problem). When a new case is presented to the diagnostic system, the trained neural network diagnostic system, the trained neural network determines inputs to the fuzzy system. Then the determines inputs to the fuzzy system. Then the fuzzy system using predefined fuzzy sets and fuzzy fuzzy system using predefined fuzzy sets and fuzzy rules, maps the given inputs to an output, and rules, maps the given inputs to an output, and thereby obtains the risk of a heart attack.thereby obtains the risk of a heart attack.

Adaptive Neuro-Fuzzy Inference System (ANFIS) Adaptive Neuro-Fuzzy Inference System (ANFIS) is a typical example of a neuro-fuzzy system with a is a typical example of a neuro-fuzzy system with a homogeneous homogeneous structure.structure.

Case study 9Case study 9:: Time-series predictionTime-series prediction

As an example, we will develop a tool to predict As an example, we will develop a tool to predict an aircraft’s trajectory during its landing aboard an aircraft’s trajectory during its landing aboard an aircraft carrier.an aircraft carrier.

Suppose, we have a database of landing Suppose, we have a database of landing trajectories of various aircraft flown by different trajectories of various aircraft flown by different pilots, and we also can use RADAR numerical pilots, and we also can use RADAR numerical data, which provide real-time trajectories of landing data, which provide real-time trajectories of landing aircraft. Our goal is to predict an aircraft’s aircraft. Our goal is to predict an aircraft’s trajectory at least two seconds in advance, based trajectory at least two seconds in advance, based on the aircraft’s current position.on the aircraft’s current position.

The landing of an aircraft, particularly aboard The landing of an aircraft, particularly aboard aircraft carriers, is an extremely complex process.aircraft carriers, is an extremely complex process.

It is affected by such variables as the flight deck’s It is affected by such variables as the flight deck’s space constraints and its motions (both pitch and space constraints and its motions (both pitch and roll), the aircraft’s ordinance and fuel load, roll), the aircraft’s ordinance and fuel load, continuous mechanical preparations, and the most continuous mechanical preparations, and the most critical of all – time constraints.critical of all – time constraints.

The ship may be heaving 10 feet up and 10 feet The ship may be heaving 10 feet up and 10 feet down, making a 20-foot displacement from a level down, making a 20-foot displacement from a level deck. In addition, it is difficult to see approaching deck. In addition, it is difficult to see approaching aircraft at night or during stormy conditions.aircraft at night or during stormy conditions.

Prediction of the aircraft’s positionPrediction of the aircraft’s position

Responsibility for the aircraft’s final approach Responsibility for the aircraft’s final approach and landing lies with the Landing Signal Officer and landing lies with the Landing Signal Officer (LSO).(LSO).

When an aircraft is within one nautical mile of When an aircraft is within one nautical mile of the landing deck, which roughly corresponds to the landing deck, which roughly corresponds to 60 seconds in real time, the aircraft’s flight is 60 seconds in real time, the aircraft’s flight is carefully observed and guided. During this carefully observed and guided. During this critical time, the LSO needs to predict the critical time, the LSO needs to predict the aircraft’s position at least two seconds ahead.aircraft’s position at least two seconds ahead.

Such problems are known in mathematics as Such problems are known in mathematics as time-series prediction time-series prediction problems.problems.

A time series can be defined as a set of A time series can be defined as a set of observations, each one being recorded at a observations, each one being recorded at a specific time. For instance, a time series can be specific time. For instance, a time series can be obtained by recording the aircraft’s positions obtained by recording the aircraft’s positions over a time interval of, say, 60 seconds before over a time interval of, say, 60 seconds before landing.landing.

Real-world time-series problems are non-linear Real-world time-series problems are non-linear and often exhibit chaotic behaviour, which and often exhibit chaotic behaviour, which make them hard to model.`make them hard to model.`

What is a time series?What is a time series?

Prediction of the aircraft’s landing trajectory is Prediction of the aircraft’s landing trajectory is mainly based on the experience of a LSO (all mainly based on the experience of a LSO (all LSOs are trained pilots).LSOs are trained pilots).

An automatic prediction system can use aircraft-An automatic prediction system can use aircraft- position data given by the ship’s RADAR, and position data given by the ship’s RADAR, and also data records of previous landings executed also data records of previous landings executed by pilots flying different types of aircraft.by pilots flying different types of aircraft.

The system is trained off-line with the past data. The system is trained off-line with the past data. Then it is presented on-line with the current Then it is presented on-line with the current motion profile, and required to predict the motion profile, and required to predict the aircraft’s motion in the next few seconds.aircraft’s motion in the next few seconds.

On-line time-series prediction of On-line time-series prediction of an aircraft’s trajectoryan aircraft’s trajectory

To predict an aircraft’s position on-line we willTo predict an aircraft’s position on-line we will use an ANFIS.use an ANFIS.

Input: current motion

Time-series

Output: predicted

-10

0

10

20

Time, s0 2 4 6

Time, s0 2 4 6 8

-10

0

10

20

predictionsystem

profile of the aircraft position of the aircraft

8Lin

e-u

p,

feet

To predict a future value for a time series, we use To predict a future value for a time series, we use values that are already known. For example, if we values that are already known. For example, if we want to predict an aircraft’s position two seconds want to predict an aircraft’s position two seconds ahead, we may use its current position data as well ahead, we may use its current position data as well as data recorded, say, 2, 4 and 6 seconds before the as data recorded, say, 2, 4 and 6 seconds before the current position. These four known values current position. These four known values represent an input pattern – a four-dimensional represent an input pattern – a four-dimensional vector of the following form:vector of the following form:

What do we use as ANFIS inputs?What do we use as ANFIS inputs?

)()2()4()6( txtxtxtx x

The ANFIS output corresponds to the trajectory The ANFIS output corresponds to the trajectory prediction: the aircraft’s position two seconds ahead, prediction: the aircraft’s position two seconds ahead, x x ((t t + 2).+ 2).

What is the ANFIS output?What is the ANFIS output?

For this case study, we will use 10 landing For this case study, we will use 10 landing trajectories – five for training and five for testing. trajectories – five for training and five for testing. Each trajectory is a time series of the aircraft’s Each trajectory is a time series of the aircraft’s position data points recorded every half a second position data points recorded every half a second over a time interval of 60 seconds before landing. over a time interval of 60 seconds before landing. Thus, a data set for each trajectory contains 121 Thus, a data set for each trajectory contains 121 values.values.

For a landing trajectory recorded over a time interval For a landing trajectory recorded over a time interval of 60 seconds, we obtain 105 training samples of 60 seconds, we obtain 105 training samples represented by a 105 represented by a 105 5 matrix. Thus, the entire data 5 matrix. Thus, the entire data set, which we use for training the ANFIS set, which we use for training the ANFIS , is , is represented by a 525 represented by a 525 5 matrix. 5 matrix.

An aircraft trajectory and a data set built to An aircraft trajectory and a data set built to train the ANFIStrain the ANFIS

-20

-10

0

10

20

0

Time, s1 2 3 4 5 6 7

Inputs

x1 x2 x3 x4

Desiredoutput

17.4 2.1 11.0 3.9 4.2 12.9 7.5 10.1 2.1 4.9

10.0 9.8 8.2 2.0 5.3Lin

e-u

p,

feet

A practical approach is to choose the smallest A practical approach is to choose the smallest number of membership functions. Thus, we may number of membership functions. Thus, we may begin with two membership functions assigned to begin with two membership functions assigned to each input variable.each input variable.

How many membership functions How many membership functions should we assign to each input variable?should we assign to each input variable?

Performance of the ANFIS with four inputs and Performance of the ANFIS with four inputs and two membership functions assigned to each input: two membership functions assigned to each input:

one epoch one epoch

0 10 20 30 40 50 60-25

-20

-15

-10

-5

0

5

10

15

Time, s

Aircraft trajectory

ANFIS prediction

20

Lin

e-u

p,

feet

Performance of the ANFIS with four inputs and Performance of the ANFIS with four inputs and two membership functions assigned to each input: two membership functions assigned to each input:

100 epochs100 epochs

0 10 20 30 40 50 60-25

-20

-15

-10

-5

0

5

10

15

Time,s

Aircrafttrajectory

ANFIS prediction

20

Lin

e-u

p,

feet

The ANFIS’s performance can be significantly The ANFIS’s performance can be significantly improved by assigning improved by assigning three membership three membership functions to each input variablefunctions to each input variable. .

How can we improve the ANFIS’s How can we improve the ANFIS’s performance?performance?

Performance of the ANFIS with four inputs and Performance of the ANFIS with four inputs and three membership functions assigned to each input three membership functions assigned to each input

after one epoch of training after one epoch of training

Lin

e-up

, fee

t

0 10 20 30 40 50 60

-25

-20

-15

-10

-5

0

5

10

15

Time, s

Aircraft trajectory

ANFIS prediction

20

-30

Lin

e-u

p,

feet

Let us, for example, examine an ANFIS with Let us, for example, examine an ANFIS with six inputs that correspond to the aircraft’s flight six inputs that correspond to the aircraft’s flight positions at (positions at (t t – 5), (– 5), (t t – 4), (– 4), (t t – 3), (– 3), (t t – 2), (– 2), (t t – 1), – 1), and and tt, respectively. The ANFIS output still , respectively. The ANFIS output still remains the two-second prediction.remains the two-second prediction.

Another way of improving time-series Another way of improving time-series prediction is to prediction is to increase the number of input increase the number of input variablesvariables..

The training data set is now represented by a The training data set is now represented by a 535 535 7 matrix.7 matrix.

Performance of the ANFIS with six inputs and two Performance of the ANFIS with six inputs and two membership functions assigned to each input: membership functions assigned to each input:

prediction after one epochprediction after one epoch

0 10 20 30 40 50 60

-25

-20

-15

-10

-5

0

5

10

15

Time, s

Aircraft trajectory

ANFIS prediction

20

-30

Lin

e-u

p,

feet

Performance of the ANFIS with six inputs and two Performance of the ANFIS with six inputs and two membership functions assigned to each input: membership functions assigned to each input:

prediction errorsprediction errors

0 10 20 30 40 50 60-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Time, s

Pre

dic

tion

err

or,

feet