PowerPoint Presentation

Feature extraction for change detectionCan you detect an abrupt change in this picture?Ludmila I KunchevaSchool of Computer ScienceBangor UniversityAnswer at the endPlan

Zeno says there is no such thing as change...If change exists, is it a good thing?Context or nothing!Feature extraction for change detection PCA backwards?

Zeno of Elea (ca. 490430 BC)If everything, when it occupies an equal space, is at rest, and if that which is in locomotion is always occupying such a space at any moment, the flying arrow is therefore motionless. as recounted by Aristotle, Physics VI:9, 239b5

No motion, no movement, NO CHANGEZenos Paradox of the Arrow

Does change exist?

Zeno says no...

Nonetheless... Change TypesPossible applications:

fraud detectionmarket analysismedical condition monitoringnetwork traffic controlUnivariate detectors (Control charts):

Shewhart's methodCUSUM (CUmulative SUM)SPRT (Wald's Sequential Probability Ratio Test)2 approachesUse an adaptive algorithm

(No need to identify the type of changeor detect change explicitly)Detect change

(Update/re-train the algorithm if necessary)Labelled dataUnlabelled dataData (all features)Labels are availableClassifierDistribution modellingError rateChangestatisticthresholdChange/NO changeClassificationData (all features)Labels are availableLabels are NOT availableClassifierDistribution modellingError rateChangestatisticthresholdChange/NO changeData (all features)Feature EXTRACTOR

Distribution modellingChangestatisticthresholdChange/NO changeFeaturesmultidimensionalData (all features)Labels are availableLabels are NOT availableClassifierGMMHMMParzen windowskernel methodsmartingalesError ratethresholdChange/NO changeData (all features)Feature EXTRACTOR

clusteringkernel methodsGMMkd-treesHotellingthresholdChange/NO changeFeaturesA change in the (unconditional) data distribution will:

render the classifier uselessmake no difference to the classification performanceimprove the classification performance

ClassificationA change in the (unconditional) data distribution will:

render the classifier uselessmake no difference to the classification performanceimprove the classification performance

Vote, please!

A change in the (unconditional) data distribution will:

render the classifier uselessmake no difference to the classification performanceimprove the classification performance

Vote, please!

ClassificationNo change in the (unconditional) data distribution will:

render the classifier uselessmake no difference to the classification performanceimprove the classification performance

No change in the (unconditional) data distribution will:

render the classifier uselessmake no difference to the classification performanceimprove the classification performance

Vote, please!

Classifier ensemblesBrain-computer interfaceMathWorks productsMy scope of interestLiteratureChange may or may not cause trouble...Is there a change ?

Is there a change?

mean(moving average)mean 2stdchangesShewhart with threshold 2 sigmaYes!

Is there a change?

No!

Is there a change?

Is there a change?Yes, for the purposes of Spot the difference.

No, as this is a bee with a flower in the sun.Is there a change?

No!

No!Is there a change?sin(10x) * randnsin(20x) * randnYes!

change detection

Change does not exist out of context!ENTER Feature Extraction

Context: Amplitude variabilityFeature: AMPLITUDE

Context: Time series patterns in a fixed window.Feature: A PATTERN IN A FIXED WINDOW

Context: Childrens puzzleFeature: PIXEL B/W VALUE

Context: Frequency variabilityFeature: FREQUENCYsin(10x) * randnsin(20x) * randnSuppose that CONTEXT is not available.

Principal Component Analysis (PCA) captures data variability.

Then why not use PCA here?Labels are NOT availableData (all features)Feature EXTRACTOR

Distribution modellingChangestatisticthresholdChange/NO changeFeatures

PCA intuition: The components corresponding to the largest eigen values are more important

But is this the case for change detection?

Distributions are similar(small sensitivity to change)Distributions are different(large sensitivity to change)PC1PC2Holds for blind:

TranslationRotationVariance change...Kuncheva L.I. and W.J. Faithfull, PCA feature extraction for change detection in multidimensional unlabelled data, IEEE Transactions on Neural Networks and Learning Systems, 25(1), 2014, 69-80Some experiments:

Take a data set with n featuresSample randomly windows W1 and W2 with K objects in each window.Calculate PCA from W1. Choose a proportion of explained variance and use the remaining (low-variance) components.Generate a random integer k between 1 and n4(a)Shuffle VALUESChoose randomly k features. For each chosen feature, shuffle randomly the values for this feature in window W2. 4(b)Shuffle FEATURESChoose randomly k features. Randomly permute the respective columns in window W2. Transform W2 using the calculated PC and keep the low-variance components.Calculate the CHANGE DETECTION CRITERION between W1 and W2. Store as NEGATIVE INSTANCE (no change).

Some experiments:

Take a data set with n featuresSample randomly windows W1 and W2 with K objects in each window.Calculate PCA from W1. Choose a proportion of explained variance and use the remaining (low-variance) components.Generate a random integer k between 1 and n4(a)Shuffle VALUESChoose randomly k features. For each chosen feature, shuffle randomly the values for this feature in window W2. 4(b)Shuffle FEATURESChoose randomly k features. Randomly permute the respective columns in window W2. Transform W2 using the calculated PC and keep the low-variance components.Calculate the CHANGE DETECTION CRITERION between W1 and W2. Store as POSITIVE INSTANCE (change).

Run 100 times for POS and 100 for NEG to get the ROC curve for a given data set.Run 100 times for POS and 100 for NEG without applying PCA to get the ROC curve for a given data set.Use the Area Under the Curve (AUC), however disputed this might have become recently...Larger AUC corresponds to better change detection

VALUE shuffle

VALUE shuffle

FEATURE shuffle

FEATURE shuffle

PCA - use the least relevant components!? ConclusionChange detection may be harmful, beneficial or indifferent to classification performanceChange does not exist out of context, therefore GENERIC algorithms for change detection are somewhat pointless...Feature extraction for change detection may not follow conventional intuition.

1-34-6Can you detect an abrupt change in this picture?Remember my little puzzle?