TB 6/22/06 1

Anomaly Detection for Prognostic Anomaly Detection for Prognostic and Health Management System and Health Management System

DevelopmentDevelopment

Tom BrothertonTom Brotherton

TB 6/22/06 2

New Stealth TechnologyNew Stealth Technology

TB 6/22/06 3

OutlineOutline

• What is Anomaly Detection– Different types of anomaly detectors

• Radial Basis Function Neural Net Anomaly Detector– The basics– Comparison with other neural net approaches– Feature ‘off-nominal’ distance measures– Training

• Implementations– Continuous = Gas turbine engine monitoring– Snap shot = Web server helicopter vibration condition indicators

• RBF NN & Boxplots• Application to detection of helicopter bearing fault• Application to monitoring fish behavior for water quality monitoring

TB 6/22/06 4

What is Anomaly Detection?What is Anomaly Detection?

• Anomaly Detection = The Detection of Any Off-Nominal Event Data– Known fault conditions– Novel event = New - never seen before data

• New type of fault• New variation of ‘known’ nominal or fault data

• What is ‘Nominal’– Sets of parameters that behave as expected

• Physics models• Statistical models

TB 6/22/06 5

ApproachesApproaches

Applicability

Acc

urac

y &

Cos

t Physics

Parametric- Estimate of physics

Empirical- Derived from collected data

•State Variable Models (derived from physics)

•JPL: BEAM (coherence = model of linear relationships)

•Neural nets (non-linear relationships)

•Academic: Support Vector

•Ex: Gas Turbine Engine Deck: Component level physics model

•Simple statistics

•Hybrid Model: Combine Physics + Empirical

•Fused empirical: BEAM + NN

TB 6/22/06 6

Empirical ModelingEmpirical Modeling

Collected ‘Nominal’ Data

IdeaIdea: Theoretical boundary (multi-dimensional ‘tube’) that data should lie within: - Nominal data is inside the boundary - Anomaly data is outside

Problem: How to estimate / approximate the boundary?Problem: How to estimate / approximate the boundary?

An anomaly

Problem: What measurement(s) caused the

anomaly?

Problem: What measurement(s) caused the

anomaly?

Problem: How far off-nominal is the anomaly / feature?

Problem: How far off-nominal is the anomaly / feature?

TB 6/22/06 7

RBF Neural Net Anomaly Detection: The IdeaRBF Neural Net Anomaly Detection: The Idea

• Dynamic data = Lots of NN basis units to model

• Piecewise stationary approximation

• Distance measure = Function of the signal set

• Individual signal distances from nominal = distance from “closest” basis unit

– Detection can be for set of signals when no single signal is anomalous

• The model can be adaptively updated to include additional data / known fault classes

• Trajectories of features relative to basis unit = Prognosis

= Sample of nominal data

= Sample of anomalous data

NN = Model for Nominal Data

‘Distance’ fromNominal Model

Yes

?

Radial Basis Function Radial Basis Function (RBF) Neural Net Model(RBF) Neural Net ModelRadial Basis Function Radial Basis Function

(RBF) Neural Net Model(RBF) Neural Net Model

TB 6/22/06 8

MLP NN

?

Why Use Radial Basis Function Neural Nets?Why Use Radial Basis Function Neural Nets?

• Radial Basis Function Neural Net– Nearest neighbor classifier– Distance metric : Measure “nominal”– Multi-layer perceptron (MLP) does not have these properties

RBF NN

?

TB 6/22/06 9

Support Vector MachineSupport Vector Machine

RBF Model

Support Vector Machine Model

• In some sense, much better model of ‘truth’ …. but- Automated selection of number of

basis units• Lots!

• Trade off between fidelity vs smoothness

• Not practical for on-wing• How to compute individual signal

distances• Loss of intuition

Training data

TB 6/22/06 10

NN = Model for Nominal Data

Distance s1

Distance s2

Feature Distance CalculationFeature Distance Calculation

?

MahalanobisMahalanobis

Nearest Neighbor Distance

TB 6/22/06 11

NN = Model for Nominal Data

Alternative Distance CalculationAlternative Distance Calculation

Truth

Closest Basis Unit

- Truth: Single Feature X = ‘Bad’-Report: Feature X = ‘OK’ & Feature Y = ‘Bad’

-Alternative Distance = Which Basis Unit gives the smallest number of individual off-nominal features -> Hamming Distance (from digital communications decoding)

TB 6/22/06 12

‘‘RBF’ NN ArchitecturesRBF’ NN Architectures

Is output for Nominal?

= 1 Yes> 1- Likely< 1- ?< 1- No 0< < <1

•••

Inpu

t fea

ture

s

Basis Units

Weights

Gaussian elliptical basis function : Fuzzy membership basis function :Rayleigh basis function :

DetectorOutput

= Gaussian Mixture ModelGood for magnitude spectral data

* Basis function is ‘matched’ to the data distributionFor those who like things fuzzy

TB 6/22/06 13

- Small number of clusters Small number of basis units Low False Alarms

Very general Missed detections

Too General ?

- Large number of clusters Good ‘tracking’ of data dynamics Large number of basis units

More sensitive to outliers More false alarms

Over Trained ?

Don’t know a-priori what are the ‘best’ settings

Training : Neural Net Architectures – How to Training : Neural Net Architectures – How to select parametersselect parameters

TB 6/22/06 15

M of N DetectionM of N Detection

Detection?False alarm?

Large scale factor

Small scale factor

• Trade off single point detection capability vs false alarm rate

Large Scale Factor / Small N- Short – high SNR anomalies

Small Scale Factor / Large N- Long – persistent – low SNR

anomalies

• Trade off single point detection capability vs false alarm rate

Large Scale Factor / Small N- Short – high SNR anomalies

Small Scale Factor / Large N- Long – persistent – low SNR

anomalies

4 points persist over time = detection

Only 2 points = false alarm

False alarms?

Idea: M of N detection allows one sample high false alarm rate – Then integrate over time to remove

Idea: M of N detection allows one sample high false alarm rate – Then integrate over time to remove

TB 6/22/06 16

AlternativesAlternatives

• This technique works well– Demonstrated by Pratt & Whitney for C-17 F117

applications• Transient engine operations

– Long time to train – lots of different types of transients– Model can become very complex

• Engine control system• On-wing memory and timing constraints

• Alternative– Combine equipment operating regime recognition with

anomaly detector– Ex: Identify steady operation and then take a snapshot of

the data• Simple statistics may suffice

TB 6/22/06 17

Example Gas Turbine OperationsExample Gas Turbine Operations

Regime recognition- Regimes:

• Transient Throttle up• Transient Throttle down• Steady state – B14 open• Steady state – B14 closed

Neural NetDetection

Neural NetDetection

Scale Signal

Off-NominalSignal

Distance

RegimeRecognition

Trained NNs

Neural NetDetection

Input Signal Vector

DetectionFlag

Neural NetSelect

Median Filter

Break the big problem in to a set of small problems

Break the big problem in to a set of small problems

TB 6/22/06 18

Anomaly Detection of Stationary Regime Anomaly Detection of Stationary Regime Detected DataDetected Data

• Web Server Implementation for Helicopter Vibration Data– Condition Indicators (CIs) = Features derived

from on-board vibration measurements

• Two types of problems:– Single CI for a component

• Simple statistics solution = Boxplot– Intuitive = Army user’s like it

• RBF neural net implementation as well

– Multi-CIs for a component• RBF neural net implementation

TB 6/22/06 19

On Board SystemOn Board System

Absorbers Hanger Bearings

Tail Gearbox

Intermediate Gearbox

TransmissionsEngines

Cockpit VMU

Advanced Rotor Smoothing / Engine

Diagnostics

• 18 Sensors Installed – Vibration• Automated Exceedance Monitoring using HUD data• Automated engine HIT, Max Power Check and exceedances• Complete aircraft vibration survey in under 30 seconds

Accelerometer

Tach Sensor

Other Connections

FWDLAT

FWDVRT

Main Rotor

MainD/S

IAC-1209Modern Signal Processing Unit

(MSPU)

USB Memory Drive

Cockpit Control Head

USB Download

Ethernet

+28VDC Power

Parameter Data

FWDSP

CPITVRT

CPITLAT

FWDXMSNVRT

FWDXMSNLAT

HB2

HB3

HB4

HB5

HB6

HB7

XSHAFT1

XSHAFT2

ENG1COMP

ENG1NOSE

ENG1AXIAL

ENG1LAT

ENG2COMP

ENG2NOSE

ENG2AXIAL

ENG2LAT

AFTLAT

AFTVRT

AFTSPCBOXOCFA

CBOXOCLAT

APU

AFTFANLAT

AFTXMSNVRT

AFTXMSNLAT

Configuration• 36 Vibration Sensors• 2 Speed Sensors• 1553 connection to HUD

CVR-FDRCVR-FDR

TB 6/22/06 20

Aircraft / Server Physical ConnectivityAircraft / Server Physical Connectivity

SCARNG

Deployed Unit

AARNG

INTERNET

AIRCRAFT OEMs

VMEPPARTNER

PC-GBS Facility

PC-GBS Facility

PC-GBS Remote

PC-GBS Remote

PC-GBS Remote

USB Memory Stick Data Download

Browser

Wireless link

TB 6/22/06 21

Aircraft / Server Logical ConnectivityAircraft / Server Logical Connectivity

-Army P-GBS-Army P-GBS

Support Team- e-mail notification- Fleet level reports

- Automated s/w upgrades

Aircraft Maintenance-Electronic help desk

- Automated data archive- Automated s/w upgrades

Fleet Statistics& Reports

Fleet Statistics& Reports

Help DeskHelp Desk

Data ArchiveA/C config filesData Archive

A/C config files

MDS ServerMDS Server

Help Training BaseElectronic Manuals

FAQs

Help Training BaseElectronic Manuals

FAQs

PrognosticsPrognosticsDiagnosticsDiagnostics

NetworkSecurityNetworkSecurity

AutomatedData ArchiveAutomated

Data Archive

AnomalyDetectionAnomalyDetection

Portable SystemPortable System

- Army F-GBS- Army F-GBSWeb ClientWeb Client

BrowserBrowser

Facility SystemsFacility Systems

AnomalyDetection

TB 6/22/06 22

Advanced Engineering on the WebAdvanced Engineering on the Web

The role of anomaly detection on the website is to detect and bring to engineering’s attention the MOST INTERESTING data = Something that has NOT been encountered before

- More normal data not really of interest

TB 6/22/06 23

Default based on boxplot statistics

User set

Single Feature Anomaly DetectionSingle Feature Anomaly Detection

BoxplotsBoxplots = Simple statistics - single feature anomaly detector. No Gaussian assumption, just counting points. They seem to work very well!

BoxplotsBoxplots = Simple statistics - single feature anomaly detector. No Gaussian assumption, just counting points. They seem to work very well!

TB 6/22/06 24

Threshold SettingThreshold Setting

TB 6/22/06 25

Anomaly AnalysisAnomaly Analysis

Summary of all aircraft

TB 6/22/06 26

The Raw DataThe Raw Data

TB 6/22/06 27

Gaussian Transformation DataGaussian Transformation Data

• Problem: How to select a “matched” basis function– Gaussian assumption? Usually violated!

• Statistical Model Fit– Transform data to be Gaussian

• Transformation stored and is part of the model– Almost always only a single basis unit is required!

• Works on single feature data• All processing “behind the scenes” done on transformed data

Original Transformed

TB 6/22/06 28

RBF Anomaly DetectionRBF Anomaly Detection

TB 6/22/06 29

RBF Anomaly DetectionRBF Anomaly Detection

TB 6/22/06 30

Case Study: Apache Swashplate Bearing Spectral Case Study: Apache Swashplate Bearing Spectral Server DataServer Data

• Anomalous data identified with RBF NN AD running on the Server– Aircraft was in Iraq

– Automatic email alert sent to users• “Evidence” sent as well

– Data reviewed by AED-Aeromechanics and IAC via iMDS website• Large peak in spectral data at 1250 Hz for tail #460

• Sidebands spaced at intervals corresponding to bearing fault frequencies

• Suspected bad swashplate bearing

Tail 460

0

1

2

3

4

5

0 2000 4000 6000

Frequency (Hz)

Ma

gn

itu

de

(g

)

Tail 460

Tail 986

Tail 460OtherA/C

Other A/C

Main SP Spectra

TB 6/22/06 31

Case Study Case Study Apache Swashplate BearingApache Swashplate Bearing

• AED-Aeromechanics acquired raw vibe data Apr 04 and received swashplate May 04 before aircraft was turned-in for D model conversion

• Swashplate disassembled by PIF per DMWR Aug 04

• Minor spalling, corrosion and broken cage discovered

• Additional algorithms developed from raw data and implemented into VMEP for release Sep 04

Broken Cage

Spalling/Corrosion

TB 6/22/06 32

Follow UpFollow Up

• Specific algorithms to identify this fault now included with the on-board system

• US Army now uses ‘on-condition’ information from the system to perform maintenance– True condition-based maintenance (CBM)

TB 6/22/06 33

Other ApplicationsOther Applications

IAC 1090 is a mobile, web-enabled automated biomonitoring system that utilizing the ventilatory and body movement patterns of the bluegill fish as a bio-sensor, much like a canary in a coal mine.

Sixteen Bluegills are placed in individual flow-through Plexiglas chambers. Each chamber is equipped with an individual water input and drainage system. By utilizing sixteen different Bluegills, the IAC 1090 samples more biosensors than any other system on the market resulting in lower false alarm rates.

All fish generate a micro volt level electric field. Each individual fish is monitored by non-contact electrodes suspended above and below each fish in a Plexiglas chamber.

The electrical signals generated by the fish’s normal movement is amplified, filtered and passed on via the internet to IAC’s Bio-Monitoring Expert (BME) software system for automated analysis.

Water Quality Bio-MonitorWater Quality Bio-Monitor

TB 6/22/06 34

BME is a neural network based expert system that provides for rapid, real time assessment of water toxicity based on the ventilatory behavior of fish. BME has shown excellent detection capabilities for toxic compounds with a low false alarm rate. False alarms, common in other similar systems, are typically generated by normal, non-toxic variations in the environment.

Automated data collection and management tools, user interfaces, and real-time data interpretation employing advanced (artificial intelligence) models of fish ventilatory behavior make BME easy to use.

Remote (Internet) access to IAC 1090 is provided through an easy-to-use graphical user interface. BME’s modular design provides users with the ability to reconfigure the system for different biomonitoring applications and biosensors

Water Quality Bio-MonitorWater Quality Bio-Monitor

TB 6/22/06 35

Questions?Questions?

Conference papers / case studies available at:

www.iac-online.comwww.iac-online.com