Real Time Intracranial EEG Experimental Interface

Abstract

Electroencephalography (EEG) measures potential differences within the brain generated by the firing of neurons. The signal can be characterized by its frequency content into special frequency bands. It is hypothesized that certain frequency components of EEG correspond to heightened cognitive abilities. The 4-8 Hz frequency band (also known as the “theta” band)  has been linked to greater memory performance. To test this hypothesis, a Brain Computer Interface (BCI) was created that used recorded EEG activity to drive a real time memory retrieval experiment.

The current system was constructed as a platform for real time cognitive experimentation. An interface was created that allowed maximum control of experimental conditions. Two classifiers were used to characterize incoming EEG activity. The spectral classifier maximized the speed under which the experiment was conducted while the Neural Network Classifier used all available spatial EEG information. The classifiers were used to characterize the incoming EEG into two states: “theta” or “not theta”. Based on this binary classification, an experimental output was modified. In this case, the experimental output was the presentation of a word to the patient on a computer screen each time a “theta” event was discerned. The hypothesis is that words presented during high theta activity will be remembered by the patient at a greater rate than words presented randomly.

AuthorsJohn Burke EE ‘07Kylash Viswanathan EE ‘07Ronald Williams EE ’07

AdvisorsProf. Nabil FarhatDr. Kareem Zaghloul Prof. Michael Kahana

Special ThanksJoshua Jacobs, Neuralynx Inc.

Demo Times: Thursday, April 19, 2007

System Overview: The Top-Level System overview is given below. EEG signals from the patient are recorded through the Neuralynx system after which they are processed by a C# program. Once processed, the signal is further processed in Matlab and fed into the classifier. The binary output of the classifier then controls the experimental output – a word is presented to the patient on a separate computer screen.

System Flowchart:

• The system takes one of two pathways based on the mode of classification selected.

Network Classifier:• A three layer feed forward network was used for classification.

• The network Classifier went through two rounds of training before implementation within the system (block diagram lower left). First, back propagation altered the network weights and, second, the threshold for the network output was optimized by maximizing the d’ value of the classifier.

Feature

SelectionPCA

Error Back Propagation

D prime maximization

Training I Training II

• The d’ value maximized the hit rate of the classifier (correctly predicted theta events) and minimized the False positive rate (see figure above).

• The classifier output is given above. The red spikes are places where the output was above the threshold. A TTL pulse was sent at each of these points.

Spectral Classifier:• In contrast to the network classifier, the spectral algorithm used information from only one electrode to classify theta events. The classifier examined the Power Spectrum of the signal (see figure at left).

• If the ratio of the power in the 4-8 Hz band relative to the 2-4 Hz and 8-10 Hz bands exceeded a given threshold, the activity was classified as a positive theta event (see figure at right).

System Interface and Summary:• The user sets system variables and monitors the EEG activity through the real-time interface (shown at left).

•All plots are updated in real time; both the EEG signal and the theta ratio (or network output) scroll across the screen.

•Although the system specs were designed to detect theta events (4-8 Hz frequency), the interface was generalized to classify any EEG frequency band of interest.

Summary: The Spectral algorithm approximates a patient’s cognitive state using information from only one electrode. The advantage using this algorithm is speed – the system can iterate an entire processing cycle in less than 40 ms. Appropriate choice of electrode may justify this compromise. The network classification algorithm, while slower, can incorporate spatial information from all electrodes. The system is designed to give the user flexibility with the algorithm, selection of electrodes, and EEG frequency bands to generalize the system for a wide range of real-time experimental paradigms.

ABSTRACT: Data acquisition for the Primordial Anisotropy Polarization Pathfinder Array (PAPPA) project balloon flight requires constant communication between the flight telescope and an onboard computer. However, conditions at flight altitude cause computers to become unreliable. A redundant computer control system, consisting of two identical computers connected to the telescope through a watchdog circuit, aims to deliver continuous computer control over the telescope.

AUTHORS: Marc Blanco CTE ‘05

Shlomo Katz EE ‘05

ADVISOR:Dr. Mark Devlin

Professor of Physics

University of Pennsylvania

DEMO TIMES:Thursday, April 21th, 2005

9:30, 10:30, 11:00, and 11:30 AM

GROUP #4

UNIVERSITY OF PENNSYLVANIA

Department ofElectrical and Systems

Engineering

Redundant Flight Computer Control Systemfor PAPPA Balloon Telescope

SYSTEM OVERVIEW:Two Linux PC's are used to control and store data from external devices. Each computer sends and receives data via serial and parallel port interfaces, and can generate analog output voltages through a National Instruments card. The computers connect to the telescope through a watchdog circuit. Each computer sends and receives data as if it alone is in control of the system, but the watchdog chooses one computer whose control data is passed on to the telescope.

Baseband Analog Bursts

A screen capture of various signals at the moment when computer 1 fails. Shortly after failure, the computer is rebooted and telescope control is switched to computer 2 (whose control signal is 0V DC). Rebooting is triggered by grounding the reboot pin, which is seen here as a decrease in noise on (4).

(For demonstration purposes, the control signal here comes from a function generator, not from computer 1. Thus, (1) is active even after the computer fails.)

At this moment, computer 1 is being rebooted (one reboot light is on, one power light is off,

and control has been switched to computer 2)

Complete setup, involving watchdog circuit and two computers

WATCHDOG

status

restart

control

data

Telescope

Computer1

Computer2

status

restart

control

control

The watchdog circuit connects 16 telescope output lines to both computers, sends 28 input lines to the telescope from one of the computers, and continuously monitors the parallel port output from each computer. In the absence of a characteristic signal on one computer's parallel port, the watchdog reboots that computer and switches telescope control to the working computer. If necessary, extra watchdog circuit cards can be mounted on top of the primary card, and used to switch additional input and output lines based on the control decisions made by the primary watchdog circuit.

Comp.1

Comp.2

16x

alive?

alive?

reboot

reboot

decision

s0

s15

Watchdog

Ethernet

parallel + reboot

parallel + reboot

serial card

analog out card

serial card

analog out card

12x

16x

12x

ao11

ao0

shared GND

shared GND

shared GND

external power connector

alive?

s0

s15

To

telescope

pointing

sensors

and

pointing

controls

Ethernet

16x

12x

ao11

shared GND

shared GND

SYSTEM IN ACTION:

Comp. 1Monitor

Comp. 2Monitor

Power OnReset

Clock

CountdownTimer

CountdownTimerComp. 1

Comp. 2

Comp. 1Power

Comp. 2Power

SwitchoverRelays

Comp. 1Reboot

Comp. 2Reboot

Roughly 0.2s delay between computer failure (3), and reboot (4) and control switchover (2)

1) Computer control signal

2) Control signal as received by telescope

3) Computer “alive” signal

4) Computer reboot pin

WATCHDOG CIRCUIT:The monitor subcircuit examines the computer’s output for correctness. If the output is incorrect, the countdown timer issues a momentary failure signal. This induces the computer power subcircuit to reboot the computer, and allows the switchover relays to choose the correct computer’s output to be sent to the telescope.

Robodog Frontal Facial RecognitionAUTHORS GROUP 5:

Jing Hu EE ’05Jessica Pannequin EE ‘05

Chanatip Kitwiwattanachai EE’ 05

DEMO TIMES: Thursday, April 21st, 2005 9:30AM, 10AM, 1PM and 1:30PM

Skin Color Analysis

ADVISORS:Dr. Daniel LeeDr. Nabil Fahrat

Gray Scale ImageFirst Crop

Feature Extraction

Second Crop

Rotation

Trained Neural

Network

Picture from Dog

Facial Database

RecognitionAlgorithm

Dog Reaction Minimum

Error

University of Pennsylvania

Department of Electrical and Systems Engineering

ABSTRACT:

Facial Recognition has drawn a significant amount of attention in the research area in the past few years. There is an increasing interest in the implementation of facial recognition systems because of the emerging demands of more efficient security systems. The ability to take into account differences in lighting conditions, facial orientation and background objects is crucial for the implementation of a successful system. Many different approaches of the problem have been developed over the past two decades. So far, each proposed method has different comparative advantages and disadvantages.

With the chosen approach of this project, the face region is first extracted from the original picture using skin color analysis. The facial features are then generated from the face region. By doing so, the background noise can be eliminated, thus increasing the recognition accuracy and decrease the computation volume of the system.

The facial features are then fed into a neural network to overcome image distortion due to lighting condition, facial expression and orientation of the face.

Finally, in order to enhance the role of human-robot interaction for which recognition is a crucial capability, the Sony’s Aibo Dog is used as the interface to the system.

Recognition Results

FeatureVector

Three most prominent SIFT Features of random

picture from database

Neural Network

Pass features through theNetwork once and adjust the weights Wij and Wjk

Final Weights Wij and Wjk

Repeat for 150,000 iterations

After alliterations

Neural Network Training

Convergence of Neural Network weights during training

For each person, 400 pictures are stored in a database, with different facial orientations or expressions. At each iteration, a picture is taken at random from the database and passed once through the neural network. The network is trained over 150,000 iterations, which allows on average each picture to be trained 75 times.

First Image:Min Error1 < 0.005?

Second Image:Min Error2 < 0.005?

Third Image:Min Error3 < 0.005?

Yes

No

Yes

Min Error1

No

Recognition

Recognition

Yes Recognition

Min Error belongs to thesame person as in 1st Image?

No

YesRecognition

YesMin Error belongs to the same person as in 1st or 2nd Image?

No

No

Start over with new group of pictures

Min Error2

Min Error3 Does Min belongto pair of pictures?

Take Minimum of 3 Errors

NoIs Min Error of single

Picture < 0.05?

Yes

Sample Set

Recognition

Recognition

No

Yes

Recognition Algorithm

a group of pictures is analyzed. At each stage, a measure of the minimal error helps determine if an immediate conclusion is possible or whether an additional picture is required.

The neural network outputs five errors, each measuring the level of similarity with one of the individuals trained by the network. The lower the error, the closer the person is to one of the trained individuals. Therefore, the smallest error indicates who was recognized in a given picture.In order to improve the robustness of the system, the following algorithm is implemented. Instead of determining recognition based on a single picture,

Graphical UserInterface

JP JH CH FR SG

0

10

20

30

40

50

60

70

80

90

100

JP JH CH FR SG

95%88%85%84%86%

86% 88%95%85%84%

87.6% Rate of Success

Convergenceof 1 element

in the Wij matrix

Neural Network Outlinewith three layers

Convergenceof 1 elementin the Wjk

matrix

Convergenceof 1 elementin the Delta Wij matrix

Convergenceof 1 elementin the Delta Wjk matrix

Note: The Delta Wij and Delta Wjk matrices correspond to the adjustment to the Weights performed at each iteration. After successful training, these

numbers should converge to zero, as shown on the above graph.

Special Thanks to:

Paul Vernaza