team 2: mind readers

Team 2: Mind Readers

Krishna Jharjaria TJ Strzelecki Rick Schuman Matt Waldersen

Project Overview The Mind Reader is a mobile brain-computer interface. Computer applications will be presented to the user

through commercially available video glasses. An EOG and commercially available EEG will be

mounted inside of a common enclosure and will enable the user to navigate and select various applications.

A dsPIC microcontroller will be used to acquire the EOG and EEG signals, the EEG signals will be analyzed by an FPGA and a BeagleBoard XM will control the virtual reality environment as well as execute all of the computer applications

Project-Specific Success Criteria

1) An ability to encode/decode data packets from a NeuroSky EEG.

2) An ability for a user to select applications based on signals from a NeuroSky EEG.

3) An ability for a user to navigate between different applications on a display using EOG signals.

4) An ability for the system to interactively train the user to effectively operate the device.

5) An ability to display a live video stream from an external camera module, and integrate applications into the video system.

Block Diagram

Component Selection RationaleMicrocontroller

Considerations Signal Processing abilities Digital Communication pins Optimized for C compiler Processing speed Resources and reference material

DSPIC33EP512MU810 Extensive DSP Library with built in FFT function 4-UART; 4-SPI; 2-I2C Optimized for C compiler ~53K of RAM Large online community

EOG Op-Amp Selection Rationale (TLC2272/4 & TLV2211 )

Low Noise Dual Polarity Model Available in PSPICE Multiple Op-Amp Models Available in DIP and SMD

configurations With exception of the TLV2211

High Accuracy

EOG Op-Amp Selection Criteria

ADC Selection Rationale (ADS1210) High Resolution

19.5 Effective Resolution (Bits RMS) Differential input and 2.5V reference

output eliminates need for virtual ground

Integrated Programmable Gain Amplifier

Integrated Digital Filter Available in both DIP and SMD

packages

Considerations Size (Logic Blocks) Number of I/O Pins Built in Functionality Power Consumption

DLP-HS-FPGA3 USB - FPGA Module 25,344 Logic Blocks (11,264 slices) 63 Available user I/O channels 32 Dedicated Multipliers, libraries for floating point arithmetic FPGA module contains: regulators, clock generator, SDRAM,

USB programmer, SPI Flash, compatible with Xilinx ISE WebPack

FPGA operates on 3.3V, sinks/sources approximately 24mA per I/O pin

Component Selection RationaleFPGA Module

BeagleBoard-xM 1.0 GHz ARM Cortex-A8 Processor 512 MB RAM 8 GB microSD HDD ▪ libraries and main program

4 USB 2.0 Slots ▪ Mouse/Keyboard dev, Webcam

Expansion Header (Data Transfer)▪ o SPI▪ o UART▪ o I2C▪ o GPIO

Component Selection RationaleBeagleBoard xM

Packaging Design

Modular design Maintains separation of digital circuitry

and sensitive analog EOG circuitry Enables the EOG circuitry to be placed

as close possible to signal electrodes Video glasses will be used in order to

ensure that the video display is directly in front of the field of view

Light Weight Single Board Computer

Packaging DesignBeagleBoard-xM Size: 3.35” x 3.45” Weight: < 1 lb.

Logitech C310 HD Cam 1280x720p frame capture < 8 oz.

Vuzix Wrap 920 Twin 640x480p Display < 3 oz.

Main Circuit BoardEOG inputs

Power SupplyMicrocontroller

Interface

FPGA and Beagle Board

Power Supply

11.1V rechargeable Lithium ion Battery

5V convertorFor FPGA and Beagle Board

3.3 V convertorFor Microcontroller

2 Switch Mode regulators

EOG inputs

Connected to EOGvia SPI

Connected to EOG via SPI

Logic Level convertorFrom 5V to 3.3V

Registered Jack (RJ11)

Micro, in-system-programmer, EEG Dongle, Oscillator

Oscillator

In System Programmer

PCB mountReset

Through holeReset Connections

Switch debouncer

Pin 13 MCRL

Pin 15 PGECPin 16 PGED

EEG input via UART

UNIVERSAL ASYNCHRONOUS RECEIVER TRANSMITTER (UART)

Baud rate = 115200 bps Specified by the NeuroSky MindWave

EEG signals FFT EEG frequency

range 4-7hz

Array size 512

Output of the FFT resides in RAM

Inbuilt FFT API: FFTComplexIP() BitReverseComplex() SquareMagnitudeCplx()4-7hz

FPGA Module, Logic Level Converter, Beagle Board

Beagle Board

FPGA Module

Logic Level ConvertorFrom 3.3V to 5V

Select button

Switch debouncer

4 bit EOG outputUp, down, left, right

2x5 bit EOG input

4 I/O for EOG3 SPI pins for EEG

PCB LayoutDesign Considerations

Main Board Multiple Voltages needed Power Traces at least 40 mil, signal traces no smaller than

10 mil Isolate oscillator Physical location of connectors Decoupling Capacitors

EOG Board Sensitive analog circuitry Distance between electrodes and filtering/amplification Physical size of EOG PCB

PCB Layout

PCB Layout – Main BoardFGPA Module

PCB Layout – Main BoardFGPA Module

PCB Layout – Main BoarddsPIC Micro

PCB Layout – Main BoarddsPIC Micro

PCB Layout – Main BoardPower Supply

PCB Layout – Main BoardPower Supply

PCB Layout – Main BoardEOG/SBC Connection & LLT

PCB Layout – Main BoardEOG/SBC Connection & LLT

EOG Theory of Operation

Electrooculography measures the signal given off of corneo-retinal potentials in the eyes

An EOG circuit consists of an instrumentation amplifier, amplifiers, low pass filtering and high pass filtering

A high pass filter is used to eliminate a naturally occurring DC drift

A low pass filter is used to remove all other external noise

Instrumentation AmplifierInstrumentation

Amplifier High Input

Impedance Includes both a

high pass filter to eliminate DC drift and a second order low pass filter

Instrumentation Amplifier

Sallen-Key 2nd Order LPFSallen-Key 2nd-Order Low Pass Filter Eliminates noise from external sources

Sallen-Key 2nd Order LPF

First Order LPF - AmplifierFirst Order Low Pass Filter – Amplifier Circuit Eliminates noise as well as amplifies the signal

First Order LPF - Amplifier

Linear Regulator

Voltage Inverter

Analog-to-Digital Converter

Reference Ground & Input Jack

EOG Signal Conditioning

Voltage (V)

Time (s)

EOG Signal Conditioning

Voltage (V)

Time (s)

Normalizing EOG Signal

∆Voltage

Sample Number

Software Design/Development StatusArtificial Neural Network

What is an Artificial Neural Network? Simplified mathematical model of the human brain Utilizes a network of “neurons” to model relationships

between inputs and outputs How does a ANN work?

ANN implement non-linear functions in each neuron This function sums weighted input values and passes them

through a non-linear function, usually a Sigmoid function This output then propagates to further network layers for

the process to be repeated Why use an FPGA implementation

Highly parallel algorithm, with transistor like output Need for quick, complex calculations

Software Design/Development StatusArtificial Neural Network

Software Design/Development StatusArtificial Neural Network

Basics of A Neuron Each neuron contains a number of synapses, one for each

previous layer neuron connected A synapses will take inputs and multiply it by a predetermined

specific weight The weighted values will then be accumulated and funneled into

a nonlinear activation function (Sigmoid Function)

Hardware Design Considerations Need multiple multipliers, one for each synapses (38 multipliers)▪ Need to condense the size of a neuron▪ Using control logic and multiple clock cycles we can lower the

required multipliers to a single multiplier per neuron (12 multipliers) Implementation of non-linear Sigmoid Function▪ Look-up Table

Artificial Neural NetworkPreliminary Block Diagram

Software Design/Development StatusArtificial Neural Network

Where we proceed from here Training and testing of preliminary ANN using

Matlab ANN toolbox▪ This will allow the weighted values to be hardcoded into

the FPGA Design and testing of single neuron in VHDL Development of a proven working topology

Software Design/Development StatusHeads-Up Display

Tentative HUD Features

•Webcam Live Feed

•EEG Signal Data display

•Home Screen• 3 x 3 application layout• Concentration to select app

•Programs• Minimal key input• Closed apps return to Home

Software DesignSingleboard Program Flow Diagram

Project Completion Timeline

October 15-19 PCB Proof of Parts

October 22-26 Simple Software Suite Sample EOG circuit ADC microcontroller

communication October 29 – Nov 2

Micro to Single-board communication

EOG ANN finished in MATLAB

Nov 5 – Nov 9 Live Feed Visual Software

Suite EEG ANN prototyped in

MATLAB PCB assembled

Nov 12 – 16 EOG ANN finished on FPGA

Nov 19 – 23 Software Test Application User Training Application

Nov 26 – Nov 30 Troubleshoot

Dec 3 – Dec 7 Project Complete

Questions?

Reference (DSPIC33EP)

Reference (PIC Programmer)

Reference (RJ-12 Connection)

Reference (Oscillator/Reset)

Reference (DC Voltage Inv)

Reference (ADS1210 ATD)

Reference (TLC2272)

Reference (TLC2272)

Reference (TLV2211)

Reference(DLP FPGA Module)

Reference (MAX764-DC Inverter)