Remote Control of a Television Set via Thought Patterns

Dr. William Hunt, Project AdvisorJosh RosnerDrew PetryGary Grimm

George Melcer

Thursday, March 11, 2010

The MindLink System

• Manipulates television functions via user thought patterns

• Provides independence to handicapped and disabled persons

• Estimated development cost at $80,940; per unit cost will be approximately $500

EEG Headset 2.4 GHz WirelessHost PC

EmoState Analysis Software

USB to RS-232 SPB & URCFirmware

Design Objectives• Minimize system costs to maximize target market• Offer intuitive interface for non-technical users• Manipulate simple command set for IR (Infrared)

controlled devices• Detect EEG (Electroencephalogram) signals in real-

time to provide reliable thought pattern analysis• Utilize Emotiv API (Application Programming

Interface) in analysis software• Interface with modified off-the-shelf remote control

Current Status

• EEG Headset– Selected and purchased headset– Commenced training with one engineer

• Host PC / EAS– Designed Pseudo code and learned about

API

• SPB /URC– Completed Circuit Design and Board Layout

EEG Headset

EEG Headset 2.4 GHz WirelessHost PC

EmoState Analysis Software

USB to RS-232 SPB & URCFirmware

Illustration

Input Propagation

Emotiv Headset

USB DongleHost PC Device Driver

EEG Signals From User

14 Sensors

2.4 GHz Wireless

Emotiv Control Panel

Cognitive Suite

Expressive Suite

EEG Headset - Problems / Issues

• Problem: Consistency of EEG signal from user• Proposed Solutions

– More training– Alter sensitivity settings– Use Expressive Suite

• Problem: Placement of headset• Proposed Solutions

– Need consistent signals to test– Seek advice from Emotiv online community

Host PC: EmoState Analysis Software

EEG Headset 2.4 GHz WirelessHost PC

EmoState Analysis Software

USB to RS-232 SPB & URCFirmware

Software ProcessEmoEngine (EDK.dll)

Emotiv EPOC

Headset

EEG and Gyro Processing

Control Logic

EmoStateBuffer

EmoState and EmoEvent query

handlingEmotive

API

C#MindLinkControl

Software

SynergyBoard

Software LogicOpen

connection to EmoEngine

Retrieve current

EmoState

New EmoEngine

Event?

ClassifyEvent

Yes

No

CognitiveEvent?

No

Satisfies criterion for

action?

YesNo

Send command to Synergy board

Yes

Close connection to EmoEngine

Init

Proc

ess

Loop

Term

inat

ion

Termination condition?

No

Yes

EAS – Problems / Issues

• Problem: Invoking and communicating with Emotiv API

• Proposed Solutions– Should include EDK.h and link with EDK.dll– Use EE_EngineConnect for communication with headset; EE_EngineRemoteConnect used for communication with EmoComposer (debugging)

• Problem: Sending commands to Synergy board• Proposed Solutions

– Propose communication of serial connection– Invoke .NET System.IO.Ports namespace and use instance of SerialPort class

Synergy Prototyping Board &

Universal Remote Control

EEG Headset 2.4 GHz WirelessHost PC

EmoState Analysis Software

USB to RS-232 SPB & URCFirmware

Synergy Prototyping BoardData via

Serial Class from EAS

USB to Serial/UART

(FT232R)

Microcontroller(ADuC7024)

Multiplexer(74HC154)

Quad, Bi-Lateral Switch Package

(CD4016)

Quad, Bi-Lateral Switch Package

(CD4016)

Quad, Bi-Lateral Switch Package

(CD4016)

Quad, Bi-Lateral Switch Package

(CD4016)

URC/TV

Synergy Prototyping BoardLow-Dropout

Regulator(LT1963)

5 Volts

OUTPUT HEADERS

OUTPUT HEADERS

SPB Layout

Synergy Prototyping Board FirmwareData via

Serial Class

from EAS

Set Clock to 41.78Mhz

Set registers to configure pins as

GPIOs

Initialize RS232 Communication

Link

URC/TV

Synergy Prototyping Board

Wait for incoming data

(blocking)

Fill char* buffer

Parse ASCII data into two integers

Execute Data

Initialization

SPB – Problems / Issues

• Problem: MOSFETs did not interface with URC as expected

• Proposed Solution– Opted for bi-lateral switches instead of MOSFETs because

we are open/closing a circuit, NOT pulling the signal high or low

• Problem: Simulation of SPB requires too much time• Proposed Solution

– Use “trial and error” approach with physical implementation of the board

• Problem: Soldering joints are weak on URC• Proposed Solution

– Use RTV silicone to isolate joints and prevent breaks

Acceptance Testing

• EPOC Headset– Train and test with Emotiv Control Panel

• EmoState Analysis Software– Utilize EmoComposer, printf stream, and debug symbols

• Synergy Prototyping Board & Universal RC– Component Testing (Switches, Microcontroller, etc.)– Board Testing (LED Blink Program)

• MindLink System– Iterate through defined thought patterns

EEG Headset 2.4 GHz WirelessHost PC

EmoState Analysis Software

USB to RS-232 SPB & URCFirmware

Projected Schedule

• March 19th

– EAS Development Complete– SPB Firmware Development Complete

• April– 16th: System Integration Complete– 30th: Final Project Summary Complete

• May 6th – Final Report Complete– Successful Project Demonstration

Budget & Cost AnalysisMindLink Venture

• Prototype– Initial Capital Expenditure: $750– Labor Costs: $80,000– Per Unit Component Cost: $500

• 10,000 Units Production over 60 Months– Total Costs: $3,700,000– Total Sales ($500 per unit): $5,000,000– Net Profits: $1,300,000

Future Work• Headset

– Complete Emotiv Control Panel Training• EmoState Analysis Software

– Port pseudo code to C#– Develop and test serial port interface class– Test and debug

• Synergy Prototyping Board– Order PCB and generate Gerber Files– Order Additional Components– Develop Firmware– Test and debug

• MindLink System– Complete component integration– Test and debug

Questions / Feedback

• EmoState Analysis Software– How to interface C# with serial ports?

• Synergy Prototyping Board– Alternative Solutions?