design of hands-free system for device manipulationwbs # task name start finish pred. 1 1 general...

Design of Hands-Free System for Device

Manipulation

Kassidy Kenney, Kimberly Harrington, Muhammad Sungkar, Angelo Huan

GDMS Sr Engineer Mike DeMichele

Current System:

Future System:

Project Overview

Technological Readiness Level : 3

Based on DOD TRL Definitions [http://www.acq.osd.mil/chieftechnologist/publications/docs/TRA2011.pdf]

“Active R&D is initiated. This includes analytical studies and laboratory studies to

physically validate the analytical predictions of separate elements of the

technology. Examples include components that are not yet integrated or

representative.”

2

AGENDA

● Context

● Interface Methods

● Stakeholders

● Problem Definition

● Current System Model

● Requirements

● Project Management 3

Physically Disabled Statistics

4

From 2012 CDC Survey of American Adults

Human- Machine Interaction

5 http://www.fda.gov/MedicalDevices/DeviceRegulatio

nandGuidance/HumanFactors

● Process driven by input from human user

● Relies on interface to continue data exchange

Challenges of Human- Machine Interaction

● Systems requiring a huge amount of user input

● Time sensitive systems

● Systems requiring user accuracy

● Systems requiring frequent operator turnover

Human Machine Interface

- The space where human- machine interaction occurs.

6

Source: Encyclopedia of Life Support Systems; http://www.eolss.net/sample-chapters/c18/e6-43-37-06.pdf

Brain-Machine Interface (BMI)

Direct communication pathway between the brain and an external device

Types:

Invasive - placed in the grey matter of the brain

Best signals quality

Risk of scar tissue on the brain

Partially Invasive - in the skull but not in the grey matter of the brain

Better signal quality

Noninvasive - uses external equipment

Poor signal quality

EEG is the most studied non invasive method due to low cost and portability

7

Human Motor Control

Natural Control Systems- Work to mirror the human control system

Human Brain

- Made of 100,000,000,000 neurons

- Neurons connect via synapses

- Neurons generate spikes of electrical activity,

- Can be seen on outer folds of cortex

8 Source: Canadian Institutes of Health Research, Institutes of Neurological, Mental Health and Addiction

EEG (Electroencephalography)

Your brain emits electrical impulses

all the time

Current applications are mostly in

the medical field; prosthetics,

stress evaluation, sleeping/brain

abnormalities and detection

9 http://www.mayoclinic.org/~/media/kcms/gbs/patient%20consumer/images/2013/08/26/10/03/my00296_im03978_bn7_eeg_resultsthu_jpg.png

EEG - How it works

EEG is a non invasive method to detect brain activity by measuring the

electrical activity along the scalp using electrodes

Event related potentials (ERP)

Correspond to a brain response related to a sensory, cognitive or motor event

Evoked potentials (EP)

Responses to stimuli.

The key in using EEG as a control method is to detect the desired ERP from the

many other ERPs and map it to a control

Poor signal quality due to being outside of the skull

10

EMOTIV - Our EEG Headset Manufacturer

Founded in 2011 and based in Southern California

Bioinformatics company whose main goal is to enhance the world’s understanding

of the human brain

Market leaders in their field

EMOTIV headsets come in many varieties and can be used for an array of

applications: interactive television, gaming, research, hands-free control system,

medicine, art, robotics, and security

11 https://www.ted.com/talks/tan_le_a_headset_that_reads_your_brainwaves?language=en

AGENDA

● Context


● Stakeholders



● Requirements

● Project Management

12

Hands-Free Control System Applications

● Reduce user workload in highly

involved human-machine interfaces

● Improve quality of life for disabled

persons

○ ie control system for a wheelchair

● Additional option in a multimodal

interaction interface

13

http://image.slidesharecdn.com/hciabdo2011-110310084613-

phpapp01/95/multimodal-interaction-an-introduction-25-728.jpg

Alternative Hands Free Control Technology

•Eye Tracking Software

•Voice Control

•Muscle contraction detection

14

http://www.elsevier.com/connect/wp-content/uploads/2013/01/656_eye-tracking1-

1024x742.jpg

AGENDA

● Context


● Stakeholders



● Requirements


15

Stakeholders

Military

Can help diagnose soldiers prone to seizures

Potential improvement in prosthetics

Potential integration with equipment

Medical Institutes

● Can be used in prosthetics and brain scans

● Set guidelines for human-machine interface

16

Stakeholders (Continued)

Industries

Create products that are compatible with EEG

Insurance

Change policies regarding EEG

Create insurance for EEG-integrated products

General Public

Consumers

17

Stakeholders Interaction Diagram

18

Stakeholder Analysis

19

Gap Analysis

The 14% of the American population with reported difficulty

performing basic motor function need an alternative to the physical

control systems found in almost every human-machine interface.

Even while technology is available to help these persons to perform

everyday tasks, the operation of this technology is reliant on the use

of a physical control interface.

20

AGENDA

● Context


● Stakeholders



● Requirements


21

22

Problem Statement

Physically disabled people need an alternate control system to allow

them to control a device without physical movement. The control

system can be applied to the operation of a wheelchair, robot or

computer in an effort to substantially increase the safety, quality of

life and independence of a physically disabled person.

Need Statement

Applications for hands-free control systems have the potential to

advance the quality of life for disabled persons and reduce user

workload exhibited by traditional control systems.

23

System Boundary Diagram

24

AGENDA

● Context


● Stakeholders



● Requirements


25

EPOC Cognitive Suite

1)Create user account

2)Train neurological signal to a basic motion

3)Repeat thought to perform signal

4)Map signal to another vehicle motion

26 Source: Emotiv EPOC

EEG Headset Alternatives

Standard EPOC

14 EEG channels and two references for accurate spatial resolution

High performance wireless device

iOS and Android compatible

EPOC+

9 axis-inertial motion sensors, Bluetooth capabilities, additional applications are enabled

Raw Data Option

Includes EEG firmware that allows the raw EEG data stream and marker events in

TestBench software

27

Physical System Diagram

28 Source:Brain-Computer Interfacing: Proposal of a Shaping System

AGENDA

● Context


● Stakeholders



● Requirements


29

CONOPS

30

> EEG headset which will be connected to export data to a computer in real-time.

Set-Up

>First time user, prompted to create a library of commands

> User will be shown a specific action which the device should perform as well as the mental command

that will map to the action. The user will be recorded focusing on this action several times, before moving

on to the next action - command pair.

> System will analyze this “training data” to create a dataset of the user’s average brain wave form for

this command. The average dataset, associated action and associated command will be stored in a

database library unique to the current user. As the library is developed, the number and range of

commands available to the user will increase.

Operation

When a user operated the system, they will use commands which they have previously trained simply by

focusing on the command that was recorded.

Project Requirements

MR: Mission Requirements

FR: Functional Requirements

TR: Technology Requirements

31

32

Mission Requirements

MR.1 The HFCS shall control a device in 3-dimensions (up/down, forward/back/left/right)

MR.2 The HFCS shall control a device in all 3 dimensions to an accuracy of xcm

MR.3 The HFCS shall be worn by an operator with head size min = X cm, max = Xcm

MR.4 The HFCS shall weigh no more than x lbs.The HFCS shall cost no more than $X.

MR.5 The HFCS shall not injure the operator or impair result in damage to cognitive functions.

MR.6 The HFCS shall employ only noninvasive technology.

MR.7 The HFCS shall not impair the operator's ability move their limbs or neck

MR.8 The HFCS shall not impair the operator's ability move from one location to the next.

MR.9 The HFCS shall allow the user to create a new command.

MR.10 The HFCS shall operate for a minimum of 3 hours, before recharging is required.

Mission Requirements (Cont.)

33

MR.11 The system shall perform action within X seconds of user command.

MR.12 The system shall correctly identify X% of the user commands.

Functional Requirements

34

FR.1 The HFCS shall record signals from the operator's brain.

FR.2 The HFCS shall isolate control event-related potential from electrical noise.

FR.3 The HFCS shall identify patterns in electrical signal.

FR.4 The HFCS shall map a unique electrical signal to given command.

FR.4.1 The HFCS shall generate speed control commands.

FR.4.2 The HFCS shall generate steering control commands.

FR.5 The HFCS shall convert user electrical signals into digital control signals.

FR.6 The HFCS shall encode the digital signal for RC communication.

FR.7 The HFCS shall send the RC signal.

Functional Block Diagram

35

Functional Hierarchy Diagram

36

DRAFT - Technology Requirements

37

TR.1 The HFCS shall use a windows 7 laptop as a server.

TR.1.1 The HFCS shall use a windows 7 laptop with Python 3.4.

TR.1.2 The HFCS shall use a windows 7 laptop with at least 4GB of RAM

TR.2 The HFCS shall use a bluetooth receiver.

TR.3 The HFCS shall use a passive electrode EEG headset.

AGENDA

● Context


● Stakeholders



● Requirements


38

Upcoming Week

Continue Requirements and CORE model

Displacement control versus derivative control systems

Analysis use of virtual simulator instead of RC car for prototype

39

Statement of Work

Objectives:

● Subset of brain signals mapped to the specific thought which generated the signal

● Documented process to identify and map new brain signals

● Increase value of a hands-free application

● Quality deliverables and products produced

● Improved realization of hands-free control systems

Scope


● Documentation

● Data Collection & Analysis Process

● Test Plan & Criteria

● Delivery of a Prototype

● Test Results

● Utility Analysis

● Plan for future expansion

40

41

WBS

42

Schedule - Part 1 of 4 MS

Proj

#

WBS

#

Task Name Start Finish Pred.

1 1 General Research Mon 8/31/15 Mon 9/7/15

2 2 Problem Definition

3 2.1 Preliminary Research Mon 9/7/15 Sun 11/1/15

4 2.2 Context Tue 9/8/15 Tue 9/29/15 1

5 2.3 Problem Statement Wed 9/30/15 Tue 10/6/15 4

6 2.4 Need Statement Wed 9/30/15 Tue 10/13/15 4

7 2.5 Stakeholder Analysis Wed 9/30/15 Tue 10/13/15 4

8 2.6 Gap Analysis Wed 9/30/15 Tue 10/13/15

9 2.7 Win-Win Wed 9/30/15 Tue 10/13/15

10 2.8 Systems Boundary Diagram Thu 10/1/15 Wed 10/7/15

Schedule - Part 2 of 4

43

11 3 CONOPS

12 3.1 Requirements Wed 10/7/15 Tue 10/20/15 5

13 3.2 Operational Scenario Wed 10/21/15 Tue 10/27/15 12

14 3.3 Analysis for areas of

improvement

Wed 10/21/15 Tue 10/27/15 12

15 3.4 Alternatives Wed 10/21/15 Tue 10/27/15 12

16 4 System Architecture Sat 10/24/15 Sun 10/25/15

17 4.1 Define Functional Architecture Wed 10/21/15 Tue 10/27/15 12

18 4.2 Define Allocated Architecture Wed 10/21/15 Tue 10/27/15 12

19 4.3 Define Interface Architecture Wed 10/21/15 Tue 10/27/15 12

20 4.4 Define Physical Architecture Wed 10/21/15 Tue 10/27/15 12


44

21 5 Model

22 5.1 Prototyping Wed 10/28/15 Tue 11/17/15 20

23 5.2 Develop test plan Wed 10/21/15 Tue 10/27/15 12

24 5.3 Perform test activities Wed 11/18/15 Tue 11/24/15 22

25 5.4 Model Accuracy Wed 11/25/15 Wed 11/25/15 24

26 6 Analysis

27 6.1 Perform Value analysis Thu 11/26/15 Wed 12/2/15 25

28 6.2 Perform Utility vs Cost analysis Thu 11/26/15 Wed 12/2/15 25

29 6.3 Provide recommendations Thu 11/26/15 Wed 12/2/15 25


45

30 7 Project Management

31 7.1 Project Budget & Tracking Mon 8/31/15 Tue 9/15/15

32 7.2 Project Schedule & WBS Mon 8/31/15 Tue 9/15/15

33 7.3 SOW Wed 9/16/15 Wed 9/16/15 32

34 7.4 Weekly tasks Mon 8/31/15 Sat 12/12/15

35 8 Documentation

36 8.1 Interim Briefings Wed 10/21/15 Mon 11/9/15

37 8.2 Final Presentation Mon 3/28/16 Tue 5/10/16

38 8.3 Final Report Mon 3/28/16 Tue 5/10/16

39 8.4 IEEE Paper Tue 3/1/16 Wed 3/30/16

40 8.5 Poster Mon 3/28/16 Tue 5/10/16

41 8.6 Proposal Presentation Wed 10/21/15 Wed 12/9/15

42 8.7 Proposal Final Report Wed 10/21/15 Wed 12/9/15

Critical Path Tasks

Current Critical Path Analysis, performed in MS Project, returns the follow tasks as

critical path tasks:

2.3 Problem Statement

3.1 Requirements

4.4 Physical Architecture

5.3 Perform Test Activities

8.2 Final Presentation

8.3 Final Report

46

Project Budget

● Labor cost - $40/ hr

● Hourly rate - $85.40

○ GMU Overhead rate 2.13

● Planned hours - 1,576

● Equipment cost - $400

● Planned Budget = $134,990.00

47

Project Risks

48

Risk # Foreseeable Risk Mitigation Strategies

1 Delivery of Equipment 1a. Order equipment asap

2 Data Collection - quality of

equipment, data quality

2a. Careful review of equipment reviews

before ordering

2b. Research data collection techniques

3 Access to raw data 3a. Contact equipment company

3b. Find contacts within Mason community

4 Data analysis timeline

driven by data collection/

equipment

4a. Develop test plan in line with system

architecture

4b. Closely monitor progress towards project

completion.

Questions?

49

design of hands-free system for device manipulationwbs # task name start finish pred. 1 1 general...

Documents