it10 0273 70

34
Smart SMS-SMS Application Management Platform BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK K.W.S.D. Kaluarachchi (IT 10 0273 70) Degree of Bachelor of Science in Information Technology Department of Information Technology Sri Lanka Institute of Information Technology October 2013 H. H. Rajamanthrie IT 10 0296 64

Upload: gamindu-udayanga

Post on 29-Nov-2015

33 views

Category:

Documents


0 download

TRANSCRIPT

Smart SMS-SMS Application Management Platform

BRAIN COMPUTER INTERFACE APPLICATION

FRAMEWORK

K.W.S.D. Kaluarachchi

(IT 10 0273 70)

Degree of Bachelor of Science in Information Technology

Department of Information Technology

Sri Lanka Institute of Information Technology

October 2013

H. H. Rajamanthrie IT 10 0296 64

Smart SMS-SMS Application Management Platform

BRAIN COMPUTER INTERFACE APPLICATION

FRAMEWORK

K.W.S.D. Kaluarachchi

(IT 10 0273 70)

Dissertation submitted in partial fulfillment of the requirements for the degree

of Science

Department of Information Technology

Sri Lanka Institute of Information Technology

October 2013

H. H. Rajamanthrie IT 10 0296 64

Declaration “I declare that this is my own work and this dissertation does not incorporate without

acknowledgment any material previously submitted for a Degree or Diploma in any

Other University or Institute of higher learning and to the best of my knowledge and

belief it does not contain any material previously published or written by another

person except where the acknowledgment is made in the text.

Also, I hereby grant to Sri Lanka Institute of Information Technology the nonexclusive

right to reproduce and distribute my dissertation, in whole or in part in print, electronic

or other medium. I retain the right to use this content in whole or part in future works

(such as articles or books) “

Signature: Date: 2013/10/23

The above candidate has carried out research for the B.Sc Dissertation under my

supervision.

Signature of the supervisor: Date: 2013/10/23

i

Brain Computer Interface Application Framework

Acknowledgement We take this opportunity to express our deep sense of gratitude to those who contributed to either our collective or individual efforts.

Our heartfelt thanks go out to the supervisor of our project, Dr. Rohana Priyantha Thilakumara and the co-supervisor, Mr. Darshika Koggalahewa for the kind patience, guidance and constant support rendered to us at all the stages of the project, right from the very be- ginning of the project.

We would also like to thank the lecturer in-charge, Mr. Jayantha Amararachchi who provided the required lecture materials and the necessary guidance to do the project.

The team extends sincere gratitude to all colleagues, who forwarded their enthusiastic ideas during the requirements gathering phase of the project. The teammates would also like to thank all the staff members of the SLIIT Malabe campus for their valuable suggestions and opinions that helped us to do the project.

Finally, we thank all who lend their kind support; friends and families who continued to give their insights, patience, support and co-operation which motivated us in reaching greater heights.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY ii

Brain Computer Interface Application Framework

Abstract Brain-Computer Interface (BCI) technology is a potentially powerful communication and control option in the interaction between Human and Computer systems. A Brain Computer Interface (BCI) is a direct communication pathway between the brain and an external device. This technology enables signals that are generated in the brain to control an external activity such as control of a keyboard of a computer or control of a wheelchair. Over the past few years, new games have been developed that are exclusively for use with an EEG headset by companies like Neurosky and Emotiv. In this project we have developed a set of programs that can be controlled by brain signals. In this report I explain about the Ludo Game which is controlled by Signals generated inside the Brain. Our research part is to find out about an effective and an efficient way to use to brain signals for this application. This Ludo Game is going to be a new experience and will define a new dimension in the Game industry. In another way, this can be an exercise for the Brain.

ilitated in game developing. By proposing this frame work we try break down barrier between brain computer interface researchers and game development community. Our proposed framework will facilitate to professional game designers as well as independent gaming enthusiast to dive in to the depth of Brain Computer Interfacing gaming world without bothering about brain wave analyzing, feature extraction and classification. Furthermore they wouldn't want to brother about devices without considering devices they would be able to develop their creative gaming ideas. The objective of the proposed project is to bring expert knowledge of Neurophysicists toward the game developers and motivate them to develop more exciting and challenging games and expand the gaming industry.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY iii

Brain Computer Interface Application Framework

Table of Contents DECLARATION 1

ACKNOWLEDGEMENT II

ABSTRACT III

LIST OF FIGURES V

LIST OF ABBREVIATIONS VI

1 INTRODUCTION 1

1.1 BACKGROUND CONTEXT 1 1.2 RESEARCH PROBLEM TO BE ADDRESSED 2 1.3 RESEARCH QUESTIONS 3

2 CONTENT 4

2.1 ADDRESSING THE LITERATURE 4 2.2 METHODOLOGY 5

2.2.1 Overview 5 2.2.2 Overview of the System Design 7 2.2.3 User Characteristics 7 2.2.4 Product Functions 9 2.2.5 Tools and Technologies 10 2.2.6 Product Constraints 11 2.2.7 Assumptions and Dependencies 11

2.3 RESEARCH FINDINGS 12

3 RESULTS AND DISCUSSION 18

4 CONCLUSION 24

5 REFERENCES 25

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY iv

Brain Computer Interface Application Framework

LIST OF FIGURES

Figure 1: System Diagram ........................................................................................................ 7 Figure 2: Student A – Attention Level Values ........................................................................ 14 Figure 3: Student A – Variance Between Attention Level Values .......................................... 14 Figure 4: Student B – Attention Level Values ........................................................................ 15 Figure 5: Student B – Variance Between Attention Level Values .......................................... 15 Figure 6: Student C - Attention Level Values while listening to a Song ................................. 17 Figure 7: Student D - Attention Level Values while listening to a Song ................................. 17 Figure 8: Attention Level Values ............................................................................................ 18 Figure 9: Running Average of Attention Level Values ........................................................... 19 Figure 10 - The simple keyboard ............................................................................................ 20 Figure 11 - Using the Simple Keyboard. ................................................................................. 21 Figure 12 - The Advance Keyboard. ....................................................................................... 22 Figure 13: Ludo Game User Interface .................................................................................... 23

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY v

Brain Computer Interface Application Framework

List of Abbreviations

EEG Electro - Electroencephalogram

BCI Brain Computer Interface

fMRI functional Magnetic Resonance Imaging

EEC Encephalogram

SSVEP Steady State Visual Evoked Potential

ALS Amyotrophic Lateral Sclerosis

API Application Programming Interface

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY vi

Brain Computer Interface Application Framework

1 INTRODUCTION

1.1 Background Context

The advent of computers was a significant breakthrough in this area of work, as it lead

the way towards what we today know as “Brain Computer Interface (BCI)”. The

computational processing aspect of computers now opened up doors to the possibility

of being able to analyze and distinguish the patterns of the electrical signals being

produced by the human brain, and thus allowing to trigger a desired outcome for a

given identified brain signal pattern. For example, when a brain signal pattern is

detected by the computer then it could be associated to trigger a tangible action like

activating a switch to turn on/off the lights. In simple terms this kind of a setup that

detects electrical signals from the brain, and computes these signal patterns to initiate

a substantial tangible action is known as a BCI.

BCI has always been a subject matter of interest in the fields of Rehabilitation and

Assistive Technology. The potential of BCI is unbounded with regards to improving

the lives people with disabilities. For example, BCI based systems could allow an

individual with severely restricted or no range of movements (e.g. spinal cord injuries)

to drive a power wheelchair, operate household appliances, etc. Or even help someone

in a vegetative state to communicate by speaking out the words that the individual

would like to say.

Primarily there are two categories of BCI systems – “invasive” and “noninvasive”.

Invasive systems interact with the brain directly via electrodes / sensors that are

implanted into the brain or its surface. While noninvasive systems interact with the

brain indirectly via electrodes / sensors placed on the surface of the head that detect

brain signal emissions (e.g. Electro-Encephalography (EEG), functional Magnetic

Resonance Imaging (fMRI), and Magnetic Sensor Systems).

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 1

Brain Computer Interface Application Framework

Noninvasive BCI systems usually involve wearing a head cap (aka EEG cap) with

multiple holes / slots to put on the electrodes at the relevant areas of the surface of the

head to detect and record the electrical signals emitted by the brain. Electro gel is used

on the electrodes to improve contact between the scalp and the electrode. Due to the

requirement of a gel such electrodes are also known as wet-electrodes. Existing

systems can have anywhere between a few to more than a 100 electrodes. The

practicalities of using wet electrodes, as an example drying up of gel, repeated cleaning

of electrodes and head skin to setup EEG Cap, irritation of sensitive skin due to

application of gel, etc. – does not make them convenient for quick setup and daily use.

Over the recent few years this has prompted the development of dry electrodes. Unlike

wet electrodes, dry electrodes do not require the use of gel and can be setup direct into

the EEG Cap. Although still in the infancy of its developmental cycle, dry electrodes

are quickly catching up in terms of detecting high quality brain signals as their wet

counterpart. Regular comparison studies are being carried out to evaluate the

performance of Wet vs. Dry Electrodes within the context of EEG based noninvasive

BCI.

1.2 Research Problem to be addressed

When it comes to Brain Computer Interface technology, the development in this field

is still less. There are few device that uses BCI technology and those have many

limitations. And another problem is using the signals that are given by the BCI devices

in a useful way. In our research we are addressing this problem of using the Brain

Signals that gets captured by a BCI device in a useful way.

It is very difficult to use brainwaves to control BCI games, because each user will have

a different distribution of brainwaves. For example, some users’ brainwaves will have

a bigger amplitude than others. Therefore, we require to address this and find a suitable

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 2

Brain Computer Interface Application Framework

method that transfers the brainwaves to the signals that are good enough to be used in

a BCI game.

We checked and tested few methods to choose a best way to use the signals. We

collected many signal samples using few SLIIT students. Then calculated them using

our methods and chose the best method.

Fourth, a systematic BCI serious game development process is needed for experts and

game developers dealing with brain functions and brainwaves. This will allow diverse

developers to quickly and easily produce games together.

1.3 Research Questions

1. The Brain Signals from person to person are different from each other. How to

handle it?

2. There can be problems in signal strength. How to handle it?

3. What is the age range and duration of brains waves taken from a person?

4. How the brain waves are collected accurately without any error occurred?

5. How the accuracy of collected brain waves is tested?

6. How the BCI Device is used in suitable Application?

7. How the BCI application is developed to suit anyone at different age ranges

and the gender?

8. How the BCI device is utilized in efficiently?

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 3

Brain Computer Interface Application Framework

2 CONTENT

2.1 Addressing the Literature

In what research has been done so far, the field of computer and information

technology has developed a wealth of devices that capture brain signals. And there are

applications that use them. These applications do not have a proper way to use the

brain signals that gets captured by the device. In our research we test and suggest a

proper method to use the captured signals.

The current applications have faults in signal handling. The brain signals gets vary

from person to person. The strengths of signals get dropped during occasions.

In the applications we create, we use Running Average. The values that are given by

the device are sent through a program function which calculates the Running Average

of those values. And then Running Average values are sent to the game. Because of

this we can avoid the big variances in the signals.

What is Running Average?

In statistics, a moving average, also called rolling average, moving mean, rolling

mean, sliding temporal average, or running average, is used to analyze a set of data

points by creating a series of averages of different subsets of the full data set.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 4

Brain Computer Interface Application Framework

2.2 Methodology

2.2.1 Overview

Not only as an assistive technology, Brain Computer Interface technology can also be

used to give entertainment in a new way taking the entertainment into a new level.

Imagine a person playing a game using only his/hers thinking power.

In the past 10 years, the gaming industry has been a growing multi-billion-dollar

business, this shows that the demand of videos games has been growing, and this

rocketing demand also attracts a vast investment on new gaming interfaces, such as

Dance Pad in PlayStation and Wii controllers in Wii, which furthered feedback to the

snowball (i.e. the demand) positively. However, while new interfaces for console

games (e.g. touch screens for NDS and remote motion sensors for Wii) has been

developed, emergence of new gaming interfaces for PC games seem to slow down

after the introduction of game pads, and we think a new gaming interface could perhaps

give birth to a new genre of games in the big PC games market, where almost been

captured by interfaces like Microsoft Kinect. Hence one may wonder if anything new

would appear in the future gaming world, and what that would be recently, brain

computer interfaces for consumer level have been released to the market, making BCI

entertainment possible.

There are communities who are involved in developing Brain Computer Interface

games. However, we see a discrepancy between the BCI games developed by the

communities. Many of the BCI games developed by the BCI community aim at testing

some psychological hypotheses or evaluating the performance of signal analysis and

classification techniques. Thus, less attention is paid to game characteristics than to

technical aspects. These games do not usually have any narrative or rich feedback or

visuals. User (i.e. player) experience evaluations are almost never carried out. This

leads to BCI games that are reliable but often not enjoyable. On the contrary, BCI

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 5

Brain Computer Interface Application Framework

games from the games community are developed with respect to game design

principles. However, the neurophysiology and signal analysis techniques they rely on

are largely unknown. Because, these games mostly make use of the commercial BCI

headsets which have their private technical details. This leads to BCI games that are

potentially entertaining but unsatisfactory in terms of feeling of control.

In proposed method, we will try to transfer some knowledge from the games and the

BCI communities into a shared preliminary framework to make them aware of each

other’s research. From the games community, we will show some game playing

motivations which can be satisfied by the features of BCI. From the BCI community,

we will take the current interaction paradigms used in general and show the ways they

can be used in games. This way, we hope to contribute to bridging the gap between

the two communities and promoting the development of entertaining and reliable BCI

games.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 6

Brain Computer Interface Application Framework

2.2.2 Overview of the System Design

Figure 1: System Diagram

2.2.3 User Characteristics

The users of the system can be categorized under user groups as follows;

1. BCI device makers. 2. BCI game developers. 3. BCI game players. 4. Disabled persons.

The characteristics of those identified user groups of the Interactive Programming

Assistance Tool are illustrated below.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 7

Brain Computer Interface Application Framework

BCI device makers.

They are the people who develops and manufactures BCI devices. There are various

brands in World Market nowadays. And from those, Neurosky and Emotiv owns a

major market share.

BCI game developers.

They develop the games which can be played using BCI devices. The most part of

these of games get controlled directly using brain signals.

BCI game players.

BCI game players get an experience which is much different from the experience that

normal game players receive. They normally use their thinking power, concentration

power to control games rather than hand.

Disabled persons.

Because this game is controlled by the signals that are generated inside the Brain, even

a person who does not have any hands can play this game.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 8

Brain Computer Interface Application Framework

2.2.4 Product Functions

BCI enhanced Ludo Game

This Ludo game is developed to be controlled by the brain waves. By wearing this

device on your head and using your attention level, you can generate the value of the

Die.

There are two main functions in this Ludo Game.

- Calculating Running Average.

We tested few methods and finally chose to use running average to get a better

signal value from the device. This function does the part of calculating running average

of the signals values that gets captured by the device.

- Get a value to map with the game in order to control it.

We have developed a Ludo game to as a sample game. This function will map the

signal values that get calculated by the “Calculating Running Average” function.

BCI enhanced Keyboard

This application is a virtual keyboard which can be controlled by the eye blink instead

of the hands. There is some functionality which provides users to press the appropriate

keys using just the eye blink than using the hand.

BCI enhanced Calculator

This application also is same as the above mentioned keyboard. It also provides the

same functionality to the users as it allows users to control the options of the calculator

using the eye blink. For calculating purposes user have to train his/her way of blinking

in order to get the correct options.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 9

Brain Computer Interface Application Framework

BCI enhanced Snake and Ladders Game

This gaming application is for entertaining the users providing opportunity to play the

game using their eye blink than using the attention and the focus level. Depending on

the eye blink user has to toss the dice and win the game. There fore it provides users a

great opportunity to play the game in a new way apart from the traditional way.

2.2.5 Tools and Technologies

• We have used latest tools and technologies to do our research and to develop

the Ludo game. The tools and technologies include the following:

- Microsoft Visual Studio 2012 - .NET Framework 4.5 - Neurosky MindWave Device

• Microsoft Visual Studio 2012 is the IDE that we used to develop the Software

components of our product. These Software components mainly include the

Ludo game, the connection classes which connect the Neurosky device with

the software components, the running average calculation classes.

• Neurosky MindWave is the device we used for the BCI device.

• Microsoft Excel was used to store signal values we have collected from some

SLIIT students. Excel was also used to analyze the stored signal values by

drawing charts and by calculating using few formulas.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 10

Brain Computer Interface Application Framework

2.2.6 Product Constraints

• Our applications are specifically developed for Windows platform and tightly

coupled with windows based computer architecture. Developers do not have

sufficient hard-ware resources to build this application on different system

architectures such as Linux platform.

• When getting Brain Signal values using the device, the device takes like a

second to calibrate. So , when you start collecting the signal values, the values

come one second after you started it.

• The BCI device’s sensor which touches the forehead should have a good

contact with the forehead. Otherwise it won’t give good readings and will say

the signal quality is poor.

• In order for the device to work properly, the batteries that are inserted to the

device should be good quality batteries with a good charge.

• The driver for the BCI device should be installed in the Computer.

2.2.7 Assumptions and Dependencies

• It is assumed that the computer which runs our applications is running on the

Windows operating system and the Visual Studio framework is installed in the

respective machine. It is essential that the Visual Studio is up and running by

means of the accurate .NET framework.

• The source codes that are to be input to the system should be developed in C#

language.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 11

Brain Computer Interface Application Framework

• Microsoft Windows based supported architecture should be available and

implemented

- x86

- x64

• Microsoft Windows based operating system (Windows XP service pack 3 or

higher version) should be available and functioning successfully.

• All the machines should support the .NET framework 3.5 or higher.

2.3 Research Findings

While we are doing the research, through the tests we did, we found out that

• The device we are using have many limitations.

• The signals can get weakened at times.

• The variations of signal values are high.

• The way persons react to a one particular activity is different, because of that

the signals their brains emit are different to each other.

The device we are using have many limitations.

Our neurosky device comes with one sensor. And it can detect only meditation,

attention, eye blink activities of the user. Because of that, the applications for this

device can be only controlled by meditating, keeping the attention and by blinking

eyes.

Other movements of a user, like moving eyes, moving the head, moving a hand, can

not be recognised by this device.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 12

Brain Computer Interface Application Framework

The signals can get weakened at times.

The signal strength depends on few factors

- Whether the sensor is touching the forehead.

- Whether the battery of the neurosky device has a good charge level.

- The distance between the user who is wearing the device and the Computer

which is running the application that is controlled by brain signals of the user.

The variations of signal values are high.

The values of signals we gained by the tests were added to excel sheets. Then we

generated charts using those values. When considering those charts, we noticed a big

variation between the signal values. We calculated the variations of those signals.

Those calculations also gave higher values.

The following charts provide some examples for these higher variations. The data were

collected from a SLIIT Student. First we asked him to play a Car Race Game, then we

asked him to listen to a song. And while he was playing and listening to the song, we

collected the data. And using those data we calculated Running Average of those

values & variations of them.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 13

Brain Computer Interface Application Framework

Figure 2: Student A – Attention Level Values

Figure 3: Student A – Variance Between Attention Level Values

0

20

40

60

80

100

120

1 28 55 82 109

136

163

190

217

244

271

298

325

352

379

406

433

460

487

514

541

568

595

622

649

676

703

730

757

784

811

838

865

892

919

946

Attention Level Values

0

10

20

30

1 27 53 79 105

131

157

183

209

235

261

287

313

339

365

391

417

443

469

495

521

547

573

599

625

651

677

703

729

755

781

807

833

859

885

911

Variance Between Attention Level Values

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 14

Brain Computer Interface Application Framework

Figure 4: Student B – Attention Level Values

Figure 5: Student B – Variance Between Attention Level Values

0

10

20

30

40

50

60

70

80

90

1 10 19 28 37 46 55 64 73 82 91 100

109

118

127

136

145

154

163

172

181

190

199

208

217

226

235

244

253

262

271

280

289

298

307

316

325

Attention Level Values

0

5

10

15

20

25

1 9 17 25 33 41 49 57 65 73 81 89 97 105

113

121

129

137

145

153

161

169

177

185

193

201

209

217

225

233

241

249

257

265

273

281

289

Variance Between Attention Level Values

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 15

Brain Computer Interface Application Framework

The way persons react to a one particular activity is different, because of that the signals their brains emit are different to each other.

The way different persons react to a particular activity is different from each other.

Because of this the Brain Signals generated by different persons for a particular

activity is different from each other.

As an example some persons might like classic songs, while some don’t. While some

may have mixed emotions towards classic songs.

The following two charts were created using the data collected which were collected

using two SLIIT Students. The data were collected when they were listening to a song.

Both of them listened to the same song. Though they listened to the same song, the

way their brain reacted to that song was different. The two charts provide evidence for

that.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 16

Brain Computer Interface Application Framework

Figure 6: Student C - Attention Level Values while listening to a Song

Figure 7: Student D - Attention Level Values while listening to a Song

0

20

40

60

80

100

120

1 10 19 28 37 46 55 64 73 82 91 100

109

118

127

136

145

154

163

172

181

190

199

208

217

226

235

244

253

262

271

280

289

298

307

316

325

Attention Level Values

0102030405060708090

1 10 19 28 37 46 55 64 73 82 91 100

109

118

127

136

145

154

163

172

181

190

199

208

217

226

235

244

253

262

271

280

289

298

307

316

325

Attention Level Values

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 17

Brain Computer Interface Application Framework

3 RESULTS AND DISCUSSION

Why using the Running Average is better

When we consider the Attention or Meditation values that are directly given by the

neurosky device, we can notice a high variance between the values. That can be a

disadvantage while playing a game or using an application that is created based on the

BCI Technology.

Imagine you are trying to move a ball up by using the attention level of your brain.

Because of the high variance, the signal values get high and low so frequently so the

ball also will be going high and down frequently. That can be annoying. Using the

Running Average can reduce the variance and the variance problem can be avoided.

When we calculate the Running Average of those values, we managed to lower that

variance.

The following two charts prove this.

The first chart shows the direct values we retrieved from the device. The second chart

shows the Running Average values we calculated by those.

Figure 8: Attention Level Values

0102030405060708090

1 10 19 28 37 46 55 64 73 82 91 100

109

118

127

136

145

154

163

172

181

190

199

208

217

226

235

244

253

262

271

280

289

298

307

316

325

Attention Level Values

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 18

Brain Computer Interface Application Framework

Figure 9: Running Average of Attention Level Values

We can notice the difference between the two charts. The first chart has a high

variance. The second chart, which is drawn using the Running Average values, has a

low variance.

Because of that fact, using the Running Average is better.

There can be occasions when there are sudden drops of signal strength. You can notice

it when the signal value suddenly goes for ‘0’ and then values start to come back again.

During that occasions the Running Average can be a solution. Because of the running

average, you can avoid sudden drops in signal values.

Why this device is not capable of controlling big games.

You may have imagined of big games that get controlled totally by brain signals. Or

you may have imagined when you think a letter, that letter gets automatically printed

on the Computer. But since this device is giving out only the Attention level values,

Meditation level values and Eyeblink values, it can not be done.

01020304050607080

1 10 19 28 37 46 55 64 73 82 91 100

109

118

127

136

145

154

163

172

181

190

199

208

217

226

235

244

253

262

271

280

289

298

307

Running Average of Attention Level Values

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 19

Brain Computer Interface Application Framework

The Mind-Controlled Keyboards.

We have developed two keyboards for disabled persons. These will be best suited for Mute Motor-disabled persons. A big challenge these type of disabled persons have is that they find it difficult to communicate with others. And this system we have developed will be helpful for them to communicate with others.

They keyboard can be operated by using Eye-Blinks, which is convenient for the users.

Figure 10 - The simple keyboard

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 20

Brain Computer Interface Application Framework

The previous image shows our simple keyboard. It can be controlled by using Eye-Blinks.

In this keyboard, first row by row gets highlighted. When the user blinks, it goes one by one in the row that was highlighted when the user blinked.

As an example : If the user want to tell someone to “Switch on a Light”, initially the user has to blink when the particular row gets highlighted. Then blink again when the Key which shows an emitted bulb gets highlighted.

Figure 11 - Using the Simple Keyboard.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 21

Brain Computer Interface Application Framework

Next we have developed a keyboard for the purposes of typing messages. This also

can be operated just like the previous keyboard.

Once the message is typed the user can make the Computer pronounce the message by

clicking on “SAY IT”.

Figure 12 - The Advance Keyboard.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 22

Brain Computer Interface Application Framework

Ludo Game we have developed.

We have developed a Ludo game also. The Ludo game is developed to provide

entertainment for the user. The value of the “Die” of this game is generated according

to the Attention level of the user. If the user is having a higher attention level, the Die

will get higher values. If the attention level of the user is low, the Die will get lower

values.

The Attention level values of the user is given to the application by the BCI device

that is worn by the user. Then Running Average of those values are automatically

generated and those Running Average values decide the value that the Die should get.

Figure 13: Ludo Game User Interface

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 23

Brain Computer Interface Application Framework

4 CONCLUSION

For a long time, researchers have been working on a marriage of human and machine

that sounds like something out of science fiction: a brain computer interface. The

technology holds great promise for people who can’t use their arms or hands normally

because they have had spinal cord injuries or suffer from conditions such as

amyotrophic lateral sclerosis (ALS) or cerebral palsy. BCI could help them control

computers, wheelchairs, televisions, or other devices with brain activity.

To success with the projects like above mentioned, the capabilities of the used BCI

devices should be in a high standard level with the fully functional options. Also there

should be many sensors which are capable of capturing brain signal within vast areas

of the brain.

Since we are using the NeuroSky mindwave head set which is contain only one sensor

is not much capable of developing advance systems and applications. And the

generated raw data also not much accurate and reliable as it can be used for a

considerable application since it does not gives a stable value and can not find a certain

pattern in order to use as an average value for controlling an application

Therefore the team decided to implement applications depending on the capabilities of

the using BCI mindwave headset. So the implemented applications are Mind

controlling Calculator and a Keyboard which is working based on the users eye blink.

Also a Ludo Game controlling based on the users attention level and the focus level as

well as a Snakes and Ladders game based on the eye blink. The sound generator is an

application working based on the alpha, beta and gamma brain signals.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 24

Brain Computer Interface Application Framework

5 REFERENCES

1. "Brain-Computer Interfaces: Beyond Medical Applications," 2013. [Online]. Available: http://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applications.

2. E. Kader, B. P. B. P. Emilie Belley, J.-A. Filion, A. Nutter, M. Parent-Vachon, M. Saulnier, B. P. Stephanie Shedleur, B. P. Tsz Ting Wan, B. B. Elissa Sitcoff and P. O. Nicol Korner-Bitensky, "MOTOR IMAGERY - Information for Patients and Families," 2010. [Online]. Available: http://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdf.

3. "P300 (neuroscience)," 2012. [Online]. Available: http://en.wikipedia.org/wiki/P300_(neuroscience). [Accessed 2013].

4. K. Nakayama and M. Mackeben, "Steady State Visual Evoked Potentials In the Alert Primate," 1982. [Online]. Available: http://visionlab.harvard.edu/members/ken/Ken%20papers%20for%20web%20page/027VisionRes82.pdf.

5. "Event-related Potential: An overview," 2011. [Online]. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3016705/. [Accessed 2013].

6. "Electroencephalogram (EEG)," WebMD, LLC., 2010. [Online]. Available: http://www.webmd.com/epilepsy/electroencephalogram-eeg-21508.

7. "What Is A Framework?," 2003. [Online]. Available: http://www.codeproject.com/Articles/5381/What-Is-A-Framework.

8. "Neurosky MindWave Mobile," Neurosky, 2012. [Online]. Available: http://neurosky.com/Products/MindWaveMobile.aspx.

9. Microsoft, "Microsoft Visual Studio," Microsoft, 2013. [Online]. Available: http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012.

10. S. Du and M. Vuskovic, "Temporal vs. Spectral Approach to Feature Extraction," [Online]. Available: http://medusa.sdsu.edu/Robotics/Neuromuscular/Our_Publications/FE_Sijiang_press.pdf.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 25

Brain Computer Interface Application Framework

11. J. B. Ochoa, "EEG Signal Classification for Brain," 2002.

SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 26