técnicas de interacção avançadas - moodle-arquivo · digital format, and samples all of the...

TécnicasdeInteracçãoAvançadasBITalinoSensors

BITalino•  h/p://bitalino.com/

•  Bitalinoisasmallsensorpla8ormwellsuitedforworkingwithphysiologicaldata•  CommunicaAonviaClassIIBluetoothv2.0•  Availablesensors:

•  EMG•  ECG•  Accelerometer•  Light•  EDA

BitalinobaseboardMicro-ControllerThe micro-controller unit (MCU)block converts the analogicsignals from the sensors to adigital format, and samples all ofthe channels. The MCU providesaccess to theBITalinoanaloganddigital channels,aswell as to theperipherals.

PowerThe power management blockprovides energy to all the otherBITalino blocks. Thismodule alsohasabuilt-inchargerthatcontrolsthe ba/ery charging. Charging ismade when the device is turnedoffbyconnecAngapower supplyto theMicro-USBport.Thisblockalso provides access to controlsignals

BluetoothA Class II Bluetooth v2.0 with arangeupto10m

Sensors:EMG

The sensor is capable of performingelectromyography (EMG) measurementsusing bipolar surface electrodes (plus aground lead), and monitors the muscleacAvaAon.

Sensors:ECG

The ECG works mostly by detecAngand amplifying the Any electricalchangeson the skin thatare causedduringtheheartmusclecycleduringeachheartbeat.

Sensors:ACC

T h i s s e n s o r c a n m e a s u r eacceleraAonsrelaAvetofreefallandthe model available is capable ofdetecAngmagnitudeanddirecAonofthis same acceleraAon, as a vectorquanAty. This resulAng vector canthen be used to sense posiAon,vibraAon,shock,fall,etc.

Sensors:EDA

Electrodermal AcAvity (EDA) can bedefined as a transient change incertain electrical properAes of theskin,associatedwiththesweatglandacAvity and elicited by any sAmulusthat evokes an arousal or orienAngresponse.

Sensors:LUX

A pho tod iode i s a t ype o fphotodetectorcapableofconverAnglightintoeithercurrentorvoltage.Inthis case the output is given involtage.

Actuators:LED

TheLEDmoduleconsists inasimpleLight-EmiUngDiode(LED)thatturnsonwhentheinputsignal ishighandturns off when the input signal islow.

BitalinoEnablingSoftware•  OpenSignals(r)evoluAon

•  SoWwarefordataacquisiAon,visualizaAonandprocessingwhileusingPLUX’sbiosignalpla8orms

•  AllowstheusertoacquiredatafromoneormulApledevices,setuptheexactacquisiAonparameters

•  Save standard ASCII text format or in the more efficient Hierarchical DataFormat(HDF)forprocessingbythirdpartytools

BitalinoSoftwaredevelopment•  MulApleProgramminglanguages•  Java(Desktopandandroid)•  C++•  C#•  BonsaiLibrary•  LabVIEW•  MATLAB•  Etc.

BitalinoJavaAPI(versionPriscilaAlves)•  Pre-Requisites(JavaWindows)•  JavaAPI•  EclipseMars(recommended)•  Bluecove-2.1.1-SNAPSHOT(bluetoothcommunicaAon)•  PCwithbluetoothdevice

•  PairBitalinodevicewithcomputer(pass:1234)

BitalinoAPI

BITalinoAPI

BITalino

BITalinoErrorTypesBITalinoExcepAon

DeviceDiscoverer

Frame

SensorDataConverter

BitalinoClass

YourApplicaAon

BitalinoObject

FramesConfig

BlueCovebluetoothPCbluetoothHardware

BitalinoClass•  MainfuncAons:•  Open–bitalino.open(StringmacAdd,intsamplingRate);•  Start–bitalino.start(int[]anChannels);•  Stop– bitalino.stop();•  Close–bitalino.close();•  Read-bitalino.read(intnSamples)•  Write–bitalino.write(intdata);•  Trigger–bitalino.trigger(int[]digitalArray);

DataStructure

DataacquiredwithNframes

ValuesfromacAvechannels

Analogchannelrawdata

MyFirstBitalino•  BasicExample•  Connect•  StartandStopStream•  CollectandShowdatafromdifferentsensors

•  SimplisAcrecognizer(EMG+LED)

MyFirstBitalino1.  NewAndroidApplicaAonProject2.  ImportBitalinoTest3.  ProperAes->JavaBuildPath->Libraries->AddExternalJars

I.  bluecove-2.1.1-SNAPSHOT.jar;4.  OrderandExporttab

MyFirstBitalino•  Imports:

importBITalino.BITalino;importBITalino.Frame;importBITalino.SensorDataConverter;importjava.u@l.Vector;importjavax.bluetooth.RemoteDevice;

MyFirstBitalino•  Somevariablesneeded:publicstaAcFrame[]dataAcquired;//setofframes(nSamples)privatestaAcfinalintACC_MIN=208;privatestaAcfinalintACC_MAX=312;//Accelerometerconverter

MyFirstBitalinoSensorDataConvertersc=newSensorDataConverter();//classthatconvertsrawdataintomeaningfuldataint[]aChannels={0,1,2};//variablethatdefinesfromwhichchannelacquiredata

MyFirstBitalino•  IniAalizesensorBITalinodevice=newBITalino();//finddevicesVector<RemoteDevice>devices=device.findDevices();System.out.println(devices);//connecttoBITalinodeviceStringmacAddress="98:D3:31:90:3D:C2";intsamplingRate=1000;Intsamples=2000;device.open(macAddress,samplingRate);

MyFirstBitalino•  Acquiredata:

//startacquisiAononanalogchannels0to2device.start(aChannels);dataAcquired=device.read(samples);for(Frameframe:dataAcquired){

System.out.println("rawdataaccX:"+frame.analog[0]+"rawdataaccY:“+frame.analog[1]+"rawdataaccZ:"+frame.analog[2]);

System.out.println("converteddataaccX:"+sc.scaleAccelerometerWithPrecision(0,frame.analog[0],ACC_MIN,ACC_MAX)+"converteddataaccY:“+sc.scaleAccelerometerWithPrecision(1,frame.analog[1],ACC_MIN,ACC_MAX)+"converteddataaccZ:"+sc.scaleAccelerometerWithPrecision(2,frame.analog[2],ACC_MIN,ACC_MAX));}

MyFirstBitalino•  Closedataacquisi.onandbluetoothconnec.on:

//stopacquisiAondevice.stop();//closebluetoothconnecAondevice.close();

Exercises

sEMGsensor•  "Electromyography(EMG)isanexperimentaltechniqueconcernedwiththedevelopment,recordingandanalysisofmyoelectric signals.

•  Myoelectric signals areformed byphysiological variaAons inthe state ofmusclefibermembranes."

sEMGsensor•  Typicalbenefitsare:•  EMGallowstodirectly“look”intothemuscle•  Itallowsmeasurementofmuscularperformance•  Helpsindecisionmakingbothbefore/aWersurgery•  Documentstreatmentandtrainingregimes•  HelpspaAentsto“find”andtraintheirmuscles•  AllowsanalysistoimprovesportsacAviAes•  Detectsmuscleresponseinergonomicstudies

sEMGsensor•  HowmusclecontracAonworks?

sEMGsensor

sEMGsensor•  sEMGsignal

sEMGsensor•  FactorsinfluencingtheEMGsignal:•  TissuecharacterisAcs•  Physiologicalcrosstalk•  Changesinthegemoetrybetweenmusclebellyandeletrodesite•  Externalnoise•  Electrodeandamplifiers

sEMGsensor•  HumanAnatomy

Howtobuildarecognizer•  Capturingthesignalofadatastream•  WindowingandOverlapping

Howtobuildarecognizer•  Pre-processingthesignal•  Bu/erworthlow-pass,high-passandbandpassfilters

•  Frequenciesdependonthesensorused•  Helpsfilteringunwantedsignalsandenhancethesignalquality(e.g.Noisefromotherdevices,etc.)

Howtobuildarecognizer•  FeatureExtracAon•  Derivedvaluesfromthesignal•  Transformlargesetsofdataintoareducedsetoffeatures•  HelpsclassificaAonalgorithmstofindpa/ernsindata•  ExamplesforEMGsensor:

•  Waveformlength,RootMeanSquare,Wamp

Howtobuildarecognizer•  Classifier•  IdenAfyingtowhichofasetofcategories(sub-populaAons)anewobservaAonbelongs

•  SupervisedLearning

WL RMS WAMP Gesture

100 200 300 WristFlexion

150 200 400 Closehand

400 100 100 Openhand

... ... ... ...

Howtobuildarecognizer

“Closehand”

WL RMS WAMP Gesture

100 200 300 WristFlexion

150 200 400 Closehand

400 100 100 Openhand

... ... ... ...

WL RMS WAMP Gesture

150 200 400 ?

Pre-processing

FeaturesExtrac@on ClassifierCapture

WEKAAPI•  WekaisacollecAonofmachinelearningalgorithmsfordataminingtasks•  SVM,RandomTree,BayesNet,etc.•  h/p://www.cs.waikato.ac.nz/ml/weka/index.html

WEKAAPI•  DataSctructure//DeclaretwonumericaWributesA]ributeA]ribute1=newA]ribute(“firstNumeric”);A]ributeA]ribute2=newA]ribute(“secondNumeric”);//DeclareanominalaWributealongwithitsvaluesFastVectorfvNominalVal=newFastVector(3);fvNominalVal.addElement(“blue”);fvNominalVal.addElement(“gray”);fvNominalVal.addElement(“black”);A]ributeA]ribute3=newA]ribute(“aNominal”, fvNominalVal);//DeclaretheclassaWributealongwithitsvaluesFastVectorfvClassVal=newFastVector(2);fvClassVal.addElement(“posi@ve”);fvClassVal.addElement(“nega@ve”);A]ributeClassA]ribute=newA]ribute(“theClass”,fvClassVal);//DeclarethefeaturevectorFastVectorfvWekaA/ributes=new FastVector(4);fvWekaA]ributes.addElement(A]ribute1);fvWekaA]ributes.addElement(A]ribute2);fvWekaA]ributes.addElement(A]ribute3);fvWekaA]ributes.addElement(ClassA]ribute);

WEKAAPI•  Trainaclassifier//CreateanemptytrainingsetInstancesisTrainingSet=newInstances("Rel",fvWekaA]ributes,10);//SetclassindexisTrainingSet.setClassIndex(3);//CreatetheinstanceInstanceiExample=newDenseInstance(4);iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(0),1.0);iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(1),0.5);iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(2),"gray");iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(3),"posi@ve");//addtheinstanceisTrainingSet.add(iExample);//CreateanaïvebayesclassifierClassifiercModel=(Classifier)newNaiveBayes();cModel.buildClassifier(isTrainingSet);

WEKAAPI•  TesttheClassifier//TestthemodelEvalua@oneTest=newEvalua@on(isTrainingSet);eTest.evaluateModel(cModel,isTes@ngSet);//PrinttheresultàlaWekaexplorer:StringstrSummary=eTest.toSummaryString();System.out.println(strSummary);//Gettheconfusionmatrixdouble[][]cmMatrix=eTest.confusionMatrix();

WEKAAPI•  Usetheclassifier//setclassaWributeunlabeled.setClassIndex(unlabeled.numA]ributes()-1);//createcopyInstanceslabeled=newInstances(unlabeled);//labelinstancesfor(inti=0;i<unlabeled.numInstances();i++){doubleclsLabel=cModel.classifyInstance(unlabeled.instance(i));labeled.instance(i).setClassValue(clsLabel);}

References•  DavidCostaandCarlosDuarte,"FromOnetoManyUsersandContexts:aClassifier

formarmandhandgestures",IUI'15,Atlanta,USA,2015•  MakingMuscle-ComputerInterfacesMorePracAcal.T.Sco/Saponas,DesneyS.Tan,

DanMorris,JimTurner,andJamesA.Landay.Proceedingsofthe 2010SIGCHI(Atlanta,GA,April10-15,2010).CHI'10.ACM,NewYork,NY,851-854

•  EnhancingInputOnandAbovetheInteracAveSurfacewithMuscleSensing.HrvojeBenko,T.Sco/Saponas,DanMorris,andDesneyS.Tan.ProceedingsofACMInteracAveTabletopsandSurfaces(Banff,Alberta,Canada,November23-25,2009).ITS'09.ACM,NewYork,NY,93-100

•  EnablingAlways-AvailableInputwithMuscle-ComputerInterfaces.T.Sco/Saponas,DesneyS.Tan,DanMorris,RavinBalakrishnan,JimTurner,andJamesA.Landay.ProceedingsACMSymposiumonUserInterfaceSoWwareandTechnology(Victoria,BC,October4-7,2009).UIST'09.ACM,NewYork,NY,167-176.

•  MakingGesturalInputfromArm-WornInerAalSensorsMorePracAcal.LouisKratz,T.Sco/Saponas,DanMorris.ProceedingsofSIGCHIConferenceonHumanFactorsinCompuAngSystems(May2012).CHI'12.ACM,NewYork,NY,1747-1750.

•  RecoFit:usingawearablesensortofind,recognize,andcountrepeAAveexercises.DanMorris,T.Sco/Saponas,AndrewGuillory,andIlyaKelner.ProceedingsoftheSIGCHIConferenceonHumanFactorsinCompuAngSystems(April2014).CHI'14.ACM,NewYork,NY,3225-3234.

• Documentsanddownloads:• h]p://@nyurl.com/h63a7bn

• Contact:• dcosta@lasige.di.fc.ul.pt