EMG-driven Human Modeling to Enhance

Human-Robot Interaction Control

Mattia Pesenti

Supervisor: Elena De MomiApril 16, 2019

Co-supervisors: Ziad Alkhoury, Bernard Bayle

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Outline

1

• Introduction

• EMG-driven Human Modeling

• Materials and Methods

• Results

• Discussion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Human-Robot Interaction

2

Introduction Human Modeling Materials and Methods Results Conclusion

Human-Robot Interaction (HRI) is an interdisciplinary field ofstudy dedicated to understanding, designing and evaluatingrobotic systems for use by or with humans.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Human-Robot Interaction

2

Introduction Human Modeling Materials and Methods Results Conclusion

Human-Robot Interaction (HRI) is an interdisciplinary field ofstudy dedicated to understanding, designing and evaluatingrobotic systems for use by or with humans.

Rehabilitation Robotics

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Human-Robot Interaction

2

Introduction Human Modeling Materials and Methods Results Conclusion

Human-Robot Interaction (HRI) is an interdisciplinary field ofstudy dedicated to understanding, designing and evaluatingrobotic systems for use by or with humans.

Rehabilitation Robotics Tele-operation

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Human-Robot Interaction

2

Introduction Human Modeling Materials and Methods Results Conclusion

Human-Robot Interaction (HRI) is an interdisciplinary field ofstudy dedicated to understanding, designing and evaluatingrobotic systems for use by or with humans.

Rehabilitation Robotics Tele-operation Robotic-guidance

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Control-oriented Modeling

3

Introduction Human Modeling Materials and Methods Results Conclusion

Control of a robotic manipulator: determination of the generalized

forces (𝜏) required to guarantee the execution of a planned task.

ControllerTask

ROBOT𝜏𝑐

𝜏ext

+-

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Control-oriented Modeling

3

Introduction Human Modeling Materials and Methods Results Conclusion

Control of a robotic manipulator: determination of the generalized

forces (𝜏) required to guarantee the execution of a planned task.

ControllerTask

ROBOT

Environment

USER

++

𝜏𝑐𝜏ext

𝜏u

𝜏e

+-

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Control-oriented Modeling

3

Introduction Human Modeling Materials and Methods Results Conclusion

Control of a robotic manipulator: determination of the generalized

forces (𝜏) required to guarantee the execution of a planned task.

ControllerTask

ROBOT

Environment

USERMODEL

++

𝜏𝑐𝜏ext

Ƹ𝜏𝑢

𝜏u

𝜏e

+-

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Outline

4

• Introduction

• EMG-driven Human Modeling

• Materials and Methods

• Results

• Discussion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

State of the Art

5

Introduction Human Modeling Materials and Methods Results Conclusion

Black-box modeling of the human arm

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

State of the Art

5

Introduction Human Modeling Materials and Methods Results Conclusion

Black-box modeling of the human arm

1. Parallel Cascade Identification (PCI) [Hashemi 2012,2015]

• Block-oriented nonlinear model: the Wiener model

• Several Wiener models (cascades) in parallel to reduce the modeling error

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

State of the Art

5

Introduction Human Modeling Materials and Methods Results Conclusion

Black-box modeling of the human arm

1. Parallel Cascade Identification (PCI) [Hashemi 2012,2015]

• Block-oriented nonlinear model: the Wiener model

• Several Wiener models (cascades) in parallel to reduce the modeling error

Dynamic linear Static nonlinearEMG Force

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

State of the Art

5

Introduction Human Modeling Materials and Methods Results Conclusion

Black-box modeling of the human arm

1. Parallel Cascade Identification (PCI) [Hashemi 2012,2015]

• Block-oriented nonlinear model: the Wiener model

• Several Wiener models (cascades) in parallel to reduce the modeling error

Dynamic linear Static nonlinearEMG Force

2. Dynamic, nonlinear polynomial model [Clancy 2012]

• Elbow torque ∝ muscular activation (extracted from the EMG)

• Grey-box approach

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Problem Statement

6

Introduction Human Modeling Materials and Methods Results Conclusion

The current state-of-the-art methods provide a black-box for the

human arm. On the other hand, these models are

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Problem Statement

6

Introduction Human Modeling Materials and Methods Results Conclusion

The current state-of-the-art methods provide a black-box for the

human arm. On the other hand, these models are

Highly nonlinear

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Problem Statement

6

Introduction Human Modeling Materials and Methods Results Conclusion

The current state-of-the-art methods provide a black-box for the

human arm. On the other hand, these models are

Highly nonlinear

Computationally complex

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Problem Statement

6

Introduction Human Modeling Materials and Methods Results Conclusion

The current state-of-the-art methods provide a black-box for the

human arm. On the other hand, these models are

Highly nonlinear

Computationally complex

Not suitable for online control problems

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Problem Statement

6

Introduction Human Modeling Materials and Methods Results Conclusion

The current state-of-the-art methods provide a black-box for the

human arm. On the other hand, these models are

Highly nonlinear

Computationally complex

Not suitable for online control problems

Accurate only for constant-posture force trials

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Motivation and Aims

7

The goal of this master thesis project has been to identify a black-box

model of the EMG-Force relationship of the human arm.

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Motivation and Aims

7

• Control-oriented human modeling

The goal of this master thesis project has been to identify a black-box

model of the EMG-Force relationship of the human arm.

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Motivation and Aims

7

• Direct I/O relationship between EMG and force

• Control-oriented human modeling

The goal of this master thesis project has been to identify a black-box

model of the EMG-Force relationship of the human arm.

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Motivation and Aims

7

• Suitable for Human-Robot Interaction

• Direct I/O relationship between EMG and force

• Control-oriented human modeling

The goal of this master thesis project has been to identify a black-box

model of the EMG-Force relationship of the human arm.

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Outline

8

• Introduction

• EMG-driven Human Modeling

• Materials and Methods

• Results

• Discussion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

Model StructureDetermination

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

Model StructureDetermination

ExperimentDesign

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

Model StructureDetermination

ExperimentDesign

Data Acquisition

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

Model StructureDetermination

ExperimentDesign

Data Acquisition

Parameters Estimation

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

Model StructureDetermination

ExperimentDesign

Data Acquisition

Parameters Estimation

Model Validation

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

System Identification

9

Introduction Human Modeling Materials and Methods Results Conclusion

System Identification is the field of modeling input/output (I/O)

dynamic systems from experimental data [Söderström, Stoica 1989].

Model StructureDetermination

ExperimentDesign

Data Acquisition

Identified ModelParameters Estimation

Model Validation

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The 1-DoF Human Arm (1/2)

10

Introduction Human Modeling Materials and Methods Results Conclusion

EMG Force1-DoF ARM

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The 1-DoF Human Arm (1/2)

10

Introduction Human Modeling Materials and Methods Results Conclusion

EMG Force

Biceps (BIC)

Triceps (TRI)

1-DoF ARM

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The 1-DoF Human Arm (1/2)

10

Introduction Human Modeling Materials and Methods Results Conclusion

EMG Force

Biceps (BIC)

Triceps (TRI)

Flexor Carpi Radialis (FCR)

Brachioradialis (BRD)

1-DoF ARM

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

𝑦 𝑘 = 𝐶𝑥(𝑘)

The 1-DoF Human Arm (2/2)

11

Introduction Human Modeling Materials and Methods Results Conclusion

Discrete-time, State-Space,

𝑢(𝑘) 𝑦(𝑘)

𝐿𝑇𝐼

𝑢(𝑘): EMG input (4 channels)𝑦(𝑘): Force output

2nd Order State-Space Model

Linear Time Invariant (LTI) Model

𝑥 𝑘 + 1 = 𝐴𝑥 𝑘 + 𝐵𝑢(𝑘)

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

𝑦 𝑘 = 𝐶𝑥(𝑘)

The 1-DoF Human Arm (2/2)

11

Introduction Human Modeling Materials and Methods Results Conclusion

Discrete-time, State-Space,

𝑢(𝑘) 𝑦(𝑘)

𝐿𝑃𝑉 ≈ 𝐿𝑇𝐼 𝑞

Linear Parameter Varying (LPV) Model

𝑞(𝑘)

𝑢(𝑘): EMG input (4 channels)𝑦(𝑘): Force output

𝑞(𝑘): Scheduling Variable2nd Order State-Space Model

𝑥 𝑘 + 1 = 𝐴𝑥 𝑘 + 𝐵𝑢(𝑘)𝑥 𝑘 + 1 = 𝐴 𝑞 𝑘 𝑥 𝑘 + 𝐵 𝑞 𝑘 𝑢(𝑘)

𝑦 𝑘 = 𝐶 𝑞 𝑘 𝑥(𝑘)

Elbow Angle

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach

More practical

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach

More practical

LTI identification tools

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach

More practical

LTI identification tools

Not always sufficient for black-boxmodels [Tóth 2007]

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach The Global approach

More practical

LTI identification tools

Not always sufficient for black-boxmodels [Tóth 2007]

A natural way for LPV systems

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach The Global approach

More practical

LTI identification tools

Not always sufficient for black-boxmodels [Tóth 2007]

A natural way for LPV systems

May be the only choice available

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach The Global approach

More practical

LTI identification tools

Not always sufficient for black-boxmodels [Tóth 2007]

A natural way for LPV systems

May be the only choice available

More complex experiment design

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LPV Identification

12

Introduction Human Modeling Materials and Methods Results Conclusion

There are two identification approaches to obtain an LPV model.

The Local approach The Global approach

More practical

LTI identification tools

Not always sufficient for black-boxmodels [Tóth 2007]

A natural way for LPV systems

May be the only choice available

More complex experiment design

Requires new identification algorithms

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Local Identification Framework [LIF]

13

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Local Identification Framework [LIF]

13

Introduction Human Modeling Materials and Methods Results Conclusion

One local trial is acquired per each value ofthe scheduling variable 𝑞 in order to sampleits range from 80° to 130°.

𝑞 = 80, 90, 95, 100, 110, 115, 120, 130 [°]

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Local Identification Framework [LIF]

13

Introduction Human Modeling Materials and Methods Results Conclusion

One local trial is acquired per each value ofthe scheduling variable 𝑞 in order to sampleits range from 80° to 130°.

𝑞 𝐿𝑇𝐼(𝑞𝑖)𝑞 = 80°

𝑞 = 130°

𝑞 = 80, 90, 95, 100, 110, 115, 120, 130 [°]

𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Local Identification Framework [LIF]

13

Introduction Human Modeling Materials and Methods Results Conclusion

One local trial is acquired per each value ofthe scheduling variable 𝑞 in order to sampleits range from 80° to 130°.

A local, linear (LTI) model is identified ateach position (N4SID algorithm).

𝑞 𝐿𝑇𝐼(𝑞𝑖)𝑞 = 80°

𝑞 = 130°

𝑞 = 80, 90, 95, 100, 110, 115, 120, 130 [°]

𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Local Identification Framework [LIF]

13

Introduction Human Modeling Materials and Methods Results Conclusion

One local trial is acquired per each value ofthe scheduling variable 𝑞 in order to sampleits range from 80° to 130°.

A local, linear (LTI) model is identified ateach position (N4SID algorithm).

The set of frozen-equivalent linearmodels is interpolated to build the LinearParameter Varying model (LIF-LPV).

𝐴 𝑞 = 𝐴0 + 𝐴1 ∙ 𝑞

𝐵 𝑞 = 𝐵0 + 𝐵1 ∙ 𝑞

𝐶 𝑞 = 𝐶0 + 𝐶1 ∙ 𝑞

𝑞 𝐿𝑇𝐼(𝑞𝑖)𝑞 = 80°

𝑞 = 130°

𝑞 = 80, 90, 95, 100, 110, 115, 120, 130 [°]

𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Global Identification Framework [GIF]

14

Introduction Human Modeling Materials and Methods Results Conclusion

Only one global trial is necessary and sufficient to identify the Linear ParameterVarying model following the global approach.

𝑞(𝑡)

𝑞 = 90°

𝑞 = 120°

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Global Identification Framework [GIF]

14

Introduction Human Modeling Materials and Methods Results Conclusion

Only one global trial is necessary and sufficient to identify the Linear ParameterVarying model following the global approach.

This is acquired while the scheduling variable follows a pre-determined time trajectory 𝑞(𝑡)

𝑞(𝑡)

𝑞 = 90°

𝑞 = 120°

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Global Identification Framework [GIF]

14

Introduction Human Modeling Materials and Methods Results Conclusion

Only one global trial is necessary and sufficient to identify the Linear ParameterVarying model following the global approach.

This is acquired while the scheduling variable follows a pre-determined time trajectory 𝑞(𝑡)

𝑞(𝑡) The trajectory is achieved by varying the coordinate of therobot at a constant velocity (3.57 °/s) between 90° and 120°.

𝑞 = 90°

𝑞 = 120°

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Global Identification Framework [GIF]

14

Introduction Human Modeling Materials and Methods Results Conclusion

Only one global trial is necessary and sufficient to identify the Linear ParameterVarying model following the global approach.

This is acquired while the scheduling variable follows a pre-determined time trajectory 𝑞(𝑡)

𝑞(𝑡) The trajectory is achieved by varying the coordinate of therobot at a constant velocity (3.57 °/s) between 90° and 120°.

One single dataset {𝑢 𝑡 , 𝑞 𝑡 , 𝑦 𝑡 } is used to identify directly the GIF-LPV modelexploiting the LPVcore toolbox [Cox 2018].

𝑞 = 90°

𝑞 = 120°

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Experimental Setup

15

Introduction Human Modeling Materials and Methods Results Conclusion

Elbow angle 𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Experimental Setup

15

Introduction Human Modeling Materials and Methods Results Conclusion

• Wireless EMG sensors

(DELSYS) [2000 Hz]

Elbow angle 𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Experimental Setup

15

Introduction Human Modeling Materials and Methods Results Conclusion

• Wireless EMG sensors

(DELSYS) [2000 Hz]

• Collaborative robot:

KUKA LBR iiwa

Elbow angle 𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Experimental Setup

15

Introduction Human Modeling Materials and Methods Results Conclusion

• Wireless EMG sensors

(DELSYS) [2000 Hz]

• Collaborative robot:

KUKA LBR iiwa

• 6-DoF Force sensor

[100 Hz]

• Microcontroller for

signals synchronization

Elbow angle 𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Experimental Setup

15

Introduction Human Modeling Materials and Methods Results Conclusion

• Wireless EMG sensors

(DELSYS) [2000 Hz]

• Collaborative robot:

KUKA LBR iiwa

• Visual force feedback

• 6-DoF Force sensor

[100 Hz]

• Microcontroller for

signals synchronization

Elbow angle 𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Experimental Setup

15

Introduction Human Modeling Materials and Methods Results Conclusion

• Wireless EMG sensors

(DELSYS) [2000 Hz]

• Collaborative robot:

KUKA LBR iiwa

• Visual force feedback

• 6-DoF Force sensor

[100 Hz]

• Study population: two

young, healthy males

• Microcontroller for

signals synchronization

Elbow angle 𝑞

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Task

16

Introduction Human Modeling Materials and Methods Results Conclusion

The user is asked to alternate flexors and extensors of the arm togenerate force peaks while interacting with the robot.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Task

16

Introduction Human Modeling Materials and Methods Results Conclusion

The user is asked to alternate flexors and extensors of the arm togenerate force peaks while interacting with the robot.

Constant amplitude (max 20 N), constant frequency force signal

Time [s]

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The Task

16

Introduction Human Modeling Materials and Methods Results Conclusion

The user is asked to alternate flexors and extensors of the arm togenerate force peaks while interacting with the robot.

Constant amplitude (max 20 N), constant frequency force signal

Time [s]

These contractions are isometric only during the local trials

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

EMG Processing

17

Introduction Human Modeling Materials and Methods Results Conclusion

Band-Pass Filter20-350 Hz

5th-2nd order

Full-Wave RectifierLow-Pass Filter

1.775 Hz2nd order

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

EMG Processing

17

Introduction Human Modeling Materials and Methods Results Conclusion

Band-Pass Filter20-350 Hz

5th-2nd order

Full-Wave RectifierLow-Pass Filter

1.775 Hz2nd order

Raw EMGAcquired at 2000 Hz

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

EMG Processing

17

Introduction Human Modeling Materials and Methods Results Conclusion

Band-Pass Filter20-350 Hz

5th-2nd order

Full-Wave RectifierLow-Pass Filter

1.775 Hz2nd order

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

EMG Processing

17

Introduction Human Modeling Materials and Methods Results Conclusion

Band-Pass Filter20-350 Hz

5th-2nd order

Full-Wave RectifierLow-Pass Filter

1.775 Hz2nd order

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

EMG Processing

17

Introduction Human Modeling Materials and Methods Results Conclusion

Muscular activation

Band-Pass Filter20-350 Hz

5th-2nd order

Full-Wave RectifierLow-Pass Filter

1.775 Hz2nd order

Muscular activation

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Model Validation

18

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Model Validation

18

Introduction Human Modeling Materials and Methods Results Conclusion

• Performance Metrics

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Model Validation

18

Introduction Human Modeling Materials and Methods Results Conclusion

• Performance Metrics

𝐹𝐼𝑇 = 100 ∙ 1 −𝑦 − ො𝑦

𝑦 − 𝑦

Goodness of FIT [%]

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Model Validation

18

Introduction Human Modeling Materials and Methods Results Conclusion

• Performance Metrics

𝐹𝐼𝑇 = 100 ∙ 1 −𝑦 − ො𝑦

𝑦 − 𝑦𝑉𝐴𝐹 = 100 ∙

𝜎2 𝑦 − ො𝑦

𝜎2 𝑦

Goodness of FIT [%] Variance Accounted For (VAF) [%]

𝑦: measured force dataො𝑦: estimated force𝑦: mean of measured force data

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Model Validation

18

Introduction Human Modeling Materials and Methods Results Conclusion

• Performance Metrics

• Comparison to a gold standard

𝐹𝐼𝑇 = 100 ∙ 1 −𝑦 − ො𝑦

𝑦 − 𝑦𝑉𝐴𝐹 = 100 ∙

𝜎2 𝑦 − ො𝑦

𝜎2 𝑦

Goodness of FIT [%] Variance Accounted For (VAF) [%]

𝑦: measured force dataො𝑦: estimated force𝑦: mean of measured force data

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Model Validation

18

Introduction Human Modeling Materials and Methods Results Conclusion

• Performance Metrics

• Comparison to a gold standard

𝐹𝐼𝑇 = 100 ∙ 1 −𝑦 − ො𝑦

𝑦 − 𝑦𝑉𝐴𝐹 = 100 ∙

𝜎2 𝑦 − ො𝑦

𝜎2 𝑦

Goodness of FIT [%] Variance Accounted For (VAF) [%]

Static nonlinear Dynamic linear𝑢 𝑦

Identification of a traditional, nonlinear model, i.e. the Hammerstein model.

𝑦: measured force dataො𝑦: estimated force𝑦: mean of measured force data

EMG input Force output

[Clancy 2012]

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

Global LPV identification

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

Global LPV identification GIF-LPV model

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

Global LPV identification

Hammerstein identification

GIF-LPV model

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

Global LPV identification

Hammerstein identification

GIF-LPV model

Hammerstein

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

Global LPV identification

Global LTI identification

Hammerstein identification

GIF-LPV model

Hammerstein

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Identification Protocol [Recap]

19

Introduction Human Modeling Materials and Methods Results Conclusion

Local data acquisition8 positions (𝑞𝑖)

Local LTI identification

Affine interpolation LIF-LPV model

Global data acquisition𝑞(𝑡)

Global LPV identification

Global LTI identification

Hammerstein identification

GIF-LPV model

LTI model

Hammerstein

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Outline

20

• Introduction

• EMG-driven Human Modeling

• Materials and Methods

• Results

• Discussion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LIF-LPV Model [Local Identification Framework]

21

Introduction Human Modeling Materials and Methods Results Conclusion

Validation on global trials of the LIF-LPV model identified using local trials by means ofLTI interpolation.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

LIF-LPV Model [Local Identification Framework]

21

Introduction Human Modeling Materials and Methods Results Conclusion

Validation on global trials of the LIF-LPV model identified using local trials by means ofLTI interpolation.

FIT [%] VAF [%]

User 1 67.77 94.63

User 2 80.42 96.22

Average 74.10 95.55

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

GIF-LPV Model [Global Identification Framework]

22

Introduction Human Modeling Materials and Methods Results Conclusion

Validation on global trials of the GIF-LPV model identified using one global trial.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

GIF-LPV Model [Global Identification Framework]

22

Introduction Human Modeling Materials and Methods Results Conclusion

FIT [%] VAF [%]

User 1 75.37 96.22

User 2 81.12 96.44

Average 78.25 95.66

Validation on global trials of the GIF-LPV model identified using one global trial.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Globally-identified LTI Model

23

Introduction Human Modeling Materials and Methods Results Conclusion

The same identification/validation datasets were used also considering an evensimpler approach, i.e. the use of a Linear Time Invariant model.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Globally-identified LTI Model

23

Introduction Human Modeling Materials and Methods Results Conclusion

The same identification/validation datasets were used also considering an evensimpler approach, i.e. the use of a Linear Time Invariant model.

FIT [%] VAF [%]

User 1 72.61 93.12

User 2 80.89 96.38

Average 76.75 94.75

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Performance Comparison

24

Introduction Human Modeling Materials and Methods Results Conclusion

LTI LIF-LPV GIF-LPV

FIT [%] 76.75 74.10 78.25

VAF [%] 94.75 95.55 95.66

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Performance Comparison

24

Introduction Human Modeling Materials and Methods Results Conclusion

LTI LIF-LPV GIF-LPV

FIT [%] 76.75 74.10 78.25

VAF [%] 94.75 95.55 95.66

• As expected, the GIF-LPV outperforms the LIF-LPV model on global trials

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Performance Comparison

24

Introduction Human Modeling Materials and Methods Results Conclusion

LTI LIF-LPV GIF-LPV

FIT [%] 76.75 74.10 78.25

VAF [%] 94.75 95.55 95.66

• As expected, the GIF-LPV outperforms the LIF-LPV model on global trials

• The globally-identified LTI model outperforms the LIF-LPV as well!

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Performance Comparison

24

Introduction Human Modeling Materials and Methods Results Conclusion

LTI LIF-LPV GIF-LPV

FIT [%] 76.75 74.10 78.25

VAF [%] 94.75 95.55 95.66

• As expected, the GIF-LPV outperforms the LIF-LPV model on global trials

• The globally-identified LTI model outperforms the LIF-LPV as well!

Both these results are due to the dynamics of 𝒒(𝒕)

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Performance Comparison

24

Introduction Human Modeling Materials and Methods Results Conclusion

LTI LIF-LPV GIF-LPV

FIT [%] 76.75 74.10 78.25

VAF [%] 94.75 95.55 95.66

Hammerstein

82.94

97.09

• As expected, the GIF-LPV outperforms the LIF-LPV model on global trials

• The globally-identified LTI model outperforms the LIF-LPV as well!

Both these results are due to the dynamics of 𝒒(𝒕)

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Outline

25

• Introduction

• EMG-driven Human Modeling

• Materials and Methods

• Results

• Discussion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Conclusion

26

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Conclusion

26

Introduction Human Modeling Materials and Methods Results Conclusion

Low-complexity, control-oriented model of the human arm.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Conclusion

26

Introduction Human Modeling Materials and Methods Results Conclusion

Low-complexity, control-oriented model of the human arm.

Accurate for both constant- and varying-posture trials

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Conclusion

26

Introduction Human Modeling Materials and Methods Results Conclusion

Low-complexity, control-oriented model of the human arm.

Accurate for both constant- and varying-posture trials

Successful LPV modeling of the EMG-Force relationship.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Conclusion

26

Introduction Human Modeling Materials and Methods Results Conclusion

Low-complexity, con𝑡) is fundamental for the model.

arm.

Accurate for both constant- and varying-posture trials

Successful LPV modeling of the EMG-Force relationship.

The dynamics of 𝑞(𝑡) is fundamental for the model.

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Future Developments

27

Introduction Human Modeling Materials and Methods Results Conclusion

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Future Developments

27

Introduction Human Modeling Materials and Methods Results Conclusion

• Acquire more subject (ongoing…)

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Future Developments

27

Introduction Human Modeling Materials and Methods Results Conclusion

• Acquire more subject (ongoing…)

• Optimize the experimental protocol

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Future Developments

27

Introduction Human Modeling Materials and Methods Results Conclusion

• Acquire more subject (ongoing…)

• Optimize the experimental protocol

• Extend the model to the 2-DoF arm

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Future Developments

27

Introduction Human Modeling Materials and Methods Results Conclusion

• Acquire more subject (ongoing…)

• Optimize the experimental protocol

• Extend the model to the 2-DoF arm

• Integrate the model in the controller of the robot

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

The End!

28

Thank you for your attention!

Grazie per l’attenzione!

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Bibliography

29

[1] J. Hashemi et al., EMG-force modeling using parallel cascade identification. 2012

[2] J. Hashemi et al., Enhanced Dynamic EMG-Force Estimation Through Calibration

and PCI Modeling. 2015

[3] E. A. Clancy et. al, Identification of Constant-Posture EMG-Torque Relationship

About the Elbow Using Nonlinear Dynamic Models. 2012

[4] T. Söderström, P. Stoica, System Identification. 1989

[5] R. Tóth et al., Discrete-time LPV I/O and state-space representations, differences

of behavior and pitfalls of interpolation. 2007

[6] P. B. Cox, Towards Efficient identification of linear parameter-varying state-space

models. 2018

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Appendix A | Elbow Angle Dependency

30

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

𝑞𝑉 = 10°𝑞𝐼 = 20°

Appendix A | Elbow Angle Dependency

30

Δ𝑞 = 𝑞𝑉 − 𝑞𝐼 = −10°, 𝐹𝐼𝑇 = 72.42%, 𝑉𝐴𝐹 = 93.67%

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

𝑞𝐼 = 20° 𝑞𝑉 = 40°

Appendix A | Elbow Angle Dependency

30

Δ𝑞 = 𝑞𝑉 − 𝑞𝐼 = −10°, 𝐹𝐼𝑇 = 72.42%, 𝑉𝐴𝐹 = 93.67%Δ𝑞 = 𝑞𝑉 − 𝑞𝐼 = 20°, 𝐹𝐼𝑇 = 72.19%, 𝑉𝐴𝐹 = 95.10%

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

𝑞𝐼 = 20° 𝑞𝑉 = 40°𝑞𝑉 = 60°

Appendix A | Elbow Angle Dependency

30

Δ𝑞 = 𝑞𝑉 − 𝑞𝐼 = −10°, 𝐹𝐼𝑇 = 72.42%, 𝑉𝐴𝐹 = 93.67%Δ𝑞 = 𝑞𝑉 − 𝑞𝐼 = 20°, 𝐹𝐼𝑇 = 72.19%, 𝑉𝐴𝐹 = 95.10%Δ𝑞 = 𝑞𝑉 − 𝑞𝐼 = 40°, 𝐹𝐼𝑇 = 19.73%, 𝑉𝐴𝐹 = 84.18%

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Appendix B | Local LTI Models

31

Introduction EMG-driven Human Modeling Materials and Methods Results Conclusion

Local linear (LTI) models are sufficiently accurate

The Local Identification Framework for LPV modeling is legitimated

FIT [%]* VAF [%]*

User 1 84.88 97.70

User 2 87.51 98.44

Average 86.20 98.07

* estimation indicators

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Appendix C | The Hill’s Model

32

Introduction EMG-driven Human Modeling Materials and Methods Results Conclusion

Phenomenological modeling: the Hill’s model

𝐹: muscular force

𝐹 𝑡 = 𝐹0 𝑓 ℓ, 𝑣, 𝑡 ∙ 𝑎(𝑡) ℓ: muscular length

𝑣: contraction velocity

Muscular force 𝐹 𝑡 ∝ muscular activation 𝑎(𝑡) (computed from the raw EMG)

Proportional to the isometric force 𝐹0

The force is proportional to 𝑓 ℓ, 𝑣, 𝑡

Computationally complex and time consuming

Not suitable for online/control-oriented applications

EMG-driven Human Modeling to Enhance Human-Robot Interaction Control

Appendix D | Clancy’s Polynomial Model

33

Introduction EMG-driven Human Modeling Materials and Methods Results Conclusion

Black-box modeling: the EMG-Force relationship

2. Dynamic, nonlinear polynomial model [Clancy 2012]

• Elbow torque (𝜏) ∝ muscular activation (𝑎)

• Grey-box approach

𝜏 𝑘 =𝑑=1

𝐷

𝑞=0

𝑄

𝑒𝑞,𝑑 𝑎𝑒𝑑 𝑘 − 𝑞 +

𝑑=1

𝐷

𝑞=0

𝑄

𝑓𝑞,𝑑 𝑎𝑓𝑑 𝑘 − 𝑞

𝑎𝑒 : extensor activation𝑎𝑓 : flexor activation