extrinsic dexterity through active slip control using deep ... · manipulate objects within a robot...

6
Extrinsic Dexterity through Active Slip Control using Deep Predictive Models Simon Stepputtis 1 , Yezhou Yang 1 and Heni Ben Amor 1 Abstract—We present a machine learning methodology for actively controlling slip, in order to increase robot dexterity. Leveraging recent insights in deep learning, we propose a Deep Predictive Model that uses tactile sensor information to reason about slip and its future influence on the manipulated object. The obtained information is then used to precisely manipulate objects within a robot end-effector using external perturbations imposed by gravity or acceleration. We show in a set of experiments that this approach can be used to increase a robot’s repertoire of motor skills. I. INTRODUCTION Throughout our lifetime, we learn to delicately grasp, manipulate, and use a wide range of objects. Repeated physical contact with our environment allows us to acquire complex motor skills, such as in-hand rotation of objects. In doing so, we also learn to actively use slip to our advantage. By contrast, in the robot grasping literature, approaches to slip detection often aim at reducing or eliminating the effects of slip [1]–[6]. To this end, they require access to robust and accurate models of slip dynamics which, due to the non- deterministic nature of slippage, is a challenging problem in its own right. In this paper, we discuss how slip can be actively con- trolled to increase robot dexterity. Previous approaches to slip control have focused on a theoretical analysis of the underlying forces, torques, and physical constraints [4]. However, in practice, such models are often infeasible since they fail to represent the uncertainty and variability inherent to manipulation tasks involving slip. We argue that a critical component necessary for active slip control is a stochastic model of dynamic object behavior under such conditions. We therefore propose a Deep Predictive Model (DPM) which can be used to effectively learn the relationship between robot actions, incurred slip, and future object poses. A key feature of this approach is that we are able to manipulate a grasped object in-hand even when using a robot gripper with a single degree of freedom (DOF). This type of action is possible by utilizing extrinsic dexterity – external perturbations and forces such as gravity [7] that are actively utilized towards object manipulation. To account for the non-deterministic nature of slippage, we use stochastic forward passes (also referred to as Monte Carlo Dropout) [8] to generate a probabilistic belief over predicted object states. Accordingly, instead of relying on 1 Simon Stepputtis, Yezhou Yang and Heni Ben Amor are with the School of Computing, Informatics and Decision Systems Engineering, Arizona State University, 660 S. Mill Ave, Tempe, AZ 85281 {sstepput, yz.yang, hbenamor} at asu.edu Fig. 1. Slip can be utilized to rotate an object to different target locations. This image shows five example positions of a green bottle. The robot is able to utilize slippage to rotate the object to the desired angle. point-estimates, we can compute and analyze a distribution over object poses, thereby gaining a deeper understanding of the prediction process [8]. Finally, we perform experiments and provide examples for manipulating multiple objects having different surface structures, shapes and masses. These experiments show how this approach can be leveraged to achieve dexterous manipulation with robot hands that have a limited number of degrees of freedom. The experiments also provide evidence indicating that such learned models generalize to new, untrained objects. II. RELATED WORK A large body of work on modeling slip has focused on prediction and prevention [6] using tactile sensing [1]–[5]. Other research focused on utilizing tactile information to classify linear or rotational slip by training a neural network to discriminate between these two types of slip [9]. Going be- yond simple detection, slip has also been used for dexterous in-hand manipulation of grasped objects [10]. An approach that only uses tactile sensing to rotate an object is described in [11]. A drawback of that approach, however, is that it still requires an external object to support the movement instead of utilizing gravity. The work presented in [3], [4], [6] focuses on grasping an object with the lowest possible force

Upload: lydiep

Post on 05-Jun-2018

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Extrinsic Dexterity through Active Slip Control using Deep ... · manipulate objects within a robot end-effector using external ... we learn to delicately grasp, ... which can be

Extrinsic Dexterity through Active Slip Control usingDeep Predictive Models

Simon Stepputtis1, Yezhou Yang1 and Heni Ben Amor1

Abstract— We present a machine learning methodology foractively controlling slip, in order to increase robot dexterity.Leveraging recent insights in deep learning, we propose aDeep Predictive Model that uses tactile sensor information toreason about slip and its future influence on the manipulatedobject. The obtained information is then used to preciselymanipulate objects within a robot end-effector using externalperturbations imposed by gravity or acceleration. We show ina set of experiments that this approach can be used to increasea robot’s repertoire of motor skills.

I. INTRODUCTION

Throughout our lifetime, we learn to delicately grasp,manipulate, and use a wide range of objects. Repeatedphysical contact with our environment allows us to acquirecomplex motor skills, such as in-hand rotation of objects. Indoing so, we also learn to actively use slip to our advantage.By contrast, in the robot grasping literature, approaches toslip detection often aim at reducing or eliminating the effectsof slip [1]–[6]. To this end, they require access to robust andaccurate models of slip dynamics which, due to the non-deterministic nature of slippage, is a challenging problem inits own right.

In this paper, we discuss how slip can be actively con-trolled to increase robot dexterity. Previous approaches toslip control have focused on a theoretical analysis of theunderlying forces, torques, and physical constraints [4].However, in practice, such models are often infeasible sincethey fail to represent the uncertainty and variability inherentto manipulation tasks involving slip. We argue that a criticalcomponent necessary for active slip control is a stochasticmodel of dynamic object behavior under such conditions.

We therefore propose a Deep Predictive Model (DPM)which can be used to effectively learn the relationshipbetween robot actions, incurred slip, and future object poses.A key feature of this approach is that we are able tomanipulate a grasped object in-hand even when using arobot gripper with a single degree of freedom (DOF). Thistype of action is possible by utilizing extrinsic dexterity– external perturbations and forces such as gravity [7]that are actively utilized towards object manipulation. Toaccount for the non-deterministic nature of slippage, weuse stochastic forward passes (also referred to as MonteCarlo Dropout) [8] to generate a probabilistic belief overpredicted object states. Accordingly, instead of relying on

1Simon Stepputtis, Yezhou Yang and Heni Ben Amor are with the Schoolof Computing, Informatics and Decision Systems Engineering, ArizonaState University, 660 S. Mill Ave, Tempe, AZ 85281 {sstepput,yz.yang, hbenamor} at asu.edu

Fig. 1. Slip can be utilized to rotate an object to different target locations.This image shows five example positions of a green bottle. The robot isable to utilize slippage to rotate the object to the desired angle.

point-estimates, we can compute and analyze a distributionover object poses, thereby gaining a deeper understanding ofthe prediction process [8]. Finally, we perform experimentsand provide examples for manipulating multiple objectshaving different surface structures, shapes and masses. Theseexperiments show how this approach can be leveraged toachieve dexterous manipulation with robot hands that havea limited number of degrees of freedom. The experimentsalso provide evidence indicating that such learned modelsgeneralize to new, untrained objects.

II. RELATED WORK

A large body of work on modeling slip has focused onprediction and prevention [6] using tactile sensing [1]–[5].Other research focused on utilizing tactile information toclassify linear or rotational slip by training a neural networkto discriminate between these two types of slip [9]. Going be-yond simple detection, slip has also been used for dexterousin-hand manipulation of grasped objects [10]. An approachthat only uses tactile sensing to rotate an object is describedin [11]. A drawback of that approach, however, is that itstill requires an external object to support the movementinstead of utilizing gravity. The work presented in [3], [4], [6]focuses on grasping an object with the lowest possible force

Page 2: Extrinsic Dexterity through Active Slip Control using Deep ... · manipulate objects within a robot end-effector using external ... we learn to delicately grasp, ... which can be

Fig. 2. Schematic visualization of the deep predictive model used in this work. An example input for the DPM is shown on the left left, containing thestatic and dynamic tactile data. The right shows and Bayesian distribution over the output of the network for one exemplary value from v.

by calculating/estimating the friction coefficient between thegripper and the object. When utilizing slippage, knowingthe friction coefficient between the gripper and the object iscrucial. Especially the work in [4] made an important effortin determining this coefficient for unknown objects whiledynamically adjusting its estimation. However, the goal ofthis paper is to present a system that utilizes slippage to rotateobjects in-hand, rather than estimating friction, resulting in asimple grid-search to determine friction. In this work, slip isnot considered an undesired source of perturbation, but ratheras an opportunity for dexterous manipulation. Compared to[10], the presented system is able to solely rely on tactilesensing for in-hand manipulation and does not need to use asupporting object as in [11]. In addition, no analytical modelof the underlying mechanics and forces is needed as in [4],since the DPM is able to learn the necessary relations tosuccessfully manipulate the grasped objects.

III. METHODOLOGY

When utilizing slip to re-orient an object in the gripper,a robot needs to predict the implications of its actions onthe pose of the manipulated object. For this purpose, a DeepPredictive Model (DPM) is trained to predict the absoluterotation of the object at run time. More specifically, we traina neural network with three hidden layers, as shown in Figure2, to act as a DPM. The first two layers are recurrent layersto leverage temporal features, followed by a flattening layerand three fully connected dense layers which are separated byDropout layers. These Dropout layers will later be used forMonte Carlo Dropout [8]. The network structure was deter-mined empirically by running a grid search. Once trained, theDPM generates an estimate of the object’s future orientationfor the next 20 time steps. Predictions are generated basedon the static tactile data, allowing it to measure absolutepressure values and locations as well as dynamic tactile data.Dynamic sensors provide information about contact changesand events involving vibrations [12]. Another benefit ofdynamic sensors is their ability to quickly respond to changesin sensation [13], which is especially useful for detectingslippage. Due to their differential property, temporal featurescan be inferred from these sensors. Formally, the predictionprocess can be represented by the following equation byutilizing a sliding window of the past five time steps:

fn(st:t−5,dt:t−5)→ vt:t+20 (1)

Fig. 3. Overview of the objects used for training and evaluation. The twoleft most objects were used for training and are referenced as “Training A”and “Training B”. Both are equipped with an acceleration sensor.

TABLE IPROPERTIES OF THE USED OBJECTS. THE OBJECT NUMBERS

CORRESPOND THE THE OBJECTS SHOWN IN FIGURE 3 FROM LEFT TO

RIGHT. S-RAD REFERS TO THE OBJECTS SURFACE RADIUS.

PropertyObject 1 3 3 4 5 6 7 8

Mass [g] 519 99 174 119 124 121 194 104S-Rad. [cm] 4.6 1.6 3.2 2 2.2 3.8 2.1 2.5Deformable yes no no yes no yes no no

where s ∈ R56 is the vector of static tactile sensor activationsand d ∈ R2 is the vector of dynamic tactile sensor values.The output v is a vector containing the absolute rotationsfor the next twenty time steps {vt, · · · , vt+20}, where onetime step is set to a hundredth of a second. It is important tonote that the predicted angle is given relative to the gripper.In addition to the recurrent layers, the neural network canalso infer latent features about the changes from the dynamicsensors without solely resorting to recurrent layers. This isdue to the sensors’ ability to capture rapid changes that arecharacteristic for slippage [13].

Since slippage is a highly nondeterministic process, we usea Bayesian neural network to predict a distribution over itsresults [8]. To this end, we leverage recent theoretical insightsfrom [8] for estimating model uncertainty within the DPM.According to [8], multiple stochastic forward passes usinga stochastic version of dropout are identical to variationalinference in Gaussian processes. This is used to validatethe accuracy of the neural network regarding different targetangles of the test data set. For this purpose we activate

Page 3: Extrinsic Dexterity through Active Slip Control using Deep ... · manipulate objects within a robot end-effector using external ... we learn to delicately grasp, ... which can be

Fig. 4. The above sequences visualize two scenarios for in-hand manipulation using active slip control. The top row shows an experiment in which slipis used to keep the object in the same global rotation even though the gripper is rotating. The bottom row shows how to rotate an object using centrifugalforces without any rotation of the robot gripper. The main difference between these two experiments is the way in which slip is induced. Slip occurs aseither the result of rotation within the gripper or as an effect of the acceleration of the gripper.

dropout at test time to gain a better understanding of howthe network performs in certain regions of the input space.The results from these analyses can be found in section IV.

A. Training

For training purposes, the ground truth data is collectedby measuring the angle of the object with an accelerationsensor that is embedded or attached to the two trainingobjects, as can be seen in Figure 3. The objects werechosen based on their form and rigidity. Training objectA is slightly deformable, has a nearly flat surface andweighs about 500 grams. The second training object, objectB, is not deformable, has a radius of 1.5 centimeters andweights about 100 grams. The acceleration sensor is usedto automatically label the collected training data. The sensoris calibrated to accurately represent the angle of the objectbetween -20 and 220 degrees by using a Gaussian process[14]. The calibration was done by firmly holding the objectin the gripper of the robot and using its controls to changethe rotation of the gripper stepwise by one degree andcollecting the 3 DOF data from the accelerometer. Zerodegree was set to represent an upright object as can be seen inFigure 1 at the object’s twelve o’ clock position, continuingcounter clockwise with increasing value. The evaluation ofthe calibrated acceleration sensor resulted in an average errorof 0.4 degrees with a maximum error of 2.1 degrees.

Formally, the training data can be written as follows(Equation 2)

(st:t−5,d[t : t− 5], vt:t+20) (2)

s denotes a concatenated list of five vectors with the mostcurrent static tactile data. d holds the corresponding dynamictactile data over the past five time steps. v represents a vectorof the next t : t + 20 object angles that where taken fromthe training data. One training example is constructed byrandomly selecting one observation at time step t0, repre-senting the current observation (s,d). The sliding windowover s goes from t−5 to t0 and the values of v range from t0to t+20. Additionally we made sure that no training exampleexceeds the boundaries of one experiment, since multipledemonstrations were used in the training process. If n is thenumber of observations from a particular training example,the maximal chosen time t0 will be tn−20.

B. Run Time

The network uses the static and dynamic tactile data tomake a prediction regarding the object’s angle for the next20 time steps. Each prediction represent the position of theobject and is used to decide whether or not the gripper needsto close to stop the object at the desired position. The benefitof predicting multiple steps into the future is the abilityto select the correct amount of lookahead to resolve anynetwork and control latencies in the pipeline. Based on thepredictions, a simple controller as shown in Equation 3 isused to close the gripper as soon as the predicted angle a isequal to or greater than the target angle.

fc(v, l)→ a (3)

An important parameter within Equation 3 is the valueof l, which indicates the amount of lookahead used to best

Page 4: Extrinsic Dexterity through Active Slip Control using Deep ... · manipulate objects within a robot end-effector using external ... we learn to delicately grasp, ... which can be

−5 0 5 10Angle [degree]

0.00

0.02

0.04

0.06

0.08

0.10

0.12De

nsity

Fig. 5. Histogram of the predicted values when providing the neuralnetwork with the same data for 500 time steps. Results are shown for timestep 19 at a target angle of 90 degrees.

compensate for latencies. During training, the network doesnot experience any latencies, but at run time the predictionof v[0] gives a rotation that would already be obsoletewhen received by the controller due to latencies in thepipeline. Using the gripper controller solely based on theseold predictions leads to constantly overshooting the targetangle. On the other hand, making control decisions basedon information that is too far into the future will result inundershooting the target. Besides the latencies, the angularvelocity of the currently slipping object influences the latencyerror. If the object has a high angular velocity, it takes longerto slow down the object due to mass inertia until it is heldfirmly in the gripper. In this work, we empirically evaluatedthat a value of 19 for l yields to a good approximation ofthe final angle.

IV. EVALUATION

For the experiments, we are using a UR5 robotic armequipped with the Robotiq adaptive two finger gripper. Thetwo fingertips of the gripper are replaced with a tactile sensor[15]. Each of the two fingers has a 4x7 sensor matrix forstatic tactile data as well as one dynamic tactile sensor toleverage temporal features.

To train the network, a total of 200 demonstrations ofa 180 degree slip were recorded, 100 with every trainingobject, resulting in 90,200 samples. Since the object has adifferent angular velocity during the slipping motion, thesesamples are not equally distributed over all angles. Analyzingthe collected data showed that there are at least 200 samplesfor every angle between 0 and 180 degrees, which wasused as the upper limit for all other angles. Without thepreviously mentioned preparation of the training data, thenetwork would learn to have a strong preference towardslow angles where the movement was still slow, resulting inmuch more training data for these regions. The differenceis roughly a factor of 25. Additionally, all angles below 13degrees were cut out of the data set, since the robot wasgiven an initial slope of 10 degrees to allow slippage of thegrasped object. After these adjustments, the dataset contained

−30 −20 −10 0 10 20 30Angle [degree]

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Dens

ity

Legendtarget 20 | σ 6.0434target 40 | σ 2.76095target 60 | σ 2.49595target 80 | σ 2.40189target 100 | σ 3.84539target 120 | σ 7.11376target 140 | σ 8.26844target 160 | σ 10.3023

Fig. 6. Stochastic forward passes for different targets. Each distribution isbased on 500 predictions of the same input. The plot shows the distributionof the predicted angle for different targets. A smaller variance indicates aDPM with higher certainty in its predictions.

32,000 samples which were then split into 70% training, 20%validation and 10% testing data. Furthermore, all data pointsare normalized before they are used in the training process.To accurately represent the movement, all data points arecollected at a rate of 100 Hz, resulting in an averageresolution of one sample per degree. The demonstration endsas soon as the object reaches a total rotation of more than 180degrees, measured with the embedded acceleration sensorswithin each training object. One training example is roughlyillustrated in Figure 1 where the values mentioned aboveare collected while the object is rotating counter clockwise.Adaptive moment estimation (Adam) [16] is used for trainingusing a learning rate of 0.001 in combination with meansquared error as loss function.

We evaluated our approach with two experiments. Thefirst experiment uses the test data from the dataset, whereasthe second experiments evaluates the network with eightdifferent objects (3) on an actual robot (1).

A. Nondeterministic evaluation

For the nondeterministic evaluation we use stochasticforward passes to analyze the certainty of the neural network.This is helpful since slippage is a highly nondeterministicprocess. With this in mind, we randomly chose samples fromthe training set that were labeled with 90 degrees at v[19].This sample was provided 500 times to the DPM and theerror between the predictions and the 90 degree target wascollected. Figure 5 shows a histogram of the error betweenthe prediction and the target.

This result indicates that the predicted values can beapproximated using an exponential distribution. We assumethat a single Gaussian is feasible to represent the data. Basedon this assumption, Figure 6 shows the normal distributionfor different target angles between 20 and 160 degrees, using20 degree steps. µ is set to zero for all targets for an easiercomparison of the variance. Smaller variances indicate thatthe network has a higher certainty for the predicted angle.As for the previous experiment, the network was provided

Page 5: Extrinsic Dexterity through Active Slip Control using Deep ... · manipulate objects within a robot end-effector using external ... we learn to delicately grasp, ... which can be

0 20 40 60 80 100

120

140

160

180

Angles [degree]

−30

−20

−10

0

10

20

30Er

ror [

degr

ee]

Fig. 7. Overview of the neural network regarding different target angleson the test dataset. The error above is accumulated over all predicted timesteps.

with the same sample 500 times for each target angle. Thesamples were randomly chosen from the test set.

The distributions over the data shows that the precision ofthe network is highest for angles between 60 and 80 degrees.Between these two angles, the variance of the predictionswith respect to the ground truth is ∼ 2.5 degrees, whereas thevariance for higher angles is over 10 degrees. Even thoughthe ground truth data was collected with 100 Hz, fast objectmovements have a reduced accuracy compared to slowerobject movements during the early slippage. This explainsthe lower precision at higher angles. Another parameter thatcontributes to the uncertainty at higher angles is the latencyin the system, since its influence is higher when the objectis moving faster.

Figure 7 gives a more general overview over the error.Each box contains the accumulated error for one target angleover all time steps of v between t0 and t19. As alreadyseen in Figure 6, the error for predictions above 100 degreesincreases quickly. The region between 60 and 100 degreesprovides a nearly constant error with ±10 degrees, or ±4degrees between the upper and lower quantile. When onlyconsidering a single future time step like in Figure 6, theerror decreases to ±2.5 degrees within the first quantile. Theslightly increased error below 60 degrees can be explainedwith the occasional inability of the tactile sensor to distin-guish an object at 0 from the same object at 180 degrees.The average rotation time over all objects was approximately4 seconds.

B. Generalization with different objects

While the nondeterministic evaluation of the neural net-work helped to understand its properties and to empiricallyevaluate the structure of the network, we use a deterministicnetwork for the robot experiments by disabling dropout atrun time. This is done to get the average of all the differentsubnetworks that are trained with dropout [17].

At run time, the force used to grasp the object is crucialsince it needs to be high enough to still hold the object whilebeing low enough to allow rotational slippage. The amount of

Trainin

g A

Trainin

g B

Raque

tballs

Lister

ine

Gillette

Sham

poo

Salt

Deo

Objects

−100

−75

−50

−25

0

25

50

75

Erro

r [de

gree

]

-2.5 -1

9.5

-6.5

-1.0

1.0

12.0 3.

0

-28.

0

Fig. 8. Evaluation of eight objects (including the two training objects).The goal was to stop the object at exactly 90 degrees. Negative valuescorrespond to not reaching the target angle whereas positive values indicateovershooting the target. The blue numbers outline the average offset.

pressure between the object and the gripper depends on thesurface properties of the gripper and the object, as well as theobject’s mass. However, these parameters remain unknown.In this work, we used a grid-search approach to find thecorrect value for each object, such that the grasp allows theobject to slip. Prior to the actual experiment, every object wasevaluated regarding the amount of force needed to allow slip.The gripper can be controlled with 255 steps between fullyopen and closed, where 255 denotes a fully closed gripper. Tocontrol the force between the gripper and object, we openedthe gripper one unit at a time and observed if the graspedobject slips. When the slip for one value exceeded 90 degreeswithin 7 seconds, the smallest of these values was taken inthe experiment later on. For this test we opened the gripperbetween 0 and 15 units after a full grasp.

In this scenario, the robot picks an object and lifts itup to allow the object to rotate. The grasping process isdone in multiple steps. First, the object is grasped withfull force, followed by opening the gripper for an object-specific amount of degrees to prepare for slip. The grasp isvalidated by ensuring that the network is predicting an anglesmaller than 90 degrees for five consecutive time steps. Thisis because the DPM was never trained to handle the graspingprocess and to prevent any errors in the prediction due tountrained data. To initiate slippage, the robot then gives aninitial slope of 10 degrees to the object. This initial slope isenough to initiate rotational slippage, since the grasp of therobot is already loosened. However, it is still tight enough toprevent most of the linear slippage. The objective is to stopthe object precisely at 90 degrees.

Figure 8 shows the results of our experiments with eightdifferent objects. The shown data are based on ten targetapproaches with each object. The order of the objects (fromleft to right) is equivalent to the order of the objects asshown in Figure 3. On the y-axis, positive values indicatethat the target of 90 degrees was overshot, while negativevalues indicate that the object did not rotate enough before

Page 6: Extrinsic Dexterity through Active Slip Control using Deep ... · manipulate objects within a robot end-effector using external ... we learn to delicately grasp, ... which can be

the network predicted that the target was reached.All experiments with the robot use the same DPM that was

also evaluated in section IV-A. It can be seen from Figure 8that the DPM is able to generalize towards different objects.The objects were chosen due to their different propertieslike mass, shape and ductility as outlined in Table I. Eventhough different objects have a general offset that causesthem to slightly over- or undershoot the 90 degree target, thevariance of the actual object angles (excluding “Training A”)is still within an average ± 8 degree limit for most of theapproaches. For these experiments, the object angles weremanually measured since six of the eight objects did nothave an acceleration sensor. An interesting observation is thatthe result for the “Training A” object is noticeably worse incomparison to all other objects while overshooting the targetby up to 90 degrees. A possible explanation for the erroris the weight of the “Training A” object. Since the objectalready had a high mass inertia when reaching the target,the gripper had a notable problem to stop the object.

C. Dynamic movements

Especially during fast movements, where an object is morelikely to slip due to acceleration or centrifugal forces, ourapproach is able to utilize normally undesirable slippage toenhance the robot’s ability to manipulate objects in-hand.Two examples of these movements can be seen in Figure 4.Preliminary results with these experiments suggest that eventhough the physics of the experiment were changed to a fullydynamic task, the DPM is able to rotate the object to a 90-degree target. For these experiments, the same DPM wasused. Initial experiments showed that the the error betweenthe target and the actual object position slightly increased.

V. CONCLUSION

In this paper, a novel approach for dexterous manipulationis presented that leverages slippage to rotate an object in-hand. We showed that a single data-driven DPM is able toutilize external dexterity to precisely manipulate an object.A benefit of our approach is that we do not need to use anexternal object and we showed that gravity and accelerationare enough to conduct complex object manipulation evenwith a 1-DOF gripper. Compared to [10] and [4], ourapproach solely relies on tactile data and does not requirea detailed model of the environment. Experiments supportthe feasibility of this approach in real-world applications andindicate how the approach can increase a robot’s repertoireof skill. They further show that the DPM can generalize itspredictive ability to different, unknown objects.

Future work will focus on using a dynamic controllerto determine which predicted time step best compensatesfor latencies occurring in the pipeline. Furthermore, we canuse information about the uncertainty to better control therobot or record an additional training point. Knowledgeabout the distribution and precision can be incorporatedinto a controller that could further improve the handling of

a grasped object. Another aspect for further investigationis the determination of the necessary friction dynamicallyat run time. A challenge with this approach is that therobot’s movement is only done once, leaving no room forexploration.

VI. ACKNOWLEDGEMENT

The authors gratefully acknowledge Robotiq Inc. for theirgenerous support of this research. All experiments wereperformed using the Robotiq Tactile Sensor beta kit.

REFERENCES

[1] F. Veiga, H. van Hoof, J. Peters, and T. Hermans, “Stabilizingnovel objects by learning to predict tactile slip,” in 2015 IEEE/RSJInternational Conference on Intelligent Robots and Systems (IROS),pp. 5065–5072, Sept 2015.

[2] Y. Ma, Y. Huang, L. Mao, P. Liu, C. Liu, and Y. Ge, “Pre-slidingdetection in robot hand grasping based on slip-tactile sensor,” in2015 IEEE International Conference on Robotics and Biomimetics(ROBIO), pp. 2603–2608, Dec 2015.

[3] M. Stachowsky, T. Hummel, M. Moussa, and H. A. Abdullah, “Aslip detection and correction strategy for precision robot grasping,”IEEE/ASME Transactions on Mechatronics, vol. 21, pp. 2214–2226,Oct 2016.

[4] A. Cirillo, P. Cirillo, G. D. Maria, C. Natale, and S. Pirozzi, “Controlof linear and rotational slippage based on six-axis force/tactile sensor,”in 2017 IEEE International Conference on Robotics and Automation(ICRA), pp. 1587–1594, May 2017.

[5] A. Cavallo, G. De Maria, C. Natale, and S. Pirozzi, “Slipping detectionand avoidance based on kalman filter,” vol. 24, 08 2014.

[6] J. S. Shaw and V. Dubey, “Design of servo actuated robotic gripperusing force control for range of objects,” in 2016 InternationalConference on Advanced Robotics and Intelligent Systems (ARIS),pp. 1–6, Aug 2016.

[7] N. C. Dafle, A. Rodriguez, R. Paolini, B. Tang, S. S. Srinivasa,M. Erdmann, M. T. Mason, I. Lundberg, H. Staab, and T. Fuhlbrigge,“Extrinsic dexterity: In-hand manipulation with external forces,” inRobotics and Automation (ICRA), 2014 IEEE International Conferenceon, pp. 1578–1585, IEEE, 2014.

[8] Y. Gal, Uncertainty in Deep Learning. PhD thesis, University ofCambridge, 2016.

[9] Z. Su, K. Hausman, Y. Chebotar, A. Molchanov, G. E. Loeb,G. S. Sukhatme, and S. Schaal, “Force estimation and slip detec-tion/classification for grip control using a biomimetic tactile sensor,” in2015 IEEE-RAS 15th International Conference on Humanoid Robots(Humanoids), pp. 297–303, Nov 2015.

[10] C. A. Jara, J. Pomares, F. A. Candelas, and F. Torres, “Controlframework for dexterous manipulation using dynamic visual servoingand tactile sensors feedback,” Sensors, vol. 14, no. 1, pp. 1787–1804,2014.

[11] H. van Hoof, T. Hermans, G. Neumann, and J. Peters, “Learning robotin-hand manipulation with tactile features,” in 2015 IEEE-RAS 15thInternational Conference on Humanoid Robots (Humanoids), pp. 121–127, Nov 2015.

[12] M. R. Cutkosky and J. Ulmen, Dynamic Tactile Sensing, pp. 389–403.Cham: Springer International Publishing, 2014.

[13] K. Stern, “What is dynamic tactile sensing?,” Jun 2017.[14] Z. Geng, F. Yang, X. Chen, and N. Wu, “Gaussian process based

modeling and experimental design for sensor calibration in driftingenvironments,” Sensors and Actuators B: Chemical, vol. 216, pp. 321–331, sep 2015.

[15] T.-H.-L. Le, A. Maslyczyk, J.-P. Roberge, and V. Duchaine, “Ahighly sensitive multimodal capacitive tactile sensor,” InternationalConference on Robotics and Automation, 2017.

[16] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,”International Conference on Learning Representations, 2015.

[17] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut-dinov, “Dropout: A simple way to prevent neural networks fromoverfitting,” Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014.