[uist 2015] flexibend: enabling interactivity of multi-part, deformable fabrications

Hi everyone, I’m Rong-Hao, and we are from National Taiwan university and Keio university.Today we are going to talk about FlexiBend, which is a shape-sensing strip that enables interactivity of multi-part, deformable fabrications.

By installing the FlexiBend into a deformable fabrication, The user interactions can be sensed by tracking the shape of sensor.

It also supports fabrications of multiple movable parts, such as widgets.

Several previous works present techniques to enable interactivity of deformable and multi-part fabrications using mechanical, optical, and capacitive sensors.

For these techniques, the hardest part for makers is installing the numerous sensors in correct ways.

Installing single sensor unit is way easier.Possible methods such as active swept-frequency sensing

passive auditory sensing

and time-domain reflectometry sensing.

However, they fail to capture high-dimensional deformations, say, they can only sense one operation at a time.

Using single camera can track multiple movable part, such as Sauron,

but the line-of-sight issues limit the physical design as well.

For both reliable tracking and easy installation, our idea is using a single shape-sensing strip for tracking.

By stuffing the sensing strip to a deformable fabrication,

or a multi-part fabrication, the user interactions can be reliably tracked based on the sensing data. Therefore, we can turn deformable or multi-part fabrications into computer input devices.

First of all, we attempted to develop thin, flexible shape-sensing strips for embedded uses in fabrications. The conventional solution, ShapeTape, turns out to be the possible candidate, but the resolution is not high enough.

Therfore, we further explore other bend sensors, including flexSense, Flex, Bend micros.Finally we found that strain gauge can be an interesting one to try, because it’s the smallest one.

A strain gauge is a thin and small bend sensor typically used for monitoring deformation in rigid structures.Deformation changes the electrical resistance of a strain gauge.

To determine how deformation actually affects the resistance reading of strain gauges. We made six short flexible strips, each of which contains one strain gauge. Each one was fit to a set of plastic molds that described semicircles of 13 different radiuses.

The experimental results show that the sensor readings are linearly related to the bend angles. So, we can move on fabricating a strain gauge array as a shape sensor.

We implement a sixteen-link strain gauge array.

and 3D printed the substrate using the flexible filament, NinjaFlex.

The sensing data of all sixteen strain gauges were collected by a main board consisting of a micro-controller, a ADC, and two multiplexers, in a 45 fps refresh rate

Because it’s resistive-based sensing, the power consumption is relatively low.

For each frame, we capture the sensor data,

subtract the background first

Then, measure the deformation by offset data.

We then obtain the curvature of sensor by mapping the sensor data to each sensor’s response curve, which is obtained from the calibration process. Please read the paper for the details.

Based on the curvature of each sensor, we can construct the shape of FlexiBend.

Now, we have a series of sixteen curvatures

The first thing is to add the spaces between the curvatures. We already know the actual length of space when it is fabricated.

Then, we turn each curvature into several control points, for example, four points.

Finally, we can use NURBS to connect the control points as a resulting shape.

To evaluate the performance of shape construction, we used a DI platform to capture the ground truth with the reconstructed shape.

We tested 17 example shapes, and the result show that the mean position error of each joint is only 7 millimeter. Although the error is cumulated at the end point, the error is still less than 2 cm, that is sufficiently accurate for our application.

To fabricate deformable object for FlexiBend, we 3D print the substrate using NinjaFlex as well. A grove is preserved on the model for installing the FlexiBend.

By placing the FlexiBend into the grove, the shape is captured at the same time. After that, the user interactions can be easily detected by resolving the differences.

For storytelling, a user can change the expression of a virtual seahorse displayed on the screen by bending the physical seahorse puppetry, making it looks confident and proud.

Using common rigid filaments, such as PLA or ABS, can also fabricate multi-part widgets such as buttons.A widget consists of an entrance, a cavity, and an exit for the FlexiBend, and a joint is used to keep the green part movable.

However, it’s not enough. Because the moving part affects the FlexiBend outside the widgets, the challenge turns out to be “how to isolate each widgets”.and that would makes the remaining parts of strain gauge reusable for other widgets.

Our solution, is the locking mechanism.The landmarks on the entrance and exit of the case lock the gear-shape pattern on the edges of the FlexiBend.

The locking mechanism fixed and isolated the segment of FlexiBend, that is loosely mounted within the widgets.Because the FlexiBend is loosely mounted, the green part is still movable.Additionally, the 3D printed physical spring can recover the FlexiBend to its original state.

So that’s how a button works, note that the button is a pressure-sensitive, analog button.

We also design switch, which has no physical spring but a lever-like structure instead

A knob with a rollable handle that deforms the FlexiBend can be used as a dial.

With the same idea, we made a slider as well.

Our experiment result shows that a 3 centimeter slider can reliably distinguish 8 positions,

Furthermore, it shows that the locking mechanism is effective,because the strain gauges outside the widget were not affected at all.

To wrap up, we show how to turn a toy pistol fabrication into an input device for a first-person shooting game.The user installs a flexiBend into the toy pistol, which has a slider and a button.

Then, the user records the states of a the input events, such as the button and the slider positions. Note that the operations on different widgets can be performed simultaneously.

Then, the user can select the weapon by moving the slider, confirm by clicking the button,

then fire and reload the bullets intuitively.

By adding a motion sensor on a FlexiBend, it further supports 3D spatial operations such as wielding the knife.

Also, from the raw data’s perspective, forming a FlexiBend into a spiral shape can sense subtle 3D operations, such as twisting, stretching, and bending it, although the 2D sensor does not know that its a spiral.

Customizing a flexiBend by making it longer and denser allows for high-resolution and simultaneous detection on more widgets, or even sensing human body as SensorTape just did. Making FlexiBend using Flexible printed circuits also make a flexiBend thinner and more durable, because it can reduce the structure deficiencies of fabrication.

Our prototype made by cooper tapes demonstrates this possibility.

Conclusion. We presented FlexiBend, a novel shape-aware strip that brings interactivity to multi-part, deformable objects with ease.

Because it is so easy to install and reusable, it can be an useful tool for iterative prototyping, because a user can easily optimize the 3D model design with the same flexiBend. Future work can also consider to provide modularized 3D modeling tools to accelerate the iterations of design and fabrication. It’s a promising area to further explore.

Thanks for your attention, and I’m glad to take all questions.