[chi2016] gaussmarbles: spherical magnetic tangibles for interacting with portable physical...
TRANSCRIPT
GaussMarbles: Spherical Magnetic Tangibles for Interacting with Portable Physical Constraints Han-Chih Kuo, Rong-Hao Liang, Long-Fei Lin, Bing-Yu Chen National Taiwan University
Interactive Surface Constructive Assembly Token+Constraint
Token+Constraint Systems for Tangible Interaction with Digital Information (Ullmer et al. TOCHI ‘05)
Tangible User Interfaces
Interactive Surface Constructive Assembly Token+Constraint
MagGetz (Hwang et al. UIST ’13)
Tangible Magnetic Appcessories (Bianchi et al. TEI ’13)
Tangible Remote Controllers for Wall-size Displays (Jansen et al. CHI ’12)
Tracking Token+Constraint Interactions on Portable PlatformsToken + Strict Constraint
Tracking Ball+Constraint Interactions on Stationary PlatformsBall + Loose ConstraintinFORM (Follmer et al. UIST ’13)
Ball + Constraint Interactions on Portable Platforms
Tracking Ball+Constraint Interactions on Stationary Displays
camera camera
magnetic-field camera
�
�
SURWRW\SH�� ZH� SULQWHG� WKH� IL[WXUHV� RXW� RI� SODVWLF� XVLQJ� D�'LPHQVLRQ� (OLWH� �'� SULQWHU�� :KLOH� SULQWHG� SODVWLF� ZDV�VDWLVIDFWRU\� IRU� SURWRW\SLQJ� WKH� IL[WXUHV�� ZH� EHOLHYH� WKDW�VWURQJHU�SODVWLF�RU�HYHQ�DOXPLQXP�LV�SUHIHUUHG��7R�SURYLGH�DGGLWLRQDO�UREXVWQHVV�WR�EUHDNLQJ��IOH[LEOH�KLQJHV�VLPLODU�WR�WKRVH�XVHG�LQ�H\H�JODVVHV�FRXOG�DOORZ�3RUWLFR¶V�DUPV�WR�EH�EHQW� RXWZDUGV�� 6XFK� D� GHVLJQ� LV� SDUWLFXODUO\� LPSRUWDQW�ZKHQ� 3RUWLFR� LV� XVHG� RQ� WKH� IORRU�� RU� XVHG� E\� FKLOGUHQ�SOD\LQJ� D� JDPH�� H�J��� RXU� 3HQDOW\� 6KRRWRXW� DSSOLFDWLRQ��GHVFULEHG�EHORZ��� )RU� RXU� SURWRW\SH��ZH� XVHG� D� ����'HOO�/DWLWXGH�;7� FRQYHUWLEOH� WDEOHW� WKDW� VXSSRUWV� ERWK� SHQ� DQG�ILQJHU�PXOWL�WRXFK� LQSXW��:H� DOVR� XVHG� D� SDLU� RI�/RJLWHFK�:HEFDP� 3UR� ����� FDPHUDV� H[WUDFWHG� IURP� WKHLU� RULJLQDO�KRXVLQJV��$V�VKRZQ�LQ�)LJXUH����HDFK�FDPHUD�LV�PRXQWHG�DW�WKH� HQG� RI� D� IROGDEOH� DUP� DWWDFKHG� WR� HDFK� VLGH� RI� WKH�VFUHHQ�� :H� GHVLJQHG� RXU� V\VWHP� VXFK� WKDW� ZKHQ� IROGHG�GRZQ��WKH�DUPV�DQG�FDPHUDV�DUH�IOXVK�DJDLQVW�WKH�WDEOHW�DQG�GR�QRW�LQWHUIHUH�ZLWK�QRUPDO�XVH���:KHQ�WKH�DUPV�DUH�UDLVHG��WKH�FDPHUDV�DUH�EHQW�GRZQ�XVLQJ�D�VLQJOH�URWDWLRQ�KLQJH�WR�DOORZ�WKHP�WR�VHH�WKH�WDEOHW�VFUHHQ�DQG� WKH� WDEOHWRS�� )LJXUH� �D� VKRZV� DQ� LOOXVWUDWLRQ� RI� WKH�XQLILHG� DUHD� YLHZDEOH� E\� WKH� WZR� FDPHUDV�� )LJXUH� �E�LOOXVWUDWHV�WKH�V\VWHP�UHDFWLQJ�WR�REMHFWV�RQ�DQG�DURXQG�WKH�VFUHHQ�� )LQDOO\�� WR� DYRLG� WKH� V\VWHP� FRQIXVLQJ� GLJLWDO�HOHPHQWV� GUDZQ� RQ� WKH� VFUHHQ� ZLWK� SK\VLFDO� REMHFWV�� ZH�DSSOLHG� OLQHDU�SRODUL]LQJ� ILOWHUV� WR� WKH� VFUHHQ�DQG�FDPHUDV���1RWH� WKDW� ZKLOH� /&'V� DUH� OLQHDUO\� SRODUL]HG�� LQ� PDQ\�WDEOHWV� WKH� WRXFK�VHQVLWLYH� HOHPHQW� GLIIXVHV� WKH� OLJKW��UHTXLULQJ�DQRWKHU�SRODUL]LQJ�OD\HU����&RPSXWHU�9LVLRQ�6\VWHP�:H� LPSOHPHQWHG� D� FRPSXWHU� YLVLRQ� V\VWHP� WR� DOORZ�DSSOLFDWLRQV� WR� VXSSRUW� LQWHUDFWLRQV� ZLWK� REMHFWV� RQ� WKH�WDEOHW� DQG� WKH� VXUIDFH� DURXQG� LW�� 7KH� YLVLRQ� V\VWHP� ZDV�ZULWWHQ�SUHGRPLQDQWO\� LQ�3\WKRQ�DQG�XVHV�,QWHO¶V�2SHQ&9�OLEUDU\��:H�QRZ�GHVFULEH� WKH�YLVLRQ�V\VWHP�LQ�VRPH�GHWDLO�WR�HQDEOH�UHDGHUV�WR�UHSOLFDWH�RXU�V\VWHP��$V� VKRZQ� LQ� )LJXUH� �� �QH[W� SDJH��� WKH� YLVLRQ� V\VWHP�ZDV�LPSOHPHQWHG� LQ� D� WKUHDGHG�� KLHUDUFKLFDO� VWUXFWXUH�� $W� WKH�FRUH�RI�WKH�YLVLRQ�V\VWHP�LV�WKH�&DPHUD�0RGXOH��ZKLFK�LV�UHVSRQVLEOH� IRU� UHWULHYLQJ� IUDPHV� IURP� D� VLQJOH� FDPHUD��
SHUIRUPLQJ� DQ� RSWLRQDO� EDFNJURXQG� VXEWUDFWLRQ� VWHS�� DQG�JDWKHULQJ� GHWHFWHG� REMHFWV� IURP� GLIIHUHQW� 9LHZ� 0RGXOHV��(DFK� 9LHZ� 0RGXOH� LV� UHVSRQVLEOH� IRU� PDQLSXODWLQJ� UDZ�FDPHUD� IUDPHV� WR� SURGXFH� D� VSHFLILF� VLPXODWHG� YLHZ�� DQG�FRQWDLQV�D� VHW�RI�REMHFW� UHFRJQLWLRQ�FODVVLILHUV� WKDW�RSHUDWH�RQ�WKH�VLPXODWHG�YLHZ��7KHVH�YLHZV�DUH�XVHIXO�VLQFH��DV�FDQ�EH� VHHQ� LQ� )LJXUH� �D�� LQ� RUGHU� IRU� WKH� FDPHUD� WR� VHH� WKH�WDEOHW� VFUHHQ� DQG� WKH� VXUURXQGLQJ� VXUIDFH� �DQG� VWLOO� EH�IROGHG�GRZQ�ZKHQ�QRW� LQ�XVH��� WKH�FDPHUDV�PXVW�YLHZ�WKH�ZRUOG� IURP� D� YHU\� REOLTXH� DQJOH�� 2XU� V\VWHP� XVHV� WZR�&DPHUD� 0RGXOHV�� RQH� IRU� HDFK� FDPHUD�� %RWK� &DPHUD�0RGXOHV� DQG� DOO�9LHZ�0RGXOHV� DUH� WKUHDGHG� WR� DOORZ� WKH�YLVLRQ�V\VWHP�HOHPHQWV�WR�RSHUDWH�FRQFXUUHQWO\��$�VLQJOH�SDVV�RI�WKH�YLVLRQ�V\VWHP�FRQVLVWV�RI�WKHVH�VWHSV����� *UDEELQJ� FDPHUD� IUDPHV�� (DFK� &DPHUD� 0RGXOH� LV�FRQQHFWHG�WR�D�VLQJOH�FDPHUD�GHYLFH��8SRQ�UHTXHVW�IURP�WKH�3HUFHSWLRQ� 0DQDJHU�� D� &DPHUD� 0RGXOH� UHWULHYHV� D� QHZ�IUDPH� IURP� WKH� FDPHUD� DQG� SDVVHV� LW� WR� HDFK� RI� LWV� 9LHZ�0RGXOHV����� 2SWLRQDO� EDFNJURXQG�VXEWUDFWLRQ�� 3RUWLFR� SURYLGHV� D�EDFNJURXQG� VXEWUDFWLRQ� FDSDELOLW\� DQG� RIIHUV� LW� DV� DQ�RSWLRQDO� VWHS� LQ� WKH� SHUFHSWLRQ� SURFHVV� SULRU� WR� YLHZ�JHQHUDWLRQ� DQG� REMHFW� UHFRJQLWLRQ�� :H� XVH� D� *DXVVLDQ�0L[WXUH�0RGHOV�DSSURDFK�IRU�EDFNJURXQG�VXEWUDFWLRQ�>���@��ZLWK�D�GHGLFDWHG�PRGHO� IRU�HDFK�FDPHUD��:KHQ�DSSOLHG��D�EDFNJURXQG�IRUHJURXQG�PDVN� LV�SURGXFHG�DQG� LV�SDVVHG� WR�HDFK�9LHZ�0RGXOH�DORQJ�ZLWK�WKH�UDZ�FDPHUD�IUDPH����� 3URGXFLQJ� VLPXODWHG� FDPHUD� YLHZV�� 8VLQJ� FDOLEUDWLRQ�KRPRJUDSK\� FRPSXWHG� GXULQJ� V\VWHP� VHWXS�� D� 9LHZ�0RGXOH�FDQ�SURGXFH�RQH�RI�D�VHW�RI�VLPXODWHG�YLHZV� WR�EH�XVHG� IRU� REMHFW� GHWHFWLRQ�� (DFK� &DPHUD� 0RGXOH� DOVR�FRQWDLQV� D� 5DZ� 9LHZ� 0RGXOH� UHVSRQVLEOH� IRU� REMHFW�UHFRJQLWLRQ�GRQH�RQ�WKH�UDZ�FDPHUD�YLHZ��VHH�)LJXUH��D�E���:H� KDYH� LPSOHPHQWHG� VXSSRUW� IRU� D� QXPEHU� RI� VLPXODWHG�YLHZV��2XU�SURRI�RI�FRQFHSW�DSSOLFDWLRQV��GHVFULEHG�EHORZ��PDNH�XVH�RI�WKH�IROORZLQJ�WZR�VLPXODWHG�YLHZV��7KH�6FUHHQ�DQG�%H]HO�9LHZ��)LJXUH��F��SURGXFHV�D�UHFWLILHG�YLHZ�RI�WKH�WDEOHW� VFUHHQ� DQG�EH]HO� DQG� LV� XVHIXO� IRU� SHUIRUPLQJ�PRUH�SUHFLVH�GHWHFWLRQ�RI�REMHFWV�SODFHG�RQ�WKH�WDEOHW��7KH�7DEOH�9LHZ� �)LJXUH� �G�H�� SURGXFHV� D� UHFWLILHG�YLHZ�RI� WKH� WDEOHW�
� �)LJXUH����3RUWLFR�V\VWHP�LOOXVWUDWLRQV��7KH�XQLILHG�DUHD�YLHZDEOH�E\�WKH�WZR�FDPHUDV��OHIW���DQG�WKH�V\VWHP�UHDFWLQJ�WR�REMHFW�RQ�DQG�DURXQG�WKH�WDEOHW��ULJKW���
Paper Session: Tangible UIST’11, October 16–19, 2011, Santa Barbara, CA, USA
349
Better Form Factors
Vision-based Object Tracking Technologies
GaussSense (Liang et al. UIST ’12)
Portico(Avrahami et al. UIST ’11)
Challenges of Magnetic TrackingOne of the bipolar magnetic field may disappear while rolling
Solution: Magnetic Regular PolyhedronExpanding magnetic fields in equal dihedral angles to make the bipolar magnetic fields visible at any direction
Solution: Magnetic Regular PolyhedronExpanding magnetic fields in equal dihedral angles to make the bipolar magnetic fields visible at any direction
Explorative Study
different sizesdifferent faces
Form factors vs. tracking performances
Form factors vs. tracking performancesExperimental Apparatus
servo motor
stabilizeranalog Hall-sensor grid
Form factors vs. tracking performancesExperimental Apparatus
servo motor
stabilizeranalog Hall-sensor grid
Form factors vs. tracking performancesExperimental Apparatus
single magnet 16mm-radius
6-face 13.5,16,18.5,21mm-radius
4, 6, 8, 12-face 16mm-radius
servo motor
stabilizeranalog Hall-sensor grid
Form factors vs. tracking performancesExperimental Apparatus
8 (units) × 5 (positions) × 4 (hover heights) × 10 (angles) × 100 (samples) = 160,000 bitmaps of magnetic fields
single magnet 16mm-radius
6-face 13.5,16,18.5,21mm-radius
4, 6, 8, 12-face 16mm-radius
Form factors vs. tracking performancesData Processing
Bipolar-blob contours North-blob contours South-blob contours
3 types of contour
8 (units) × 5 (positions) × 4 (hover heights) × 10 (angles) × 100 (samples) = 160,000 bitmaps of magnetic fields
(max Intensity M and blob Area A)
Form factors vs. tracking performancesData Processing
Bipolar-blob contours North-blob contours
1. Centroid of contours
2. Centroid of pixels (in all contours)
3. Centroid of mass (in all contours)
3 types of contour x
8 (units) × 5 (positions) × 4 (hover heights) × 10 (angles) × 100 (samples) = 160,000 bitmaps of magnetic fields
(max Intensity M and blob Area A) 3 types of centroid
South-blob contours
9 Distributions of centroids
Form factors vs. tracking performancesData Processing
Bipolar-blob contours North-blob contours
1. Centroid of contours
2. Centroid of pixels (in all contours)
3. Centroid of mass (in all contours)
3 types of contour3 types of centroidx d
8 (units) × 5 (positions) × 4 (hover heights) × 10 (angles) × 100 (samples) = 160,000 bitmaps of magnetic fields
(max Intensity M and blob Area A) =
South-blob contours
9 Distributions of centroids
Form factors vs. tracking performancesData Processing
Bipolar-blob contours North-blob contours
1. Centroid of contours
2. Centroid of pixels (in all contours)
3. Centroid of mass (in all contours)
d
mean(d),std(d): measured dispersion
8 (units) × 5 (positions) × 4 (hover heights) × 10 (angles) × 100 (samples) = 160,000 bitmaps of magnetic fields
3 types of contour3 types of centroidx(max Intensity M and blob Area A) =
South-blob contours
0
1
2
3
4
5
6
1M P4 6M 8M 12M
IN-OC IN-OP IN-OM IS-OC IS-OP IS-OM IB-OC IB-OP IB-OMOc-IN Op-IN Om-IN Oc-IS Op-IS Om-IS Oc-IB Op-IB Om- IBN
/ A
N /
A
N /
A
0
1
2
3
4
56
(mm)
dist
ance
(d)
1. Bi-polar centroid of mass is the most stable feature for xy-plane tracking.
single magnet 16mm-radius
4-face 16mm-radius
6-face 16mm-radius
8-face 16mm-radius
12-face 16mm-radius
Summary of Findings
Summary of Findings
0
1
2
3
4
5
6
1M P4 6M 8M 12M
IN-OC IN-OP IN-OM IS-OC IS-OP IS-OM IB-OC IB-OP IB-OMOc-IN Op-IN Om-IN Oc-IS Op-IS Om-IS Oc-IB Op-IB Om- IBN
/ A
N /
A
N /
A
0
1
2
3
4
56
(mm)
dist
ance
(d)
1. Bi-polar centroid of mass is the most stable feature for xy-plane tracking.
single magnet 16mm-radius
4-face 16mm-radius
6-face 16mm-radius
8-face 16mm-radius
12-face 16mm-radius
2. Polyhedrons support z-axis tracking by using the product of south-blob area and intensity (AS × MS), and a single magnet does not.
3 mm
6 mm
9 mm
0
50
100
150
200
0
50
100
150
200
0
50
100
150
200
0
50000
100000
150000
0
50000
100000
150000
0
50000
100000
150000
(gau
ss)
200
150
100
50
0
150000
100000
50000
0
(gau
ss x
mm
2 )
a b4-face 6-face 8-face 12-face
Summary of Findings
012345678
1 2 3 4
��1 ��2 ��3 ��4
h = 12 mm
h = 9 mm
h = 6 mm
h = 3 mm
012345678
1 2 3 4
��1 ��2 ��3 ��4mm
h = 3 mm h = 12 mmh = 9 mmh = 6 mm
Om- IB24
0
2
4
0
6
8
202020
(mm) (mm)
dist
ance
(d)
dist
ance
(d)
a b
3. More faces yield greater accuracy, but accuracy drops as sensing distance increases.
(cont’d)
4face 6face 8face 12face
012345678
1 2 3 4
��1 ��2 ��3 ��4
h = 12 mm
h = 9 mm
h = 6 mm
h = 3 mm
012345678
1 2 3 4
��1 ��2 ��3 ��4mm
h = 3 mm h = 12 mmh = 9 mmh = 6 mm
Om- IB24
0
2
4
0
6
8
202020
(mm) (mm)
dist
ance
(d)
dist
ance
(d)
a b
3. More faces yield greater accuracy, but accuracy drops as sensing distance increases.
Summary of Findings
4face 6face 8face 12face 4. Smaller polyhedrons yield slightly greater accuracy.
012345678
1 2 3 4
��1 ��2 ��3 ��4
h = 12 mm
h = 9 mm
h = 6 mm
h = 3 mm
012345678
1 2 3 4
��1 ��2 ��3 ��4mm
h = 3 mm h = 12 mmh = 9 mmh = 6 mm
Om- IB24
0
2
4
0
6
8
202020
(mm) (mm)
dist
ance
(d)
dist
ance
(d)
a b
13.5mm radius
16mm radius
19.5mm radius
21mm radius
(cont’d)
Designing Ball + Constraint Interactions on Portable PlatformsApplications
Interacting with Constraints on a DisplayUsing Physical Constraint to Manipulate a Ball
Interacting with Constraints on a DisplayEmbodied Gestures
Interacting with Constraints on a DisplayBall+Constraint Interactions on Handheld Displays
Interacting with Constraints on a Display
conductive rubber
Touch interactions on a magnetic sphere
Interacting with Constraints on a DisplayClay-made territory as a continuous constraint
Interacting with Constraints on a DisplayAround-Device Interactions
Interacting with a Constraint Nearby a DisplayUsing Physical Constraints to Bridge Digital- and Real-world Experiences
Interacting with a Constraint Nearby a DisplayUsing Physical Constraints to Bridge Digital- and Real-world Experiences
Interacting with a Constraint Distant from a Display
Velcro strap
High-friction Materials as Physical Constraints
Interacting with a Constraint Distant from a Display
Velcro strap
High-friction Materials as Physical Constraints
GaussMarbles: Spherical Magnetic Tangibles for Interacting with Portable Physical Constraints Han-Chih Kuo, Rong-Hao Liang, Long-Fei Lin, Bing-Yu Chen National Taiwan University
Thanks! Questions?
The proposed magnetic regular polyhedron design enables • stable 3D tracking by an analog Hall-sensor grid
without priori knowledge • the explorations of ball+constraint interaction
on portable platforms
Conclusion