Upload File

toward mixed-initiative clustering yifen huang tom m. mitchell carnegie mellon university agents...

  • Home
  • Documents
  • Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009
27
Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Upload: gertrude-hawkins

Post on 21-Jan-2016

214 views

Category:

Documents


2 download

Report

Tags:

TRANSCRIPT

Page 1: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Toward Mixed-Initiative Clustering

Yifen HuangTom M. Mitchell

Carnegie Mellon University

Agents that Learn from Human TeachersMarch 23, 2009

Page 2: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Semi-supervised clustering:A user performs an oracle role.

Unsupervised clustering:A machine builds the model alone.

Page 3: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Semi-supervised clustering:A user performs an oracle role.

Unsupervised clustering:A machine builds the model alone.

Mixed-Initiative Clustering

Page 4: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Key Question

• How can autonomous clustering algorithms be extended to enable mixed-initiative clustering approaches involving an iterative sequence of computer-suggested and user-suggested revisions to converge to a useful hierarchical clustering?

– From autonomous clustering to mixed-initiative clustering

– From flat feedback to hierarchical feedback

Page 5: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Activity X contains this list of emails:## ## ## ## ##An email from Andrea

Thomaz belongs to your AAAI symposium activity.

Adam Cheyer is a key-person to your CALO activity.

Page 6: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Too lazy to comment.

Activity X contains this list of emails:## ## ## ## ##An email from Andrea

Thomaz belongs to your AAAI symposium activity.

What the hell is this?? DELETE!

This is correct.Adam Cheyer is a key-

person to your CALO activity.

Page 7: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Computer-to-user language: hypotheses

User-to-computer language: modified hypotheses

Model adaptation algorithm

Framework for Mixed-Initiative Clustering

Page 8: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

User Interface

Page 9: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Communicative Languages inSemi-Supervised Clustering

Cluster Document

Page 10: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Communicative Languages inSemi-Supervised Clustering

Cluster DocumentConfirmRemove

Page 11: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Enriching Languages in Flat Clustering

Cluster Document

Word Person

ConfirmRemove

Page 12: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Enriching Languages in Flat Clustering

Cluster Document

Word Person

ConfirmRemove

ConfirmRemove

ConfirmRemove

Page 13: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Enriching Languages in Hierarchical Clustering

Cluster Document

Word Person

ClusterConfirmRemove

ConfirmRemove

ConfirmRemove

Page 14: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Cluster Document

Move

Word Person

Cluster

Move

MoveMergeAddSplit

ConfirmRemove

ConfirmRemove

ConfirmRemove

Enriching Languages in Hierarchical Clustering

Page 15: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Experiment Design

• Can mixed-initiative clustering help a user achieve the result faster?

• Can mixed-initiative clustering help a machine build a better model?

Page 16: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Dataset

• An email dataset of one of the authors– 623 emails– 6684 unique words and 135 individual people– Manually sorted into a hierarchy of 15 cluster nodes

including a root, 3 intermediate nodes and 11 leaf nodes

Page 17: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Feedback Sessions

• Five initial hierarchical clustering results

• Two feedback sessions on each result– Diligent session– Lazy session

Page 18: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Diligent User

Cluster Document

Move

Word Person

Cluster

Move

MoveMergeAddSplit

ConfirmRemove

ConfirmRemove

ConfirmRemove

Page 19: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Lazy User

Cluster Document

Move

Word Person

Cluster

Move

MoveMergeAddSplit

ConfirmRemove

ConfirmRemove

ConfirmRemove

Page 20: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Lazy User vs. Diligent User

Page 21: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Measurement• User feedback is equivalent to edge modification.• Edge Modification Ratio (EMR) equals the ratio of

edges needed to be modified in order to reach the reference hierarchy.

1

3 42 5

6 7 8

9

e1 e4

10 11 12 13 14 15

16 17 18 19 20 21

22 23 24

25 26 27 28 29e9

e3

e2

e6e5

e10e11

e12

e13e14

e7

e8

e15

e16

e17

e18

e19e20 e21

e22e23

e24e25e26

e27 e28

Considering hierarchical accuracy with user feedback

e3e8

e18

e21

e12

50.18

28EMR

Page 22: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Good Results (4/5)

Page 23: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Bad Result (1/5)

Page 24: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

One More StepToward Mixed-Initiative

Clustering

Yifen HuangTom M. Mitchell

Carnegie Mellon University

Agents that Learn from Human TeachersMarch 23, 2009

Page 25: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Low-Latency Mixed-Initiative Clustering

C:\Documents and Settings\Yifen Huang\My Documents\Workspace\mixed-initiative-learning\demo.jar
Page 26: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Elaborated Framework for Mixed-Initiative Clustering

Page 27: Toward Mixed-Initiative Clustering Yifen Huang Tom M. Mitchell Carnegie Mellon University Agents that Learn from Human Teachers March 23, 2009

Future Work

• Feasibility study of the low-latency mixed-initiative interface


Crowdsourcing New Product Ideas under Consumer Learning · Crowdsourcing New Product Ideas under Consumer Learning Yan Huang Heinz College Carnegie Mellon University [email protected]

Clickjacking: Attacks and Defenses Lin-Shung Huang, Alexander Moshchuk, Helen J. Wang, Stuart Schechter, and Collin Jackson Carnegie Mellon University

Carnegie Mellon University TARTANS ABROAD...Above & Top Right: Eric Huang (Economics,’19) is studying abroad at the London School of Economic this summer term. Right (Both): Emily

Carnegie Mellon Efficient Inference for Distributions on Permutations Jonathan Huang CMU Carlos Guestrin CMU Leonidas Guibas Stanford

Carnegie Mellon Universitycschafer/cmspbs.pdf · 2009. 2. 26. · Carnegie Mellon University

Carnegie MellonCarnegie MellonCarnegie Mellon ...coursecatalog.web.cmu.edu/previous/0406catalog.pdf1 Carnegie Mellon University does not discriminate and Carnegie Mellon University

1 Carnegie Mellon Proxylab and stuff 15-213: Introduction to Computer Systems Recitation 13: November 19, 2012 Donald Huang (donaldh) Section M

Talking to Yourself for Fun and Profit Lin-Shung Huang ∗, Eric Y. Chen ∗, Adam Barth †, Eric Rescorla ‡ and Collin Jackson ∗ ∗ Carnegie Mellon University

Chu-Ren Huang Academia Sinica cwn.ling.sinica.tw/huang/huang.htm

Body Temperature Huang Qin Huang Qin （ Tel 2995285 ）

The BNY Mellon Funds · BNY Mellon Large Cap Stock Fund BNY Mellon Income Stock Fund BNY Mellon Mid Cap Stock Fund BNY Mellon Small Cap Stock Fund BNY Mellon U.S.Core Equity 130/30

Protecting Browsers from Cross-Origin CSS Attacks Lin-Shung Huang, Zack Weinberg Carnegie Mellon University Chris Evans Google Collin Jackson Carnegie

Web Security: Browsersinst.eecs.berkeley.edu/~cs161/sp14/slides/2.19.Browsers.pdf · From Clickjacking: Attacks and Defenses, by Lin-Shung Huang et al, Carnegie Mellon University

Combining Non-Expert and Expert Crowd Work to Convert …Convert Web APIs to Dialog Systems Ting-Hao K. Huang [email protected] Carnegie Mellon University Pittsburgh, PA USA Walter

RMA SECURITIES FINANCE & COLLATERAL MANAGEMENT … · BNY Mellon Larry Mannix BNY Mellon Rich Marquis BNY Mellon Andrea Pfenning BNY Mellon Charles Post BNY Mellon Kevin Reinert BNY

Action-Reaction: Forecasting the Dynamics of Human Interaction · Action-Reaction: Forecasting the Dynamics of Human Interaction De-An Huang and Kris M. Kitani Carnegie Mellon University,

Carnegie Mellon University Notice · Carnegie Mellon University Notice This video and all related information and materials (“materials”) are owned by Carnegie Mellon ... MELLON

Huang Huang - Michigan State University

Mellon Thesis

Huang Yuyuan Zhang Sihui Zhou Juan Zhu Wentao Huang Qianni

Mellon Center News - Illinois Wesleyan University · Mellon Center News Winter 2016 2016-18 Mellon Foundation Grant Engaging Tomorrow’s Students The Andrew W. Mellon Foundation

The Socio-monetary Incentives of Online Social Network Malware Campaigns Ting-Kai Huang (Google) Bruno Ribeiro (Carnegie Mellon University) Harsha M. Madhyastha

· PDF filePoltronieri Palmiro CNR-ISPA Agri-food Department Institute of Sciences of Food Productions Yifen Wang Ph. D. Engineering Auburn University Department of

SSR MELLON MAYS FELLOWS NEWSLETTER - …...rooklyn, NY 11201 Web: Portal: mellon-fellows.ssrc.org LinkedIn: ssrc-mellon mays Twitter: @ssrcmellonmays SSR -MELLON MAYS FELLOWS NEWSLETTER

AQUAINT 18-Month Workshop 1 Light Semantic Processing for QA Language Technologies Institute, Carnegie Mellon B. Van Durme, Y. Huang, A. Kupsc and E. Nyberg

Improvements in Diabetic Neuropathy and Nephropathy After ......Safwaan Adam1,2 & Shazli Azmi1,3 & Jan H. Ho1,3 & Yifen Liu1 & Maryam Ferdousi1,3 & Tarza Siahmansur1 & Alise Kalteniece

Web Security: Browsers - University of California, Berkeleycs161/sp14/slides/2.19.Browsers.pdf · From Clickjacking: Attacks and Defenses, by Lin-Shung Huang et al, Carnegie Mellon

AuburnReport - Auburn · PDF fileJohn Fulton, Puneet Srivastava and Yifen Wang, ... Chemistry and Biochemistry; Benson ... just made so much sense for us to tap

Carnegie Mellon Facts 1993 · Carnegie Mellon Facts 1993 Volume 7 Patrick J. Roach, Editor University Planning end Budget Carnegie Mellon University March, 1993. ... CARNEGIE MELLON

Wen-Ting Huang Jau-Chi Huang

Original Recipes, Myths and Music from the Netherlands Made by: Yifen Bensmann Jessie Collart Lisa Rozendal Anne-Sophie Otten Xhanthy den Haage

James Hays and Alexei A. Efros Carnegie Mellon University CVPR 2008 1 IM2GPS: estimating geographic information from a single image Wen-Tsai Huang

Carnegie Mellon