Upload File

the clustering problem

34
The Clustering Problem Yongsub Lim Applied Algorithm Laboratory KAIST

Upload: gannon-winters

Post on 31-Dec-2015

37 views

Category:

Documents


0 download

Report

Tags:

DESCRIPTION

The Clustering Problem. Yongsub Lim Applied Algorithm Laboratory KAIST. Contents. The Clustering Problem Basic Algorithms K-Means K-Clustering of Max. Spacing Two-Phase Algorithms Other Algorithms. The Clustering Problem. Given data, it is to discover “meaningful” groups - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: The Clustering Problem

The Clustering Problem

Yongsub LimApplied Algorithm Laboratory

KAIST

Page 2: The Clustering Problem

04/19/2023 The Clustering Problem 2

Contents

• The Clustering Problem• Basic Algorithms

K-Means K-Clustering of Max. Spacing

• Two-Phase Algorithms• Other Algorithms

Page 3: The Clustering Problem

04/19/2023 The Clustering Problem 3

The Clustering Problem

• Given data, it is to discover “mean-ingful” groups

• Data in same group are similar, and• Data between different groups are

not similar

Page 4: The Clustering Problem

04/19/2023 The Clustering Problem 4

Example of clustering

1x

2x

Page 5: The Clustering Problem

04/19/2023 The Clustering Problem 5

Example of clustering

1x

2x

Page 6: The Clustering Problem

04/19/2023 The Clustering Problem 6

Example of clustering

1x

2x

Page 7: The Clustering Problem

04/19/2023 The Clustering Problem 7

Applications of Clustering

• The image segmentation problem can be considered as a clustering of pixels of an image

• In unsupervised learning, before making a decision rule, we classify unlabeled training data through clus-tering

Page 8: The Clustering Problem

04/19/2023 The Clustering Problem 8

Applications of Clustering

• In a network or a graph, we can do grouping vertices which are highly connected within one group

• Clustering is also useful in biology to classify genes

Page 9: The Clustering Problem

04/19/2023 The Clustering Problem 9

Basic Algorithms

• Two algorithms will be introduced

• K-Means computes iteratively centers of K clusters

• K-Clustering of Max. Spacing uses a minimum spanning tree

• Objective functions of theses are dif-ferent

Page 10: The Clustering Problem

04/19/2023 The Clustering Problem 10

K-Means

• Determine means of K clusters ran-domly

• At each iteration, Every data belongs to a cluster whose

mean is the nearest one among K means

Re-compute means of all clusters

Page 11: The Clustering Problem

04/19/2023 The Clustering Problem 11

K-Means

• Objective is to minimize the sum of distance of centers of clusters and their members

• It is clustering for high density in one cluster

Page 12: The Clustering Problem

04/19/2023 The Clustering Problem 12

K-Means Algorithm

• Worst caseInitial two cen-ters randomly chosen

This may be not what we want!!!

Page 13: The Clustering Problem

04/19/2023 The Clustering Problem 13

K-Clustering of Max. Spacing

• Given data, find K clusters which maximize the minimum distances be-tween all pairs of clusters

• spacing: min. distance between any pair of data in different clusters

Page 14: The Clustering Problem

04/19/2023 The Clustering Problem 14

K-Clustering of Max. Spacing

Page 15: The Clustering Problem

04/19/2023 The Clustering Problem 15

K-Clustering of Max. Spacing

• Consider given data to a complete graph with Euclidean distance

• Compute a MST

• Delete the K-1 most expensive edges of a MST

Page 16: The Clustering Problem

04/19/2023 The Clustering Problem 16

K-Clustering of Max. Spacing

Calg

Copt

≤ spacing of Calg

Page 17: The Clustering Problem

04/19/2023 The Clustering Problem 17

K-Clustering of Max. Spacing

• It is no randomness

• Objective seems to be better or more reasonable than K-means

Page 18: The Clustering Problem

04/19/2023 The Clustering Problem 18

K-Means vs. Max. Spacing

• Good clustering is High density in one cluster (K-Means) Long dist. between clusters (Max. Spac-

ing)

>

Page 19: The Clustering Problem

04/19/2023 The Clustering Problem 19

K-Means vs. Max. Spacing

Page 20: The Clustering Problem

04/19/2023 The Clustering Problem 20

Two-Phase Algorithms

• Two algorithms will be introduced

• In the first phase, both do clustering without restriction on K

• In second phase, if # of clusters are larger than K, merge using Max. Spacing

Page 21: The Clustering Problem

04/19/2023 The Clustering Problem 21

Hierarchical EMST

Oleksandr Grygorash, Yan zhou, Zach Jorgensen, Minimum spanning Tree Based Clustering Algorithms

Page 22: The Clustering Problem

04/19/2023 The Clustering Problem 22

Hierarchical EMST

• HEMST removes all edges with weights greater than the threshold (mean+std. of edges)

• If # of clusters is less than a given K, same with Max. Spacing

• If not, it runs Max. Spacing on data set each of which is nearest to the center of its cluster

Oleksandr Grygorash, Yan zhou, Zach Jorgensen, Minimum spanning Tree Based Clustering Algorithms

Page 23: The Clustering Problem

04/19/2023 The Clustering Problem 23

Hierarchical EMST

Oleksandr Grygorash, Yan zhou, Zach Jorgensen, Minimum spanning Tree Based Clustering Algorithms

Page 24: The Clustering Problem

04/19/2023 The Clustering Problem 24

Hierarchical EMST

Oleksandr Grygorash, Yan zhou, Zach Jorgensen, Minimum spanning Tree Based Clustering Algorithms

Page 25: The Clustering Problem

04/19/2023 The Clustering Problem 25

Hierarchical EMST

Oleksandr Grygorash, Yan zhou, Zach Jorgensen, Minimum spanning Tree Based Clustering Algorithms

Page 26: The Clustering Problem

04/19/2023 The Clustering Problem 26

Hierarchical EMST

Oleksandr Grygorash, Yan zhou, Zach Jorgensen, Minimum spanning Tree Based Clustering Algorithms

Page 27: The Clustering Problem

04/19/2023 The Clustering Problem 27

Modified K-Means Process

• MKF, in the first phase, is similar to K-Means

• The difference is that if data is far enough from all clusters, it becomes the center of the new cluster

• While running, if # of clusters is larger than a threshold, the two nearest clus-ters are merged

• In the second phase, apply Max. Spaing

M.F. Jiang, S.S. Tseng, C.M. Su, Two-phase clustering process for outliers detection

Page 28: The Clustering Problem

04/19/2023 The Clustering Problem 28

Modified K-Means Process

• This scheme can identify outliers by using Max. Spacing

M.F. Jiang, S.S. Tseng, C.M. Su, Two-phase clustering process for outliers detection

Page 29: The Clustering Problem

04/19/2023 The Clustering Problem 29

Modified K-Means Process

M.F. Jiang, S.S. Tseng, C.M. Su, Two-phase clustering process for outliers detection

Page 30: The Clustering Problem

04/19/2023 The Clustering Problem 30

Modified K-Means Process

M.F. Jiang, S.S. Tseng, C.M. Su, Two-phase clustering process for outliers detection

Page 31: The Clustering Problem

04/19/2023 The Clustering Problem 31

Two-Phase Algorithms

• Both give more weights to members in small sets in the first phase

• A small set will be the most likely clustered data, so it is reasonable to decrease distances between them

Page 32: The Clustering Problem

04/19/2023 The Clustering Problem 32

Other Algorithms

• HCS uses min-cut of a graph

• It recursively separate data to dis-joint two subsets (min-cut) until all clusters are highly connected

• A graph is highly connected if the min. # of edges whose removal disconnects the graph is greater than |V|/2

Erez Hartuv, Ron Shamir, a clustering algorithm based on graph connectivity

Page 33: The Clustering Problem

04/19/2023 The Clustering Problem 33

Other Algorithms

• Voting

• Apply K-Means N times

• If any pair of data belonged to same cluster greater than threshold t times, they are grouped

Ana L.N. Fred, Anil K. Jain, Data Clustering Using Evidence ac-cumulation

Page 34: The Clustering Problem

04/19/2023 The Clustering Problem 34

Thanks


1 Microarray Clustering. 2 Outline Microarrays Hierarchical Clustering K-Means Clustering Corrupted Cliques Problem CAST Clustering Algorithm

Deep Subspace Clustering Networkspapers.nips.cc/paper/6608-deep-subspace-clustering-networks.pdf · Subspace clustering has become an important problem as it has found various applications

Consistent Algorithms for Clustering Time Serieskhaleghi/Publications_files/khaleghi16a.pdf · Consistent Algorithms for Clustering Time Series 1.1 Problem Setup We consider two variants

An Efficient Differential Memetic Algorithm for Clustering ... · An Efficient Differential Memetic Algorithm for Clustering Problem Lei Jiang, Datong Xie Abstract—Clustering

A SURVEY OF TEXT CLUSTERING ALGORITHMScharuaggarwal.net/text-cluster.pdf · 78 MININGTEXTDATA jects is measured with the use of a similarity function. The problem of clustering can

Nirit Gourgy. What is clustering? Metric Spaces K-center Clustering problem & 2- approximation algorithm K-median Clustering problem & approximation

Scalable Graph Clustering using Stochastic Flows · Venu Satuluri and Srinivasan Parthasarathy Scalable Graph Clustering using Stochastic Flows Problem Statement Graph Clustering:

Lecture 12: Unsupervised learning. Clustering€¦ · Lecture 12: Unsupervised learning. Clustering Introduction to unsupervised learning Clustering problem K-means clustering Hierarchical

Clustering and Probability (Chap 7). Review from Last Lecture Defined the K-means problem for formalizing the notion of clustering. Discussed the K-means

Clustering with Bregman Divergencesjmlr.csail.mit.edu/papers/volume6/banerjee05b/banerjee05...3. Based on our analysis of the Bregman clustering problem, we present a meta hard clustering

D-IMPACT: A Data Preprocessing Algorithm to Improve the ... · Attraction, Clustering, Data Preprocessing, Density, Shrinking 1. Introduction 1.1. Clustering Problem and Data Preprocessing

The Clustering Problem

Methods for Clustering Data with Missing ValuesTo handle the problem of clustering data with missing values, we look carefully at the method of multiple imputation combined with clustering

Distributed Clustering via LSH Based Data Partitioningproceedings.mlr.press/v80/bhaskara18a/bhaskara18a.pdf · MapReduce framework). 1. Introduction Clustering is a fundamental problem

Toeplitz Inverse Covariance-Based Clustering of ... · Toeplitz Inverse Covariance-Based Clustering of ... We solve the TICC problem through alternating ... constrained inverse covariance

Clustering Motion - Princeton University Computer Science · Clustering Motion ∗ Sariel Har-Peled ... Clustering is a central problem in computer science. It is related to unsupervised

Scalable Clustering Methods for the Name Disambiguation Problemnike.psu.edu/publications/j-kais12.pdf · 2011-04-07 · Scalable Clustering Methods for the Name Disambiguation Problem

Rearrangement Clustering: Pitfalls, Remedies, and Applicationsjmlr.csail.mit.edu/papers/volume7/climer06a/climer06a.pdf · of the objects, rearrangement clustering is the problem

Robust Subspace Clustering with Compressed Dataanalysis (e.g., clustering) of compressed data. We therefore study in this paper a novel problem called compressive robust subspace clustering,

Software Module Clustering as a Multi–Objective Search Problem

Behavior pattern clustering in blockchain networks · The problem of behavior pattern clustering in blockchain networks can be formalized as below. Definition 1 Given the sequences

Clustering Algorithms Mu-Yu Lu. What is Clustering? Clustering can be considered the most important unsupervised learning problem; so, as every other

Data Mining K-Clustering Problem - CORE · Elham Karoussi Data Mining, K-Clustering Problem 11 core's of data mining and the measure of similarity and dissimilarity of data. It provides

Clustering and optimization in genetic data: the problem of Tag-SNPs selection

The CLICK clustering algorithmrshamir/abdbm/pres/17/Click.pdf · Performance of CLICK vs other clustering algorithms Elements Time Problem Original Algorithm CLICK Improvement (min)*

Task Clustering Problem with Large Communication Delays

Clustering. The Problem of Clustering Given a set of points, with a notion of distance between points, group the points into some number of clusters,

Optimal Clustering of Products for Regression-Type and ...matthewalanham.com/Students/2018_MWDSI_Optimal Clustering of...Some retailers frame the prediction problem as a classification

Text Clustering Hongning Wang CS@UVa. Today’s lecture Clustering of text documents – Problem overview Applications – Distance metrics – Two basic categories

Log Clustering based Problem Identification for Online ... · Log Clustering based Problem Identification for ... experiments on two Hadoop-based applications and two ... there is

Clustering 10716, Spring 2020 Pradeep Ravikumar (amending ...pradeepr/716/readings/lec13.pdf · Pradeep Ravikumar (amending notes from Larry Wasserman) 1 The Clustering Problem In

Data Clustering: A Reviewnicholas/clustering/p264-jain.pdf · broad appeal and usefulness as one of the steps in exploratory data analysis. However, clustering is a difficult problem

Large Scale Clustering - sites.cs.ucsb.eduveronika/MAE_LargeScaleClustering_strnadova.pdf · Clustering in Genetic Mapping ^The problem of genetic mapping can essentially be divided

The clustering problem: Hierarchical clustering algorithmelbo.gs.washington.edu/courses/GS_373_17_sp/slides/14-Networks... · Hierarchical clustering algorithm: 1. Assign each object

An Enhanced Agglomerative Clustering Algorithm for Solving Vehicle Routing Problem