Upload File

photo clustering of social events by extending phototoc to a rich context

  • Home
  • Technology
  • Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context
36
Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context Daniel Manchón-Vizuete Irene Gris-Sarabia Xavier Giró-i-Nieto ICMR 2014 Workshop on Social Events in Web Multimedia, Glasgow (Scotland/UK), 01/04/2014

Upload: xavier-giro

Post on 24-May-2015

235 views

Category:

Technology


0 download

Report

Tags:

DESCRIPTION

Presented at ICMR 2014 Workshop on Social Events in Web Multimedia (SEWM). http://mklab2.iti.gr/sewm14/ The popularisation of the storage of photos on the cloud has opened new opportunities and challenges for the organisa- tion and extension of photo collections. This paper presents a light computational solution for the clustering of web pho- tos based on social events. The proposal combines a first over-segmentation of the photo collections of each user based on temporal cues, as previously proposed in PhotoTOC. On a second stage, the resulting mini-clusters are merged based on contextual metadata such as geolocation, keywords and user IDs. Results indicate that, although temporal cues are very relevant for event clustering, robust solutions should also consider all these additional features.

TRANSCRIPT

Page 1: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Daniel Manchón-Vizuete Irene Gris-Sarabia Xavier Giró-i-Nieto

ICMR 2014 Workshop on Social Events in Web Multimedia, Glasgow (Scotland/UK), 01/04/2014

Page 2: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

OutlineMotivation Related workApproachResultsFuture workConclusions

Photo Clustering of Social

Events by Extending

PhotoTOC to a Rich Context

D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto

Page 3: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Motivation

Page 4: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Motivation

Page 5: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Motivation

Credit: National Institute on Aging and FDA.

Page 6: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Motivation

Page 7: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Motivation

Credit: Alan Smeaton and Cathal Gurrin (Dublin City University, 2013)

Page 8: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

OutlineMotivation Related workApproachResultsFuture workConclusions

Photo Clustering of Social

Events by Extending

PhotoTOC to a Rich Context

D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto

Page 9: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Related work

PhotoTOC[Platt et al, PACRIM 2003]

Page 10: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Related work

[Becker, Naaman & Gravano, ACM WSDM 2010]

Rich context

User-provided annotations

Automatically generated information

TitleDescription

Creation timeGeo-location

Page 11: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

OutlineMotivation Related workApproachResultsFuture workConclusions

Photo Clustering of Social

Events by Extending

PhotoTOC to a Rich Context

D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto

Page 12: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(a) Temporal sorting by each user independently

Hi, I’m John. Hi, I’m Emily.

Page 13: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(b) Temporal-based oversegmentation in mini-clusters

PhotoTOC[Platt et al, PacRim 2003]

Page 14: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(b) Temporal-based oversegmentation in mini-clusters

Page 15: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

? tavg(·) avg(·) avg(·)avg(·)

Page 16: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

Weightedmodalities

● creation (or upload) time● geolocation● textual labels● same user

Page 17: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

Geolocation (d=haversine)Time stamp (d=L1)

Text labels (d=Jaccard) Same user (d=boolean)

Page 18: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

Page 19: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

Mean and std. deviation learned on

pairs of photos within the same training event.

Page 20: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

phi function

Page 21: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

decision threhold

Page 22: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Approach(c) Sequential merging of mini-clusters

Page 23: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

OutlineMotivation Related workApproachExperimentsFuture workConclusions

Photo Clustering of Social

Events by Extending

PhotoTOC to a Rich Context

D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto

Page 24: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

DatasetReSEED - Social Event Detection Dataset

Timo Reuter, Symeon Papadopoulos, Vasilios Mezaris & Philipp Cimiano (ACM MM Sys 2014)[Paper: http://dx.doi.org/10.1145/2557642.2563674] [Dataset: http://greententacle.techfak.uni-bielefeld.de/reseed/]

http://dx.doi.org/10.1145/2557642.2563674
http://greententacle.techfak.uni-bielefeld.de/reseed/
Page 25: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Results

Page 26: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Results

Page 27: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Results

Diversity of cameras (qualities)

Page 28: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Diversity of locations

Results

Page 29: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Results

Visual redundancy (not exploited)

Page 30: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Results

False split

Page 31: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Results

False merge

Page 32: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

OutlineMotivation Related workApproachExperimentsFuture workConclusions

Photo Clustering of Social

Events by Extending

PhotoTOC to a Rich Context

D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto

Page 33: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Future work

● Median-based normalisation for robustness against outliers.

● Better optimisation of parameters K and d.

● Improve fusion approach (SVM ?).

● Event-dependent merging criteria.

● Efficient use of visual features (higher computation).

Page 34: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

OutlineMotivation Related workApproachExperimentsFuture workConclusions

Photo Clustering of Social

Events by Extending

PhotoTOC to a Rich Context

D. Manchón-Vizuete I. Gris-Sarabia X. Giró-i-Nieto

Page 35: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Conclusions● All considered context (geo, user, tags) can help.

● Watch out for outliers.

● Divide and conquer… sequentially.

Thank you

SEWM !

Page 36: Photo Clustering of Social Events by Extending PhotoTOC to a Rich Context

Thank you !

[email protected]

http://bitsearch.blogspot.com

https://imatge.upc.edu/web/people/xavier-giro

@DocXavi

/ProfessorXavi

http://bitsearch.blogspot.com
http://bitsearch.blogspot.com
https://imatge.upc.edu/web/people/xavier-giro
https://imatge.upc.edu/web/people/xavier-giro

PhotoTOC: Automatic Clustering for Browsing Personal Photographs

Data Mining Techniques: Clustering. 2 Today Clustering Distance Measures Graph-based Techniques K-Means Clustering Tools and Software for Clustering

BIRCH: Balanced Iterative Reducing and Clustering Using Hierarchies A hierarchical clustering method. It introduces two concepts : Clustering feature Clustering

Extending Extending

FUZZY CLUSTERING 2009/2010. 2 What is Data Clustering? Fuzzy C-Means Clustering Subtractive Clustering Data Clustering Using the Clustering GUI

Introduction to Web Clustering - Università degli Studi ... · Introduction to Web Clustering Some Web Clustering engines ... for Web Clustering Web data ... for Web Clustering Classic

Affinity Clustering: Hierarchical Clustering at Scalepapers.nips.cc/paper/7262-affinity-clustering-hierarchical-clustering-at-scale.pdf · Affinity Clustering: Hierarchical Clustering

Collaborative Clustering for Entity Clustering

Clustering IV. Outline Impossibility theorem for clustering Density-based clustering and subspace clustering Bi-clustering or co-clustering

1 Microarray Clustering. 2 Outline Microarrays Hierarchical Clustering K-Means Clustering Corrupted Cliques Problem CAST Clustering Algorithm

Extending K-Means Clustering for Analysis of Quantitative ... · 1. Introduction A Quantitative Structure-Activity Relationship (QSAR) is a study of the dependence upon chemical structure

Canopy Clustering and K-Means Clustering

Ward’s Hierarchical Clustering Method: Clustering ... · Ward’s Hierarchical Clustering Method: Clustering Criterion and Agglomerative Algorithm Fionn Murtagh (1) and Pierre Legendre

Tutorial 8 Clustering 1. General Methods –Unsupervised Clustering Hierarchical clustering K-means clustering Expression data –GEO –UCSC –ArrayExpress

Clustering Supervised vs. Unsupervised Learning Examples of clustering in Web IR Characteristics of clustering Clustering algorithms Cluster Labeling 1

PhotoTOC: Automatic Clustering for Browsing Personal … · 2018-01-04 · PhotoTOC: Automatic Clustering for Browsing Personal Photographs John C. Platt, Mary Czerwinski, Brent A

CATA 2010 March 2010 Jewels, Himalayas and Fireworks, Extending Methods for Visualizing N Dimensional Clustering W. Jockheck Dept. of Computer Science

Clustering k-mean clustering

Wasserstein Clustering...Bob Dylan in Slow Train Coming, 1979 Abstract Clustering is partitionning the data into groups. Deep approaches for clustering are promising for extending

Guest Lecture: Clusteringcvml.ist.ac.at/talks/clustering-core2018.pdfsingle linkage clustering, complete linkage clustering, average linkage clustering Graph-based clustering spectral

Clustering: Partition Clustering

iGraph: A Graph-BasedTechnique for Visual Analytics of ...cwang11/research/vda15-igraph.pdf · with quantum treemaps and bubblemaps for zoomable image browsing,3 PhotoTOC for automatic

DATA MINING - CLUSTERING. Clustering 4 Clustering - unsupervised classification 4 Clustering - the process of grouping physical or abstract objects into

Introduction to Web Clustering - uniroma2.it · Introduction to Web Clustering ... June 26, 2009. Outline I Introduction to Web Clustering I Some Web Clustering engines ... for clustering

Lecture outline Clustering aggregation – Reference: A. Gionis, H. Mannila, P. Tsaparas: Clustering aggregation, ICDE 2004 Co-clustering (or bi-clustering)

Bi-Clustering Jinze Liu. Outline The Curse of Dimensionality Co-Clustering Partition-based hard clustering Subspace-Clustering Pattern-based 2

Ward’s Hierarchical Clustering Method: Clustering

Clustering. 2 Outline Introduction K-means clustering Hierarchical clustering: COBWEB

Photo Clustering of Social Events by Extending PhotoTOC to ...€¦ · Barcelona, Catalonia [email protected] ABSTRACT ... ever the algorithm generated a cluster with more than

CSE601 Clustering Ensemble - University at Buffalojing/cse601/fa12/materials/clustering_ensem… · – Clustering ensemble – Clustering in MapReduce – Semi-supervised clustering,

Clustering 2: Hierarchical clustering

Chapter19 Clustering Analysis. Content Similarity coefficient Hierarchical clustering analysis Dynamic clustering analysis Ordered sample clustering analysis

The effectiveness of social bubbles as part of a Covid-19 ...€¦ · 05.06.2020  · Background: During the Covid-19 lockdown, contact clustering in social bubbles may allow extending

Bi-Clustering. 2 Data Mining: Clustering Where K-means clustering minimizes

Clustering in Ratemaking: Applications in Territories ... · Clustering in Ratemaking: Applications in Territories Clustering OVERVIEW OF CLUSTERING ¾Purpose of Clustering in Insurance