large-scale machine learning - new york university lecun large-scale machine learning large-scale...

5

Yann LeCun Large-Scale Machine Learning Large-Scale Machine Learning John Langford Yann LeCun Microsoft Research Courant Institute John Langford Yann LeCun Microsoft Research Courant Institute

Upload: vocong

Post on 23-Jun-2018

232 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Large-Scale Machine Learning - New York University LeCun Large-Scale Machine Learning Large-Scale Machine Learning John Langford Yann LeCun Microsoft Research Courant Institute John

Yann LeCun

Large-Scale Machine Learning

Large-Scale Machine Learning

John Langford Yann LeCun Microsoft Research Courant Institute

John Langford Yann LeCun Microsoft Research Courant Institute

Page 2: Large-Scale Machine Learning - New York University LeCun Large-Scale Machine Learning Large-Scale Machine Learning John Langford Yann LeCun Microsoft Research Courant Institute John

Yann LeCun

What is Data Science?What is Data Science?

Data Science: automatically extracting knowledge from dataMathematics & StatisticsMachine LearningDomain Expertise

Applications in BusinessLots and lots

Applications in the SciencesAstronomy, CosmologyHigh-energy PhysicsBiology, GenomicsNeuroscienceThe Social Sciences

Medicine

Government

[after Drew Conway's Data Science Venn Diagram]

Mathematics &

StatisticsComputation

Domain Expertise

conventional

research

Danger

Zone!

Machine

Learning

Data

Science

Page 3: Large-Scale Machine Learning - New York University LeCun Large-Scale Machine Learning Large-Scale Machine Learning John Langford Yann LeCun Microsoft Research Courant Institute John

Yann LeCun

Large Scale Machine LearningLarge Scale Machine Learning

Class website:http://cilvr.cs.nyu.edu/doku.php?id=courses:bigdata:starthttp://cilvr.cs.nyu.edu courses big data→ →

Forum, discussion, Q&A on Piazzahttps://piazza.com/class#spring2013/csciga3033002

Evaluation:Programming assignmentsProjectFinal exam

Computing infrastructure100-node cluster, 8 CPUs/node, Hadoop (donated by Yahoo! Labs)

SoftwareTorch: http://www.torch.ch/Vowpal Wabbit: https://github.com/JohnLangford/vowpal_wabbit/wiki

http://cilvr.cs.nyu.edu/doku.php?id=courses:bigdata:start

http://cilvr.cs.nyu.edu/

https://piazza.com/class#spring2013/csciga3033002

Page 4: Large-Scale Machine Learning - New York University LeCun Large-Scale Machine Learning Large-Scale Machine Learning John Langford Yann LeCun Microsoft Research Courant Institute John

Yann LeCun

Big Data?Big Data?

Data often comes to in the form of a tableN: dimension of each vector (possibly very sparse)T: number of training samples (possibly infinite)

Big Data is large T, or large N, or bothLarge T, small N: great!Infinite T, small N: on-line / streamingSmall T, large N: hell!

Problems:(distributed) data storage and accesscan't use algo super-linear in TLarge N: overfittingParallelizingDealing with unbalanced setRepresenting high-dim data

N

T

Page 5: Large-Scale Machine Learning - New York University LeCun Large-Scale Machine Learning Large-Scale Machine Learning John Langford Yann LeCun Microsoft Research Courant Institute John

Yann LeCun

SyllabusSyllabusIntro

Online Linear learning

2nd order optimization methods

LBFGS

Online Non-linear learning

Boosted Decision Trees

Hadoop, Allreduce

Parallel learning, OpenMP, CUDA

Inverted Indicies & Predictive Indexing

Hashing, LSH, linear/non-linear dimensionality reduction

Feature Learning, deep learning

Many Classes

Active Learning

Exploration and Learning

A Tutorial on Energy-Based Learning - Yann LeCunyann.lecun.com/exdb/publis/orig/lecun-06.pdfA Tutorial on Energy-Based Learning Yann LeCun, Sumit Chopra, Raia Hadsell, Marc’Aurelio

Optimization Methods for Large-Scale Machine Learningusers.iems.northwestern.edu/~nocedal/PDFfiles/SIAM... · OPTIMIZATION METHODS FOR LARGE-SCALE MACHINE LEARNING 225 Machine learning

Lecun 20060816-ciar-01-energy based learning

Machine learning at Scale

Large-scale Matrix Factorizationkijungs/etc/10-405.pdf · Large-scale Matrix Factorization ... Large-scale Matrix Factorization (by Kijung Shin) 16/99 Machine 1 Machine 2 Machine

Introduction to The Time Machine Large-Scale Research ... · Introduction to The Time Machine Large-Scale Research Initiative The Time Machine is an international collaborative project

Y LeCun MA Ranzato - CILVR at NYU – Computational Intelligence, Learning… · 2016-01-27Y LeCun MA Ranzato Deep Learning ... Image recognition ... AI, Neuroscience, Cognitive

MACHINE VISE EXPLODED VIEW MACHINE VISE …ddgt.net/StudentPages/jviscia/pdf/machine_vise.pdf · machine vise scale 1 / 2 machine vise exploded view scale 1 / 4 digital design graphics

Spark + H20 = Machine Learning at scale

NIPS'2015 Tutorial Geoff Hinton, Yoshua Bengio & Yann LeCun

Transformation Invariance in Pattern Recognition - Yann LeCun

OSCON: Apache Mahout - Mammoth Scale Machine Learning

Small scale and large scale capsule filling machine

Fundamentals of Machine Learningaykut/classes/... · • New neural architectures - Convolutional neural nets (LeCun et al., 1989) - Long-short term memory networks (LSTM) (Schmidhuber,

Scaling Learning Algorithms Towards AI...Yoshua Bengio and Yann LeCun, Scaling Learning Algorithms towards AI, Large-Scale Kernel Machine, 2007. Leslie Lamport, Deep Learning and Convolutional

Machine Intelligence at Google Scale

"Convolutional Networks: Unleashing the Potential of Machine Learning for Robust Perception Systems," a Presentation from Yann LeCun of Facebook and NYU

Toward large-scale vulnerability discovery using Machine ... · Toward large-scale vulnerability discovery using Machine ... by presenting a scalable machine learning approach that

Y LeCun MA Ranzato 2014-04-13 · Y LeCun MA Ranzato This Basic Model has not evolved much since the 50's The first learning machine: the Perceptron Built at Cornell in 1960 The Perceptron

Large-Scale Machine Learning at Twitter

35 virtual machine scale and mobility

Large Scale Machine learning with Spark

Large Scale Causal Inference with Machine Learningweb.stanford.edu/~thaipham/papers/Thai_Pham_PhD_Thesis_Defense.pdf · Large Scale Causal Inference with Machine Learning Ph.D. Thesis

Large Scale Machine Translation Architectures Qin Gao

Four Moves to Machine Learning at Scale

Large-scale Machine Learning using DryadLINQ

Machine auscultation: enabling machine diagnostics using ... · Machine auscultation: enabling machine diagnostics using convolutional neural networks and large-scale machine audio

Machine Learning at Netflix Scale

1 Machine Learning-Based Classification of Patterns of EEG Synchronization for Seizure Prediction Piotr Mirowski, Deepak Madhavan MD, Yann LeCun PhD, Ruben

Large-Scale Machine Learning over Graphs

Bertrand LeCun Operations Research bertrandlc@google · If M1

Making Machine Learning Scale: Single Machine and Distributed

Large-scale machine learning and convex optimizationfbach/gradsto_bonn_2016.pdf · 2016-01-12 · Machine learning for “big data” • Large-scale machine learning: large d, large

Large-Scale Graph-Structured Machine Learning -

Y LeCun MA Ranzato - Computational Intelligence, Learning ... Sparse Coding Pooling ... Y LeCun MA Ranzato Deep Learning and Feature Learning Today ... Y LeCun MA Ranzato SHALLOW DEEP