robust optimization and applications in machine learning

Robust Optimization andApplications in Machine Learning

Time-series prediction via linear least-squares

Predicted output

Properties of solution

Non-linear prediction and kernels

Properties of solution

What is a kernel, anyway?

SVM, LR, LS, MPM, PCA, CCA, FDA…

Example: 2nd-order polynomial kernel

Example: 2nd-order polynomial kernel

A classical way to use kernels

Transduction framework

Important property of kernel matrices

Kernel optimization in least-squares

Kernel optimization for least-squares

Kernel optimization via SDP or SOCP

A non-classical way to use kernels

Kernel optimization in other problems

Kernel optimization in SVM classifiers

Kernel optimization in SVM classifiers (cont’d)

Link with robust optimization

Kernel optimization and data fusionmRNA

expression data

upstream region data

(TF binding sites)

protein-protein interaction data

hydrophobicity data

sequence data

(gene, protein)

Challenge

Example of a Kernel for Genomic Data: Pairwise Comparison

Kernel

1 0 0 1 0 1 0 11 0 1 0 1 1 0 10 0 0 0 1 1 0 00 0 1 0 1 1 0 10 0 1 0 1 0 0 11 0 0 0 0 0 0 10 0 1 0 1 0 0 0

protein

protein

2

Example of a Kernel for Genomic Data: Linear Interaction Kernel

Exampe of a Kernel for Genomic Data: Diffusion Kernel

Learning the Optimal Kernel

K

Learning the Optimal KernelIntegrate constructed Integrate constructed kernelskernels

Learn a linear mix

Large margin classifier Large margin classifier (SVM)(SVM)

Maximize the margin

Yeast Protein Function Prediction

Yeast Protein Function Prediction

MRF

SDP/SVM(binary)

SDP/SVM(enriched)

Yeast Protein Function Prediction

MRF

SDP/SVM(binary)

SDP/SVM(enriched)

Yeast Protein Function Prediction

Part 3: summary