![Page 1: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/1.jpg)
SupervisedLearningAlgorithms
Analysisof
Differentapproaches
EvgeniyMarinovMLConsultant
PhilipYankovx8academy
![Page 2: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/2.jpg)
MLDefiniCon
• ThereareplentyofdefiniCons...• Informal:Thefieldofstudythatgivescomputerstheabilitytolearnwithoutbeingexplicitlyprogrammed(ArthurSamuel,1959)
• Formal:AcomputerprogramissaidtolearnfromexperienceE,withrespecttosometaskT,andsomeperformancemeasureP,ifitsperformanceonTasmeasuredbyPimproveswithexperienceE(TomMitchell,1998).
![Page 3: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/3.jpg)
FromWikipedia
• Machinelearningis:– asubfieldofcomputersciencethatevolvedfromthestudyofpaRernrecogniConandinAIinthe1980s(MLisaseparatefieldflourishingfromthe1990s,firstbenefitedfromstaCsCcsandthenfromtheincreasingavailabilityofdigiCzedinformaConatthatCme).
![Page 4: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/4.jpg)
WhyML?
![Page 5: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/5.jpg)
WhyML?
![Page 6: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/6.jpg)
KeyfactorsenablingMLgrowthtoday
• CloudCompu)ng• InternetofThings• BigData(+UnstructuredData)
![Page 7: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/7.jpg)
WhyDataissoimportant?
![Page 8: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/8.jpg)
WhyDataissoimportant?
• GooglePhotos– Unlimitedstorage
• Googlevoice– OK,Google
![Page 9: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/9.jpg)
![Page 10: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/10.jpg)
Nowadays
• ItissoeasytogetdatayouneedandtouseanAPIorserviceofsomecompanytoexperimentwiththem
![Page 11: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/11.jpg)
MethodsforcollecCngdata
![Page 12: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/12.jpg)
MethodsforcollecCngdata
• Download– Spreadsheet– Text
• API• Crawling/scraping
![Page 13: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/13.jpg)
SupervisedLearning
![Page 14: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/14.jpg)
Task Description
![Page 15: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/15.jpg)
Pipeline
![Page 16: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/16.jpg)
IniCalexample
![Page 17: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/17.jpg)
NotaCon
![Page 18: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/18.jpg)
![Page 19: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/19.jpg)
• Asdasd
• Asdasd
• Asdasd
• Asdasd
TheregressionfuncConf(x)
![Page 20: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/20.jpg)
• as• as
• as
![Page 21: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/21.jpg)
Howtoevaluateourmodel?
![Page 22: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/22.jpg)
Pipeline
![Page 23: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/23.jpg)
Assessing the Model Accuracy
![Page 24: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/24.jpg)
![Page 25: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/25.jpg)
Bias-variancetrade-off
![Page 26: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/26.jpg)
Bias-variancetrade-off
![Page 27: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/27.jpg)
Cross-validaCon
![Page 28: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/28.jpg)
GeneralizaConErrorandOverfi`ng
![Page 29: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/29.jpg)
ChoosingaModelbydatatypesofresponse
![Page 30: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/30.jpg)
Pipeline
![Page 31: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/31.jpg)
DatatypesandGeneralizedLinearmodel
• SimpleandGenerallinearmodels• RestricConsofthelinearmodel• DatatypeoftheresponseY1) (General)LinearmodelR,Y~Gaussian(µ,σ^2)--conCnuous2) LogisCcregression{0,1},Y~Bernoulli(p)--binarydata3)Poissonregression{0,1,...},Y~Poisson(µ)--counCngdata
![Page 32: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/32.jpg)
SimpleandGenerallinearmodels
Simple:General:
![Page 33: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/33.jpg)
ErroroftheGeneralLinearmodel
ClicktoaddText
![Page 34: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/34.jpg)
RestricConsofLinearmodels
AlthoughtheGenerallinearmodelisausefulframework,itisnotappropriateinthefollowingcases:• TherangeofYisrestricted(e.g.binary,count,posiCve/negaCve)
• Var[Y]dependsonthemeanE[Y](fortheGaussiantheyareindependent)
Name Mean Variance
Bernoulli(p) p p(1 - p)
Binomial(p, n) np np(1 - p)
Poisson(p) p p
![Page 35: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/35.jpg)
BinaryresponseY–{0,1}• TheBernoulli(p)isdiscreter.v.withtwopossibleoutcomes:• pandq=1–p• TheparameterpdoesnotchangeoverCme• Bernoulliisbuildingblockforothermorecomplicated
distribuCons
• Examples:• Coinflips{Heads,Tails}–ifunbiased• thenp=0.5• ClickonAd,Fail/SuccessonExam
![Page 36: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/36.jpg)
GeneralizedLinearmodel-IntuiCon
![Page 37: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/37.jpg)
ExponenCalFamily
![Page 38: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/38.jpg)
Generallinearmodel
![Page 39: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/39.jpg)
Binary Data
![Page 40: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/40.jpg)
ModelingCounCng/PoissonData
![Page 41: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/41.jpg)
MaximizingtheLog-LikelihoodandParametersesCmaCon
![Page 42: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/42.jpg)
Preprocessing
![Page 43: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/43.jpg)
Pipeline
![Page 44: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/44.jpg)
Problemswithfeaturetypes
• Bignumberoffeatures->DimensionalityreducCon->SVD,PCA– Dimensionalityreduc)on:“compress”thedatafromahigh-dimensionalrepresentaConintoalower-dimensionalone(usefulforvisualizaConorasaninternaltransformaConforotherMLalgorithms)
• Sparsefeatures->Hashing
![Page 45: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/45.jpg)
• Insteadofusingtwocoordinates(𝒙,𝒚)todescribepointlocaCons,let’suseonlyonecoordinate(𝒛)
• Point’sposiConisitslocaConalongvector𝒗↓𝟏 • Howtochoose𝒗↓𝟏 ?Minimizereconstruc)onerror
SVD–DimensionalityReducCon
v1
first right singular vector
Movie 1 rating
Mov
ie 2
ratin
g
![Page 46: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/46.jpg)
SVD-DimensionalityReducCon
Moredetails• Q:Howexactlyisdim.reduc)ondone?• A:Setsmallestsingularvaluestozero
46
0.56 0.59 0.56 0.09 0.09 0.12 -0.02 0.12 -0.69 -0.69 0.40 -0.80 0.40 0.09 0.09
x x
1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2
0.13 0.02 -0.01 0.41 0.07 -0.03 0.55 0.09 -0.04 0.68 0.11 -0.05 0.15 -0.59 0.65 0.07 -0.73 -0.67 0.07 -0.29 0.32
12.4 0 0 0 9.5 0 0 0 1.3
≈
![Page 47: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/47.jpg)
SVD-DimensionalityReducCon
Moredetails• Q:Howexactlyisdim.reduc)ondone?• A:Setsmallestsingularvaluestozero
47
x x
1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2
0.13 0.02 -0.01 0.41 0.07 -0.03 0.55 0.09 -0.04 0.68 0.11 -0.05 0.15 -0.59 0.65 0.07 -0.73 -0.67 0.07 -0.29 0.32
12.4 0 0 0 9.5 0 0 0 1.3
0.56 0.59 0.56 0.09 0.09 0.12 -0.02 0.12 -0.69 -0.69 0.40 -0.80 0.40 0.09 0.09
≈
![Page 48: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/48.jpg)
SVD-DimensionalityReducCon
Moredetails• Q:Howexactlyisdim.reduc)ondone?• A:Setsmallestsingularvaluestozero
≈ x x
1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2
0.13 0.02 0.41 0.07 0.55 0.09 0.68 0.11 0.15 -0.59 0.07 -0.73 0.07 -0.29
12.4 0 0 9.5
0.56 0.59 0.56 0.09 0.09 0.12 -0.02 0.12 -0.69 -0.69
![Page 49: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/49.jpg)
ǁA-BǁF =√Σij (Aij-Bij)2 is“small”
SVD–DimensionalityReducCon(PCAgeneralizaCon)
Moredetails• Q:Howexactlyisdim.reduc)ondone?• A:Setsmallestsingularvaluestozero
≈
1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2
0.92 0.95 0.92 0.01 0.01 2.91 3.01 2.91 -0.01 -0.01 3.90 4.04 3.90 0.01 0.01 4.82 5.00 4.82 0.03 0.03 0.70 0.53 0.70 4.11 4.11 -0.69 1.34 -0.69 4.78 4.78 0.32 0.23 0.32 2.01 2.01
Frobeniusnorm:ǁMǁF =√Σij Mij
2
![Page 50: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/50.jpg)
Feature selection - example
![Page 51: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/51.jpg)
Dummy Encoding
![Page 52: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/52.jpg)
(De)MoCvaCon
![Page 53: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/53.jpg)
SoluContothoseproblemswithfeatures
![Page 54: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/54.jpg)
Pipeline
![Page 55: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/55.jpg)
Factorization Machine (degree 2)
![Page 56: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/56.jpg)
General Applications of FMs
![Page 57: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/57.jpg)
SummaryPipeline
![Page 58: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/58.jpg)
Pipeline
![Page 59: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/59.jpg)
FromprototypetoproducCon
• PrototypevsProducConCme?–model(pipeline)shouldstaythesame
![Page 60: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/60.jpg)
Libraries
![Page 61: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/61.jpg)
QuesCons?
![Page 62: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/62.jpg)
Thankyou!!!
![Page 63: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/63.jpg)
References• hRps://www.coursera.org/learn/machine-learning
• hRp://www.cs.cmu.edu/~tom/• hRp://scikit-learn.org/stable/• hRp://www.scalanlp.org/• hRp://www.algo.uni-konstanz.de/members/rendle/pdf/Rendle2010FM.pdf
• hRps://securityintelligence.com/factorizaCon-machines-a-new-way-of-looking-at-machine-learning/
![Page 64: Supervised Learning Algorithms - Analysis of different approaches](https://reader031.vdocument.in/reader031/viewer/2022030309/58f206ca1a28abcb448b456f/html5/thumbnails/64.jpg)
References
• AnIntroducContoGeneralizedLinearModels–AnneReDobson,AdrianBarneR
• ApplyingGeneralizedLinearModels–JamesLindsey
• hRps://www.codementor.io/jadianes/building-a-recommender-with-apache-spark-python-example-app-part1-du1083qbw
• hRps://www.chrisstucchio.com/blog/index.html