monitoring and analyzing london's air quality with pentaho ... · monitoring and analyzing...
TRANSCRIPT
MonitoringandAnalyzing London'sAirQualitywithPentahoMarkSemenenko,PentahoSalesEngineer,HitachiVantaraAnyaRumyantseva,PentahoDataScientist,HitachiVantara
Pentaho
Agenda
DevelopingPentahoDemoshowcasingtheplatform’send-to-endcapabilities
• Pentahoasanend-to-endplatform• UseCaseOverview• Demonstration
• Summary
Pentahoprovidesflexiblemanagementforend-to-enddataflows…
DataPrepDataEngineering Analytics
Ingestion Processing Blending DataDelivery DataDiscovery/Analysis
Analysis&Dashboards
Administration Security LifecycleManagement
DataProvenance
DynamicDataPipeline Monitoring Automation
…andMachineLearningcapabilities!
DataTransformationandMachineLearning
Workflows
Fullautomatedproduction
Dashboarddecisionsupport
ypred
DataEngineering DataPreparation Analytics
X
Dataco
nnectors
DataSource1
DataSource2
DataSourcen
Aim:developademoshowingafullspectrumofPentahocapabilities
OurUseCase“LondonAirQualityMonitoring”
Why?
• Airqualityaffectswell-beingofresidentsandvisitorsofthecity• 9000earlydeathsayearinLondonduetoairpollution• Airpollutionmonitoringandmitigationarecrucialforcitycouncils
• Theusecaseiseasytounderstandandtofollow
DataSources
• LondonAirQualityNetwork(96stationsacrossthecity)• RoadTrafficCounts(numberandvehicletype)
• MeteorologicalParameters(wind,temperature,humidity)
Collecting,PreparingandBlending
Demonstration
Collecting,PreparingandBlending
AirQualityPrediction
PentahoMachineLearningOrchestration
1 2
4
3
1
PredictivemodelforAirQualitywasaddedtothedemo.
*Long-ShortTermMemoryneuralnetworkwasusedtomakepredictions
Demonstration
OperationalisingPredictiveModelinPentaho
Whatwehaveshowninthedemo:
• Featureengineeringanddatapreparation• Scheduledmodeltrainingandupdate
• Modelapplicationonnewdatastreams
• Trackingandstoringmodelperformancemetrics
• DisplayingpredictionsinPentahodashboards
OperationalizemachinelearningmodelsinPentaho!
Summary
Whatwecoveredtoday:
• Pentahoprovidesasingleconsistentexperiencefordevelopingdataproducts• Theplatformallowstoputmachinelearningalgorithmsintoproduction
• Informativedashboardsfordecisionmakingbasedonadvancedanalyticsanddatatransformationinthebackend