introducing reactive machine learning
TRANSCRIPT
![Page 1: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/1.jpg)
Introducing Reactive Machine Learning
Jeff Smith @jeffksmithjr
![Page 2: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/2.jpg)
x.ai is a personal assistant who schedules meetings for you
![Page 3: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/3.jpg)
You
![Page 4: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/4.jpg)
nom nom, the data dogScala & Python Spark & Akka Couchbase Machine Learning
![Page 5: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/5.jpg)
Reactive + Machine Learning
![Page 6: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/6.jpg)
Machine Learning Systems
![Page 7: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/7.jpg)
Traits of Reactive Systems
![Page 8: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/8.jpg)
Responsive
![Page 9: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/9.jpg)
Resilient
![Page 10: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/10.jpg)
Elastic
![Page 11: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/11.jpg)
Message-Driven
![Page 12: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/12.jpg)
Reactive Strategies
![Page 13: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/13.jpg)
Reactive Machine Learning
![Page 14: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/14.jpg)
Reactive Machine Learning
![Page 15: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/15.jpg)
Collecting Data
![Page 16: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/16.jpg)
Machine Learning Systems
![Page 17: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/17.jpg)
![Page 18: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/18.jpg)
![Page 19: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/19.jpg)
Uncertainty Interval
27 33
![Page 20: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/20.jpg)
Immutable Factscase class PreyReading(sensorId: Int, locationId: Int, timestamp: Long, animalsLowerBound: Double, animalsUpperBound: Double, percentZebras: Double)
implicit val preyReadingFormatter = Json.format[PreyReading]
![Page 21: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/21.jpg)
Distributed Data Storage
![Page 22: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/22.jpg)
Partition Tolerance
![Page 23: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/23.jpg)
Distributed Data Storage
![Page 24: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/24.jpg)
Immutable Factsval reading = PreyReading(36, 12, currentTimeMillis(), 12.0, 18.0, 0.60)
val setDoc = bucket.set[PreyReading](readingId(reading), reading)
![Page 25: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/25.jpg)
Scaling with Distributed Databases
![Page 26: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/26.jpg)
Almost Immutable Data Model{ "sensor_id": 123, "readings": [ { "sensor_id": 456, "timestamp": 1457403598289, "lower_bound": 12.0, "upper_bound": 18.0, "percent_zebras": 0.60 } ] }
![Page 27: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/27.jpg)
Hot Key
![Page 28: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/28.jpg)
Truly Immutable Data Model{ "sensor_id_reading_id": 123456, "timestamp": 1457403598289, "lower_bound": 12.0, "upper_bound": 18.0, "percent_zebras": 0.60 }
![Page 29: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/29.jpg)
Generating Features
![Page 30: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/30.jpg)
Machine Learning Systems
![Page 31: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/31.jpg)
Feature Generation
Raw Data FeaturesFeature Generation Pipeline
![Page 32: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/32.jpg)
Microblogging Data
![Page 33: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/33.jpg)
Pipeline Failure
Raw Data FeaturesFeature Generation Pipeline
Raw Data FeaturesFeature Generation Pipeline
![Page 34: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/34.jpg)
Supervising Feature Generation
Raw Data FeaturesFeature Generation Pipeline
Supervision
![Page 35: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/35.jpg)
Original Features
object SquawkLength extends FeatureType[Int]
object Super extends LabelType[Boolean]
val originalFeatures: Set[FeatureType] = Set(SquawkLength)val label = Super
![Page 36: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/36.jpg)
Basic Features
object PastSquawks extends FeatureType[Int]
val basicFeatures = originalFeatures + PastSquawks
![Page 37: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/37.jpg)
More Features
object MobileSquawker extends FeatureType[Boolean]
val moreFeatures = basicFeatures + MobileSquawker
![Page 38: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/38.jpg)
Feature Collections
case class FeatureCollection(id: Int, createdAt: DateTime, features: Set[_ <: FeatureType[_]], label: LabelType[_])
![Page 39: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/39.jpg)
Feature Collectionsval earlierCollection = FeatureCollection(101, earlier, basicFeatures, label)
val latestCollection = FeatureCollection(202, now, moreFeatures, label)
val featureCollections = sc.parallelize( Seq(earlierCollection, latestCollection))
![Page 40: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/40.jpg)
Fallback Collections
val FallbackCollection = FeatureCollection(404, beginningOfTime, originalFeatures, label)
![Page 41: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/41.jpg)
Fallback Collectionsdef validCollection(collections: RDD[FeatureCollection], invalidFeatures: Set[FeatureType[_]]) = { val validCollections = collections.filter( fc => !fc.features .exists(invalidFeatures.contains)) .sortBy(collection => collection.id) if (validCollections.count() > 0) { validCollections.first() } else FallbackCollection}
![Page 42: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/42.jpg)
Learning Models
![Page 43: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/43.jpg)
Machine Learning Systems
![Page 44: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/44.jpg)
Learning Models
Features ModelModel Learning Pipeline
![Page 45: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/45.jpg)
Models of Love
![Page 46: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/46.jpg)
Traits of Spark
![Page 47: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/47.jpg)
Reactive Strategies in Spark
![Page 48: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/48.jpg)
Data Preparationval labelIndexer = new StringIndexer() .setInputCol("label") .setOutputCol("indexedLabel") .fit(instances)
val featureIndexer = new VectorIndexer() .setInputCol("features") .setOutputCol("indexedFeatures") .fit(instances)
val Array(trainingData, testingData) = instances.randomSplit( Array(0.8, 0.2))
![Page 49: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/49.jpg)
Learning a Modelval decisionTree = new DecisionTreeClassifier() .setLabelCol("indexedLabel") .setFeaturesCol("indexedFeatures")
val labelConverter = new IndexToString() .setInputCol("prediction") .setOutputCol("predictedLabel") .setLabels(labelIndexer.labels)
val pipeline = new Pipeline() .setStages(Array(labelIndexer, featureIndexer, decisionTree, labelConverter))
![Page 50: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/50.jpg)
Evolving Modeling Strategiesval randomForest = new RandomForestClassifier() .setLabelCol("indexedLabel") .setFeaturesCol("indexedFeatures")
val revisedPipeline = new Pipeline() .setStages(Array(labelIndexer, featureIndexer, randomForest, labelConverter))
![Page 51: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/51.jpg)
Deep Models of Artistic Style
![Page 52: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/52.jpg)
Refactoring Command Line Tools> python neural-art-tf.py -m vgg -mp ./vgg -c ./images/bear.jpg -s ./images/style.jpg -w 800
def produce_art(content_image_path, style_image_path, model_path, model_type, width, alpha, beta, num_iters):
![Page 53: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/53.jpg)
Exposing a Serviceclass NeuralServer(object): def generate(self, content_image_path, style_image_path, model_path, model_type, width, alpha, beta, num_iters): produce_art(content_image_path, style_image_path, model_path, model_type, width, alpha, beta, num_iters) return True
daemon = Pyro4.Daemon() ns = Pyro4.locateNS() uri = daemon.register(NeuralServer) ns.register("neuralserver", uri) daemon.requestLoop()
![Page 54: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/54.jpg)
Encoding Model Types
object ModelType extends Enumeration { type ModelType = Value val VGG = Value("VGG") val I2V = Value("I2V") }
![Page 55: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/55.jpg)
Encoding Valid Configurationcase class JobConfiguration(contentPath: String, stylePath: String, modelPath: String, modelType: ModelType, width: Integer = 800, alpha: java.lang.Double = 1.0, beta: java.lang.Double = 200.0, iterations: Integer = 5000)
![Page 56: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/56.jpg)
Finding the Serviceval ns = NameServerProxy.locateNS(null) val remoteServer = new PyroProxy(ns.lookup("neuralserver"))
![Page 57: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/57.jpg)
Calling the Servicedef callServer(remoteServer: PyroProxy, jobConfiguration: JobConfiguration) = { Future.firstCompletedOf( List( timedOut, Future { remoteServer.call("generate", jobConfiguration.contentPath, jobConfiguration.stylePath, jobConfiguration.modelPath, jobConfiguration.modelType.toString, jobConfiguration.width, jobConfiguration.alpha, jobConfiguration.beta, jobConfiguration.iterations).asInstanceOf[Boolean] }))}
![Page 58: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/58.jpg)
Profiles with Style
![Page 59: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/59.jpg)
Hybrid Model learning
Features ModelModel Learning Pipeline
![Page 60: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/60.jpg)
Publishing Models
![Page 61: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/61.jpg)
Machine Learning Systems
![Page 62: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/62.jpg)
Publishing Models
Model Predictive ServicePublishing Process
![Page 63: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/63.jpg)
Building Lineages
val rawData: RawData val featureSet: Set[FeatureType] val model: ClassificationModel val modelMetrics: BinaryClassificationMetrics
![Page 64: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/64.jpg)
Predicting
![Page 65: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/65.jpg)
Machine Learning Systems
![Page 66: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/66.jpg)
Predicting
Client Predictive ServiceRequest> <Prediction
![Page 67: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/67.jpg)
Models as Servicesdef predict(model: Features => Prediction) (features: Features): Future[Either[String, Prediction]] = { ??? }
![Page 68: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/68.jpg)
Models as Servicesval routes = { pathPrefix("model") { (post & entity(as[PredictionRequest])) { context => complete { predict(model: Features => Prediction)(extractFeatures(context)) .map[ToResponseMarshallable] { case Right(prediction) => PredictionSummary(prediction) case Left(errorMessage) => BadRequest -> errorMessage } } } } }
![Page 69: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/69.jpg)
Clojure Functions
![Page 70: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/70.jpg)
Clojure Predictive Service
![Page 71: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/71.jpg)
Predicting
Client Predictive ServiceRequest
![Page 72: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/72.jpg)
Summary
![Page 73: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/73.jpg)
Machine Learning Systems
![Page 74: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/74.jpg)
Traits of Reactive Systems
![Page 75: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/75.jpg)
Reactive Strategies
![Page 76: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/76.jpg)
Reactive Machine Learning
![Page 77: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/77.jpg)
For Later
![Page 78: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/78.jpg)
manning.com reactivemachinelearning.com medium.com/data-engineering
M A N N I N G
Jeff SmithUse the code reactnymu for 39% off!
![Page 80: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/80.jpg)
Thank You!
![Page 81: Introducing Reactive Machine Learning](https://reader034.vdocument.in/reader034/viewer/2022051520/58727be11a28abc7068b5565/html5/thumbnails/81.jpg)
Code: reactnymu
Jeff Smith @jeffksmithjr reactivemachinelearning.com
M A N N I N G
Jeff Smith