Upload File

machine data. big data. where do you point the firehose?

32
Matthew D. Smith 24 March 2015 @darthfiber @fieldcloud [email protected] http://fieldcloud.com +33 9 77 19 99 33

Upload: fieldcloud-sas

Post on 18-Jul-2015

126 views

Category:

Data & Analytics


3 download

Report

Tags:

TRANSCRIPT

Page 1: Machine Data. Big Data. Where do you point the firehose?

Matthew D. Smith 24 March 2015 @darthfiber @fieldcloud [email protected] http://fieldcloud.com +33 9 77 19 99 33

Page 2: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

about us

we offer professional services, connectivity solutions and manufacture ruggedized, low power data gateway

computing platforms.

we help you securely capture, transfer and store your production measurement data

inside your firewall or in the cloud…

Page 3: Machine Data. Big Data. Where do you point the firehose?
Page 4: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 5: Machine Data. Big Data. Where do you point the firehose?
Page 6: Machine Data. Big Data. Where do you point the firehose?
Page 7: Machine Data. Big Data. Where do you point the firehose?
Page 8: Machine Data. Big Data. Where do you point the firehose?
Page 9: Machine Data. Big Data. Where do you point the firehose?
Page 10: Machine Data. Big Data. Where do you point the firehose?
Page 11: Machine Data. Big Data. Where do you point the firehose?
Page 12: Machine Data. Big Data. Where do you point the firehose?
Page 13: Machine Data. Big Data. Where do you point the firehose?
Page 14: Machine Data. Big Data. Where do you point the firehose?
Page 15: Machine Data. Big Data. Where do you point the firehose?
Page 16: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 17: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

from sensor to engineer desktop

Wellbore/Wellhead

Raw Data Telemetry

Physical properties to be measured

Sensors Raw Signal

HQ Office Environment

Advanced Data Analysis

Tools

Central Server Environment

Long Term Storage Engineer Desktop

Surface Production Facility

Acquisition Systems

Edge Application

Gateway

Engineering Values Secure Data Delivery

FSK/optical downhole

connectivity

surface acquisition systems

PLCs/RTUs/DCS

ruggedized connectivity and processing

long term data storage

system operation platform

3rd Party Software

Analytics

Visualization

DTS, P/T Gauges, Electric and Hydraulic Valves, Electrical Submersible Pumps (ESP), Multiphase Flow Meters (MPFM)

Page 18: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 19: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 20: Machine Data. Big Data. Where do you point the firehose?
Page 21: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 22: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 23: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 24: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 25: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 26: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

<$/MB

>bps

Page 27: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 28: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 29: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 30: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 31: Machine Data. Big Data. Where do you point the firehose?

the secure data logistics specialists

Page 32: Machine Data. Big Data. Where do you point the firehose?

Realtime Analytics on the Twitter Firehose with Cassandra

Consuming Data Lakelearnandbecurious.cloud/data/decks/Consuming_Data_Lake.pdf · BI & Data Visualization Kinesis Streams & Firehose Batch EMR Hadoop, Spark, Presto Redshift Data Warehouse

arXiv:1306.5204v1 [cs.SI] 21 Jun 2013 · Streaming API and the Firehose. The Data From December 14th, 2011 - January 10th, 2012 we col-lected tweets from the Twitter Firehose matching

Package ‘RTCGAToolbox’ - Bioconductor · Package ‘RTCGAToolbox’ April 8, 2020 Type Package Title A new tool for exporting TCGA Firehose data Version 2.16.2 Description Managing

EC-TEL 2008 - Firehose session introduction to automatical metadata extraction

Firehose instability in heliospheric and astrophysical plasmas

2013-07-11 1 Out-of-the-Box Computing Patents pending Google7/11 2013 Drinking from the Firehose The Belt machine model in the Mill CPU Architecture

FIREHOSE,PAGERANK AND NVGRAPH - CLSAC · 2019-11-03 · FIREHOSE,PAGERANK AND NVGRAPH GPU ACCELERATED ANALYTICS CHESAPEAKE LARGE SCALE DATA ANALYTICS, OCT 2016 Joe Eaton Ph.D. 2 Agenda

Harness the po wer of your data · 2020. 8. 2. · AWS Snowball AWS Snowmobile AWS Database Migration Service AWS IoT Core Amazon Kinesis Data Firehose ... Using an AWS data lake

Processing Big Data in Motion - IEEE Computer Society · Amazon Kinesis Firehose • Zero Admin: Capture and deliver streaming data into S3, Redshift, and other destinations without

Photography - A Sip from a Firehose!

Drinking From The Firehose

Amazon Kinesis Firehose - Firehose

Building Big Data Storage Solutions (Data Lakes) for ... · Introduction 1 Amazon S3 as the Data Lake Storage Platform 2 Data Ingestion Methods 3 Amazon Kinesis Firehose 4 AWS Snowball

Drupal Theming from a firehose

Analyzing Big Data on the Fly - QCon New York · 2014-08-27 · Core Concepts recapped • Data record ~ tweet • Stream ~ all tweets (the Twitter Firehose) • Partition key ~ Twitter

Drinking From The Firehose - The Erlang Way

2016 11 03-GDC-Firehose-Rungdac.broadinstitute.org/runs/docs/2016_11_03-GDC-Firehose-Run.pdfGDAC Firehose Integration With The Genomic Data Commons [email protected] 2016_11_18

Sipping from the firehose

Future of IT - Store & Retrieve Data Anywhere · DATA TRANSFER S3 Transfer Acceleration AWS Storage and File Gateway AWS Direct Connect Amazon Kinesis Firehose AWS Server Migration

Shazam ~ Leopard Shazam enjoys his firehose … Helping Wildcats Shazam ~ Leopard Shazam enjoys his firehose hammock Projects you and your school or organization can do to help wildcats

Drinking from a firehose

Rf Firehose Friction Revised

Drinking from a Firehose: Large-Scale Search in Consumer ...files.meetup.com/1331888/SearchInVideos-SemanticWebMeetup.pdf · Drinking from a Firehose: Large-Scale Search in Consumer-Produced

SUMMIT - Amazon Web Services... · Kinesis Data. Streams. Kinesis. Data. Analytics. Kinesis Data Firehose • Capture • Transform • Load data streams • Process data streams

A Scala Firehose - jaoo.dkjaoo.dk/dl/jaoo-aarhus-2008/slides/BillVenners_scalaFirehoseCompre… · A Scala Firehose Bill Venners President Artima, Inc. . Enough about me: • Run

Firehose Engineering

Taming the Firehose

Amazon Kinesis Data Firehose API Reference - API Reference · Amazon Kinesis Data Firehose API Reference API Reference CreateDeliveryStream CreateDeliveryStream Creates a Kinesis

Flex 4 components from the firehose

Drinking from the Firehose - Real-time Metrics

Big “Serverless” Data - Big Data Days · Serverless in big data processing Amazon Kinesis Video Streams Amazon Kinesis Data Streams Amazon Kinesis Data Firehose ... Amazon Kinesis

(BDT320) New! Streaming Data Flows with Amazon Kinesis Firehose

Stream Data Analytics with Amazon Kinesis Firehose & Redshift - AWS August Webinar Series

Real-Time Serverless Machine Learning · 2017. 10. 20. · AWS Lambda Amazon S3 Data Storage ... Lambda DynamoDB Amazon S3 JavaScript SDK Amazon Rekognition Amazon Kinesis Firehose