re-imagination of enterprise architecture - part 1 - v2
TRANSCRIPT
Research & InnovationAPI & Platform
Business Strategy & Digital TransformationNew Usages, Connected Business & Mobility
Re-Imagination of Enterprise Architecture
William El Kaim – April 2015 – Part 1 - V 2.3
Copyright © William El Kaim 2015 3
Plan
• The Entrepreneurial Age
• Rise of Platforms & Ecosystems
• Business Agility Through API
• Looking for the Next Gen
Architecture
• RestFul Architecture
• Antifragile Architecture
• MicroService Architecture
• 3rd Generation Mobile Architecture
• (Big-)Data Architecture (Re-)
Invented
• New Databases That Scales High
• The Devops Movement
• From Virtual Machines to Containers
• Programmatic Infrastucture
• Backend as a Service
• User Experience
• Are You Ready?
Copyright © William El Kaim 2015 4
The Entrepreneurial Age
Copyright © William El Kaim 2015 5
Companies Were Designed like Machines …
• The average life expectancy of a company in the S&P 500 has dropped
precipitously, from 75 years (in 1937) to 15 years in a more recent study.
Copyright © William El Kaim 2015 6
Companies Were Designed like Machines …
Copyright © William El Kaim 2015 7
Companies Were Designed like Machines …
• Companies were built on a corporate model of “knowledge stocks”
• Developing a proprietary product breakthrough and then defending that innovative
advantage against rival companies for as long as possible.
• The issue is now, because of the increasingly global nature of business
competition, the value of a major proprietary breakthrough or invention
erodes in value much more quickly than in the mid-20th Century.
• Companies should create broad networks and finding innovation at “the
edge” of their business rather than a proprietary core
Source: John Hagel III Copyright © William El Kaim 2015 8
Red Queen Effect
Source: MeedabyteCopyright © William El Kaim 2015 9
Seven Forces At Work
• New Pressure On Prices And Margins
• Competitors Emerge From Unexpected Places
• Winner-takes-all Dynamics
• Plug-and-play Business Models
• Growing Talent Mismatches
• Converging Global Supply And Demand
• Relentlessly Evolving Business Models At Higher Velocity
Source: McKinsey Strategic Principles for competing in the digital ageCopyright © William El Kaim 2015 10
Managing The Strategic Challenges: Six Big
Decisions• Decision 1: Buy or sell businesses in your portfolio?
• Decision 2: Lead your customers or follow them?
• Decision 3: Cooperate or compete with new attackers?
• Decision 4: Diversify or double down on digital initiatives?
• Decision 5: Keep digital businesses separate or integrate them with current
non digital ones?
• Decision 6: Delegate or own the digital agenda?
Source: McKinsey Strategic Principles for competing in the digital ageCopyright © William El Kaim 2015 11
RE-imagination of Everything
• You should not only re-invent your brand and product, you should re-imagine
everything (Mary Meeker 2012)
• The risk if not doing so? Being victim of innovative disruptions (see
Christensen’s video)
• Christensen predicts also the end of consulting as we know it today.
• This is also the end of the Enterprise Architect, and the tools and process he used!
• The Schumpeter creative destruction theory, is maximized in the digital
business world
• Winner-Take-All Effect + Network effects
• First Time User Experience is key (30/50% of dev), no more documentation!
Copyright © William El Kaim 2015 12
The New Digital Operating Model …
Source: Aaron Dignan Medium blog PostCopyright © William El Kaim 2015 13
Put customers at the center of the digital operating model
Source: PWCCopyright © William El Kaim 2015 14
Put customers at the center of the digital operating model
Key characteristics of the digital operating model contrasted with the way many companies typically operate
Source: PWCCopyright © William El Kaim 2015 15
Change Your Digital Metabolism!
Copyright © William El Kaim 2015 16
From System of Records to System of Engagement
Copyright © William El Kaim 2015 17
Embrace Innovation (S-curve)
Source: Mastering 2020 - Rolland Berger ConsultingCopyright © William El Kaim 2015 18
Innovate in Business Models
• From inside-out to outside-in
• be open to external ideas
• Personalized offers in a mass market world …
• It’s the age of the platform, use API to sell through (target developers)
• Become a payment operator and offer a digital wallet
• Invest in digital marketing and advertising
• Target business niches and become the leader first and fast
• Be innovative in business models (niche)
• Bundle-unbundle your service offer (“a la carte”)
Excerpted from Big Bang Disruption: Strategy in the Age of Devastating Innovation by Larry Downes and Paul Nunes
Copyright © William El Kaim 2015 19
Invest in Algorithms
20Copyright © William El Kaim 2015 https://algorithmia.com/
Rise of Platforms & Ecosystems
Copyright © William El Kaim 2015 21
Companies Like Living Organism
• To design the connected company focus on the company as a complex
ecosystem, a set of connections and potential connections, a decentralized
organism that has eyes and ears everywhere that people touch the
company, whether they are employees, partners, customers or suppliers.
• Ecosystem:
• Companies to act as decentralized ecosystems, tolerant of “activities at the margins.”
• Very active in partnerships and joint ventures.
• Companies with a strong, shared culture.
• Everyone in the company understood the company’s values.
• Active listening
• Companies with their eyes and ears focused on the world around them and constantly
seeking for opportunities.
Source: Aaron Dignan Medium blog PostCopyright © William El Kaim 2015 22
From Product to Platforms
Copyright © William El Kaim 2015 23
1. All teams will expose their data…2. Teams must communicate through interfaces.3. … no other form of interprocess communication
allowed4. Interfaces, without exception, must be
externalizable.5. Anyone who doesn’t do this will be fired.
Platform: Openess and API
A platform is a system that can be… adapted to countless needs and niches that the platform’s original developers could not possibly have contemplated…”
Mark Andreessen
Copyright © William El Kaim 2015 24
Across all platforms, we observe the following 3 layers
What is a Platform?
Source: Platform Thinking
Every platform is a different configuration of this stack
Copyright © William El Kaim 2015 25
What is a Platform?
Source: Platform ThinkingCopyright © William El Kaim 2015 26
What is a Platform?
• Platform is one of the most misunderstood ideas in the world of the Digital
world.
• Platforms can be accidental or intentional.
• A platform is a foundational product that moves beyond product status by
encouraging others to build, play, and/or iterate on top of it.
• The value and utility of the system is continually being discovered and expanded not just
by the organization, but by its users and customers.
• Platforms are shared innovation engines that outsource the costly and
uncertain discovery process.
• Many platforms today are 100% software, but they don’t have to be.
• AirBnB and Uber turned the physical world (cars and housing) into a platform for
millions.
Source: Aaron Dignan Medium blog PostCopyright © William El Kaim 2015 27
Platform and Ecosystems
• Users are building businesses on the back of the platform, and in some
cases changing how they operate in order to better serve the platform.
• Establishing a platform in the center of a robust digital ecosystem requires a
digital operating model, one that is appropriately permeable to third parties
that can co-create new value from what a company and others have to offer.
• The Shift
• From a platform the company builds upon to a platform the world builds upon!
• Example
• Leap motion’s device was built as a platform from day one, and developers have
invented countless uses, including iron man inspired rocket design
Source: Aaron Dignan Medium blog PostCopyright © William El Kaim 2015 28
The Cathedral or The Bazaar?
• The short-lived digital bazaar based on networked individuals working in small teams and localized physically all over the world, but digitally near• Any idea, innovation, will be funded (by the crowd
sometimes), created (maker movement), and sold directly to the public (no more intermediary!).
• The product and team will stay alive until the demand stops. This could be imagined as the generalization of the open source, open innovation and kick starter movement.
• The mega-monopolistic brands (or the cathedral). • Each industry will see the rise of two or three walled digital
ecosystems controlled by a short number of well recognized brands.
• Short-lived digital bazaar will emerge and live in each niches not targeted by the mega-monopolistic brands.
• Influenced by Eric Raymond's The Cathedral and the Bazar: Musings on Linux and Open Source by an Accidental Revolutionary, 2001, O'Reilly Media
Copyright © William El Kaim 2015 29
Technology Platform Evolution
Source: PWCCopyright © William El Kaim 2015 30
Importance of Channels
Copyright © William El Kaim 2015 31
Importance of Channels
In the linear interpretation of business, value
creation takes place within the firm, which turns
blocks and primary products in complex products
and services, which then submits to the market in a shareholder value maximization process.
Business as a platform: the company’s goal becomes
to maximize the opportunities of value exchange
between peers through two core activities: the creation of
blocks, modules and contexts for creation (eg, APIs, Tools,
Contests, Conferences …) and the identification,
establishment and improvement of channels for the exchange of value between peers (eg: marketplaces).
Source: MeedabyteCopyright © William El Kaim 2015 32
Importance of Channels
• Channels must be
implemented to facilitate
emerging value
exchanges.
• In platforms, currencies
such as trust and
reputation may be
required to facilitate
transactions and should
be clearly valued.
Source: Simone Cicero - meedabyte.com
Copyright © William El Kaim 2015 33
Platform Business Model
Weill & Woerner (2013), “Optimizing Your Digital Business Model,” MIT Sloan Management Review
Copyright © William El Kaim 2015 34
Platform Design Canvas
Copyright © William El Kaim 2015 35
Platform and APIs
The new digital, networked, real-time society forces us to start thinking and acting as an ecosystem.
Ecosystems are developed using platforms to glue services via API and funds to encourage startups and partners to hook in
Source: PWC, Exploiting the growing value from information
Build your own Minimum Viable Digital Platform
And create Open APIs To encourage startups and partners
to hook in
Copyright © William El Kaim 2015 36
Business Agility Through API
Copyright © William El Kaim 2015 37
Business Agility Through API
Copyright © William El Kaim 2015 38
API Values
• APIs unlock distribution channels by allowing data, content and services to be accessible and usable on any device, anywhere.
• By opening up business assets to other parties, APIs ease considerably partnership process. Potential partners are able to make use of the API to design new products and services.
Copyright © William El Kaim 2015 39
API typology
Copyright © William El Kaim 2015 40
API Usage
• An API can be seen as
• A technical “plumbing” between dispersed systems
• A Way to feed/extend applications/web sites with added value services and data
• An API is targeted towards DEVELOPPERS or System Integrator
Copyright © William El Kaim 2015 41
From API to Open API?
• An open API does not mean free!
• An open API means:
• Openly documented
• Available via self-service (i.e. developers can sign up on a website, get an API key, with
no hassle)
• and using open Internet technologies (SOAP, REST, RSS).
• When opening up data through an Open API (whether it is private, partner or
public), the Open API provider does the partnership work once, partners then
need only onboard themselves and use their own resources as often as they
like for marginal additional cost to the provider.
• An open API provider creates the infrastructure and then each partner does the
technical, business and legal work on their end.
“Open” Means “As Open as You’d Like”
Copyright © William El Kaim 2015 42
API Billionaires
Copyright © William El Kaim 2015 43
Open API: to Grow and Innovate
Give access to what you do best and use what others do best
Copyright © William El Kaim 2015 44
Open APIs Create Ecosystems
Copyright © William El Kaim 2015 45
Finding Open API Via Yellow Pages
• Programmable Web
• ApiHUb
• The right API
• Mashape
• Find Web API
• Api for That
• Exicon API directory
• apis.io
Copyright © William El Kaim 2015 46
Why Using Open API?
Copyright © William El Kaim 2015 47
Purpose of drives Open API type!
Copyright © William El Kaim 2015 48
API Driven Companies
Copyright © William El Kaim 2015 49
Four Types Of Business Apise.g
The API is the product1
The API projects the product2
The API powers and feeds the product4
The API promotes the product3
Copyright © William El Kaim 2015 50
Four Types Of Business Apis
Digging Deeper
Is the product
• Direct revenue
• Utility / Pay per transaction
• Tiered pricing bands
Projects the product
• Reach more places
• Provides more utility
• Enable mobile
• Allow deeper integration
Promotes the product
• Biz Dev / Lead Gen
• User acquisition
• Advertising
• Brand promotion
• Affiliate programs
Powers and feeds the product
• Content acquisition
• Partner tie-in
• Internal innovation
1 2 3 4
Each type of API has different potential business value associated with it
Copyright © William El Kaim 2015 51
API Represent a Shift In Traditional Business
Models
Copyright © William El Kaim 2015 52
The Cloud API Market
• API Management
• Microsoft Acquires Apiphany, an API management service to integrate with Windows
Azure.
• Intel bought Mashery a 125 persons firm for about $180 million.
• CA acquired Layer 7.
• Apigee and WSO2 API Manager are the only one still a pure players!
• API Yellow Pages
• Mulesoft acquired Programmable Web.
• Backend As a Service
• Facebook bought Parse.
Copyright © William El Kaim 2015 53
Describing Your Api
• Two main languages
• RAML (RESTful API modeling language)
• API Blueprint (http://apiblueprint.org/)
• Can describe Web APIs
• XML or JSON driven representations
• proper HTTP methods usage
• markup languages (XML, JSON, YAML, MarkDown)
• Can generate code
• client SDKs
• server skeleton
Copyright © William El Kaim 2015 54
API Blueprint
• Apiary with API Blueprint was the first company dedicated specifically to API
design
• Apiary.io’s API definition language designed to allow anyone, not just developers to
design APIs
• Objectives
• API Blueprint is all about simplicity, and allowing people to have a structured
conversation around an API, with the people who are actually going to be using it.
• Apiary is the conduit for this conversation, allowing developers to easily create a mock
interface from the blueprint, share with consumers, and iterate as necessary.
• Tutorial here / Code in GitHub
Source: Api Evangelist BlogCopyright © William El Kaim 2015 55
RAML
• RESTful API Modeling Language (RAML)• A concise, expressive language for describing RESTful APIs. RAML is built on broadly
used standards such as YAML and JSON and is a non-proprietary, vendor-neutral open spec.
• RAML is developed by Mulesoft• The motivation behind RAML was about seeking one source of truth, and a basic
representation of an API.
• RAML gives the ability to design an API in a format that allowed them to sit down with stakeholders in a webinar and get instant feedback (bringing the API to forefront, not the implementation).
• Tools• Swagger
• Google API Discover Service
• Apiary.io (Api Blueprint)
• Mulesoft AnyPoint API Portal (RAML)
Source: Api Evangelist BlogCopyright © William El Kaim 2015 56
API Integration
• API integration services and tooling, for testing, monitoring, and
transforming API calls is one of the fastest growing segments of the API
space.
• Solid solutions from:
• SmartBear
• Runscope
• TheRightAPI
• Nomos Software,
• API Science,
• APITools is definitely raising the stakes with open sourcing theirs offering.
Copyright © William El Kaim 2015 57
API Mgt
• WebShell : http://webshell.io/
• Apigee : http://apigee.com/ and chrome console
• Mashery http://www.mashery.com/
• 3scale : http://3scale.net/
• ApiPhany : http://apiphany.com/
• Apiary.io : http://apiary.io/
• Strikeiron: http://www.strikeiron.com/
• Intel ExpressWay: http://cloudsecurity.intel.com/api-management
• ApiSpark: http://apispark.com/
Copyright © William El Kaim 2015 58
API Monitoring
• Developers want to build on the services they provide and the APIs are
entirely reliable.
• Stormpath provides authentication and user management for developers.
• PagerDuty is a service that offers IT monitoring tools through alerts via SMS, phone,
email, iOS or Android.
• Runscope offers a service for monitoring API traffic.
Copyright © William El Kaim 2015 59
Dos & Don’ts: Tips To Avoid Pitfalls
• Define revenue value chain
• Deploy "sense and respond" and innovation toolkits rather than applications with fixed
functionality
• Propose several business models
• Adapted for multiple distribution channels
• Think DATA (Stop thinking Application/IT product)
• Adopt a flow based vision where real time data is valorised
• Implement Open API
• Invest on Business Analysis for finding the most valuable travel services to offer /build.
• Enhance User Experience
• Let users select their best in class solutions for each delivery channel
Copyright © William El Kaim 2015 60
Dos & Don’ts: Conversations
• Use standard and dedicated to developer collaboration and social tools to
ease discussions with developer
• GitHub
• GitHub is a web-based hosting service for software development projects. Originally
born as a project to simplify sharing code, GitHub has grown into the largest code host in
the world. GitHub offers both commercial plans and free accounts for open source
projects.
• Geeklist
• Geekli.st is an achievement-based social portfolio builder for developers where they can
communicate with colleagues and employers and build credibility in the workplace.
• Stackoverflow
• Stack Overflow is a free programming Q & A site. Stack Overflow is collaboratively built
and maintained by its members.
Copyright © William El Kaim 2015 61
Looking for the Next Gen Architecture
Copyright © William El Kaim 2015 62
Three-tier Architecture Lacks Scalability
• Designed in an era where the idea of elasticity and rapid scaling did not broadly exist.
• Functional components of the application are packaged together as a unit (the monolith), so the only way you can respond to changing levels of client demand is to scale the entire application.
• Applications running on the three-tier architecture are typically unable to scale specific pieces of the application independently because the entire application is coupled together.
• Regardless of whether you have an e–commerce store, a social media application, or a blog, a basic requirement for today’s applications is the ability to scale up and down on demand; preferably at as low cost as possible.
Source: Andreas SchroederCopyright © William El Kaim 2015 63
Three-tier Architecture Lacks Scalability
Shift of all the UI logic from the server to the client
Source: Octo technologyCopyright © William El Kaim 2015 64
Three-tier Architecture Lacks Scalability
• Still huge codebases (one per layer)
• … with the same impact on development, building, and deployment
• Scaling works better, but still limited
• Staff growth is limited: roughly speaking, one team per layer works well
• Developers become specialists on their layer
• Communication between teams is biased by layer experience (or lack
thereof)
Source: Andreas SchroederCopyright © William El Kaim 2015 65
Four-Tier Engagement Platform
Source: Mobile Needs A Four-Tier Engagement Platform, Forrester report, October 18, 2013Copyright © William El Kaim 2015 66
Four-Tier Engagement Platform
• Client tier
• Packages the unique capabilities of both the experience and the device requesting
information—anything from various mobile clients and wearables, to objects within the
Internet of Things.
• Frees developers from having to customize development to each mobile device, which
allows them instead to focus on building out a single app, increasing productivity, and
decreasing maintenance load.
• Delivery tier
• Optimizes delivery of the digital experience to the user using intelligence received from
the client layer.
• Uses intelligence-driven solutions such as content delivery networks (CDNs) and on-the-
fly optimization tools such as those used for compressing images to decrease bandwidth
• Utilizes sophisticated caching algorithms and tools that enable devops to monitor and
resolve application performance and delivery issues in real time.
Source: Tibco BlogCopyright © William El Kaim 2015 67
Four-Tier Engagement Platform
• Aggregation tier• Serves as the center of application logic, performing tasks like translating between
SOAP to JSON encoding or combining third-party and in-house algorithms to perform complex calculations
• Manages the API layer and facilitates simple service composition.
• This tier connects app requests to the right services with bidirectional, real-time translation to the right data format for back-end and third-party systems, as well as client requests.
• Services tier• Continues to maintain functionality for data both internally and externally.
• By architecting this layer to continuously deploy services, the rate of consumption does not affect the other tiers within the system.
• Without a strong services layer providing the foundation, and ultimately the execution for your application, maintenance and updating can be costly and difficult.
• The services tier is designed for a microservices approach, one that is designed to be open and pluggable, and focuses on the integration and composition of existing services a company has already built as well as new open source libraries.
Source: Tibco BlogCopyright © William El Kaim 2015 68
Four-Tier Engagement Platform – So What?
• The most dramatic difference in this new model is the client tier
• User-facing layer has its own independent set of functionality that leverages the delivery,
aggregation, and services layers beneath it to create device-specific and highly tailored
experiences.
• Isolation gives front-end and user-experience designers much more control to create
memorable digital experiences by tailoring them to the specific user context
Copyright © William El Kaim 2015 69
“Next-Gen” Architecture
• Technology that Scales• Common migrations to {Python, Go, JVM languages}
• Concurrency
• Asynchrony
• Independent systems• Fit-for-purpose systems: analytics, search
• Layered services: caching, etc.
• Event queue
• Scalable persistence• Break up the monolithic database
• Functional partitioning
• Sharding
• Scalable Infrastructure• IaaS for some systems
Looking for the “Minimal Viable Architecture”
Source: Randy SoupCopyright © William El Kaim 2015 70
RESTFul Architecture
Copyright © William El Kaim 2015 73
REST and RESTful
Copyright © William El Kaim 2015 74
REST: Design for comprehensibility
Copyright © William El Kaim 2015 75
Entering JSON
• JSON is a resource-based data transfer mechanism, to further simplify the
process of communication between the information seeker (the client), and
the system containing the information to be consumed (the server).
• JSON is derived from the JavaScript scripting language, which is widely
used in web browsers to enhance interfaces and build dynamic websites.
• Like REST, JSON is noted for its simplicity and usability
• With JSON, data is sent in plain text, which makes it easy to read and understand.
• Because so many web client programs are written in JavaScript, JSON data
arrives ready to be consumed without needing further manipulation.
• At the same time, JSON lacks display capabilities and has a limited amount of
development tool support.
Source: PWCCopyright © William El Kaim 2015 76
Entering JSON
Copyright © William El Kaim 2015 77
RESTful Architecture and REST Languages
Copyright © William El Kaim 2015 78
REST Attachments
Copyright © William El Kaim 2015 79
SOAP vs. REST
• For many enterprises, the path to web services begins with the adoption of a
service-oriented architecture (SOA), which uses SOAP as a method for
exchanging information.
• In today’s web service world, both SOAP and REST are used as methods of
communication.
• There are several factors behind the popularity of REST when contrasted
with SOAP.
• REST uses simple HTTP and therefore standard commands—such as GET, PUT, POST,
and DELETE—to coordinate communication between clients and servers.
• SOAP has no widely accepted methods corresponding to GET, PUT, POST, and
DELETE, which leaves developers free to define their own semantics. But the result can
be complex, proprietary mechanisms to connect components.
Source: PWCCopyright © William El Kaim 2015 80
SOAP vs. REST
• Familiarity
• Since REST is closely related to web design, web developers do not face a steep
learning curve. REST is also language and platform agnostic.
• SOAP requires a significant skill set in SOA-specific development and delivery tools.
• Efficient with bandwidth.
• The use of the existing web infrastructure eliminates the need for an additional
messaging layer in RESTful APIs. Coupled with the fact that REST uses those short
request and response sequences, these APIs consume considerably less bandwidth
than SOAP-based APIs.
• Scalability.
• With simpler component implementations and reduced complexity in the connection
semantics, RESTful services can scale—as evident from several services that register
more than 1 billion API calls each month.
Copyright © William El Kaim 2015 81
SOAP vs. REST
Source: PWCCopyright © William El Kaim 2015 82
SOAP vs. REST …
Copyright © William El Kaim 2015 83
JSON vs. HTML
Copyright © William El Kaim 2015 84
REST Versioning
Copyright © William El Kaim 2015 85
Richardson Maturity Model
• Level 0: HTTP as a transport system for remote interactions
• Level 1: Resources
• Level 2: HTTP Verbs
• Level 3: Hypermedia Controls
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 86
Level 0: HTTP As A Transport System
• The starting point for the model is using HTTP as a transport system for
remote interactions, but without using any of the mechanisms of the web.
• Essentially what you are doing here is using HTTP as a tunneling
mechanism for your own remote interaction mechanism, usually based
on Remote Procedure Invocation.
RPC style system. It's simple as it's just slinging plain old XML (POX) back and forth. If you use SOAP or XML-RPC it's basically the same mechanism, the only difference is that you wrap the XML messages in some kind of envelope.
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 87
Level 1 - Resources
• The first step towards the Glory of Rest is to introduce resources.
• So now rather than making all our requests to a singular service endpoint,
we now start talking to individual resources.
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 88
Level 2 - HTTP Verbs
• Using the HTTP verbs as closely as possible to how they are used in HTTP
itself.
• Use of GET for a request like this is crucial. HTTP defines GET as a safe operation, that
is it doesn't make any significant changes to the state of anything. This allows us to
invoke GETs safely any number of times in any order and get the same results each
time.
• n important consequence of this is that it allows any participant in the routing of requests
to use caching, which is a key element in making the web perform as well as it does.
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 89
Introduction to HATEOAS
• HATEOAS is a constraint of REST.
• The principle is that a client interacts with a network application entirely
through hypermedia provided dynamically by application servers.
• A REST client needs no prior knowledge about how to interact with any particular
application or server beyond a generic understanding of hypermedia.
• In a service-oriented architecture (SOA), clients and servers interact through a
fixed interface shared through documentation or an interface description language (IDL).
• RESTful service can be described as well to be available for the client code-
generation, RSDL (RESTful Service Description Language) using dynamic metadata
collection to achieve this goal.
• The HATEOAS constraint serves to decouple client and server in a way that
allows the server to evolve functionality independently.
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 90
Level 3 - Hypermedia Controls
• The final level introduces HATEOAS (Hypertext As The Engine Of
Application State).
• It addresses the question of how to get from a list open slots to knowing what to do to
book an appointment.
• Hypermedia controls tell us what we can do next, and the URI of the
resource we need to manipulate to do it.
• Rather than us having to know where to post our appointment request, the hypermedia
controls in the response tell us how to do it.
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 91
Level 3 - Hypermedia Controls
• One obvious benefit of hypermedia controls is that it allows the server to
change its URI scheme without breaking clients.
• A further benefit is that it helps client developers explore the protocol.
• The links give client developers a hint as to what may be possible next.
• Similarly it allows the server team to advertise new capabilities by putting
new links in the responses.
• If the client developers are keeping an eye out for unknown links these links can be a
trigger for further exploration.
• There's no absolute standard as to how to represent hypermedia controls.
Source: http://martinfowler.com/articles/richardsonMaturityModel.htmlCopyright © William El Kaim 2015 92
Using Hypermedia-Style Messages
• With a Hypermedia API, the API will use a registered media type such
as HAL or Collection-JSON, providing a common framework for developers
to communicate with the API
• Reducing the unknowns in API integration.
• Two major options for a hypermedia type.
• Use an existing type: XHTML, Collection+JSON, HAL, Siren
• Build your own
• Two paths: Make a custom type, or use the profile link relation.
• If you'd like to make a custom type, read Building Hypermedia APIs in HTML5 and Node.
Building a custom type is just as much art as science.
Copyright © William El Kaim 2015 93
Examples of Hypermedia API
• Amazon AppStream REST API
• Manage applications hosted on Amazon AppStream and to manage client sessions
connecting to those applications.
• FoxyCart
• A hypermedia example from the world of commerce.
• FamilySearch
• Discovering and managing your family history.
• Huddle
• An enteprise example of hypermdia APIs from the content collaboration platform huddle.
• PayPal REST API
• One of the key features of the PayPal REST API is HATEOAS
• VerticalResponse
Source: API evangelistCopyright © William El Kaim 2015 94
Richardson Maturity Model Synthesis
• Level 1 tackles the question of handling complexity by using divide and
conquer, breaking a large service endpoint down into multiple resources.
• Level 2 introduces a standard set of verbs so that we handle similar
situations in the same way, removing unnecessary variation.
• Level 3 introduces discoverability, providing a way of making a protocol more
self-documenting.
Copyright © William El Kaim 2015 95
A new Initiative API Commons: Why?
• APIs are transforming the web in radical new ways are clearly leading a
great deal of innovation and value
• This rapid growth however also brings with potentially huge costs
• namely the need to create client (and server) code for potentially hundreds of thousands
or millions of unique, slightly different APIs.
• While there are some solutions to help reduce this cost (automated code
generation, or more flexible intelligent client code) the unlikely to make much
of a dent in the overall problem in the short and mid term.
• A further problem is that, despite recent copyright victories, the reuse of API
interfaces (the definitions of them / data models / call partners etc.) remain a
legal grey area and reuse of interface patterns may result in legal risks.
Copyright © William El Kaim 2015 96
API Definitions: Explicitly Open And Shareable?
• API Commons proposes to encourage API designers to declare APIs the
produce to be "In the Commons" much the way creative work can be
licensed under Creative Commons or code can be open sourced.
• Initiative from Kin Lane the API Evangelist
• The objective of doing this is to:
• Make it explicit when an API in whole or part could be re-used
• Over time build up a valuable set of reusable API interface resources
• The most popular of which may in turn encourage shared development of shared client
(or server) code
• Remove legal risk from API Interface design reuse
• Dedicated web site: http://apicommons.org/
Copyright © William El Kaim 2015 97
REST client
• Postman
• Quadrillian
• Apigee Console
• RestClient.net
• SoapUI
• Apikitchen
• Hackst
• Tibco ActiveMatrix BusinessWorks V6
Copyright © William El Kaim 2015 98
How to Design a REST API?
http://blog.octo.com/en/design-a-rest-api/Copyright © William El Kaim 2015 99
Antifragile Architecture
Copyright © William El Kaim 2015 100
Antifragile
• In Antifragile, Nassim Taleb discusses the behavior of
complex systems and distinguishes three kinds:
• those that are fragile: Shatters when exposed to even a
small stressor. Unlike risk, fragility is actually measurable!
• those that are robust or resilient: Fragile with a thicker skin,
vulnerable to unanticipated events
• those that are antifragile: when exposed to stress it gets
stronger .
• These types of systems differ in how they respond to
volatility: “The fragile wants tranquility, the antifragile
grows from disorder, and the robust doesn’t care too
much.”
Copyright © William El Kaim 2015 101
Antifragile While fragile systems are easily injured and suffer from
volatility, antifragile systems grow stronger in response to volatility. So-called robust systems remain unchanged.
Copyright © William El Kaim 2015 102
Antifragility as an Outgrowth of Agile
Source: PWCCopyright © William El Kaim 2015 103
Antifragile: “Black Swan” Problem
• “Black Swan”• Impossibility of calculating the risks of consequential rare events and predicting their
occurrence.
• Taleb proposes that systems should be built to handle Black Swan events – unpredictable
and irregular events of massive consequence – instead of building systems based on
known, predictable patterns.
• Systems are generally purposely designed to deal with known risks so when
a shock to a system occurs that was not predicted, all hell breaks loose.
Copyright © William El Kaim 2015 104
Antifragile: Event Driven Architecture
• Developing software as an aggregation of events that respond to change in
data or state is not a new trend.
• Cloud lets developers the ability to focus on the projects they are doing and
not the infrastructure.
• And Functional reactive programming takes this to the next level.
• By idealizing continuous event streams and building subscribers to the event streams, it
simplifies the idea of event driven systems.
• For developers, it is about minimizing the moving parts of building large scale event
driven systems.
• A definition of a modern online application is now a collection of services that
persist their state outside itself, run independently on independent
infrastructure, can be scaled horizontally and upgraded to avoid minimum or
no downtime to the end user.
Copyright © William El Kaim 2015 105
Anti-Fragile IT Systems
• For many years, the focus in IT has been on building robust systems that
invested heavily in avoiding failures.
• To accomplish this goal, methodical processes were implemented to guide IT
through a list of known use cases so that systems could try to avoid failing
and have a plan for recovery if a failure did occur.
• This approach often led to long delivery cycles and a high degree of
complexity which in reality, increased the risk of failure and created fragile
systems.
• Fragile systems are those systems that cannot adapt to unpredictable,
volatile, and random events often referred to as “shocks to the system”.
• There is a fundamental difference in designing systems that do not fail
versus designing systems that expect all parts of the system to fail.
Source: Mike KavisCopyright © William El Kaim 2015 106
Anti-Fragile IT Systems
• What should be done?
• Assume everything will fail
• Cause failure to validate resiliency
• Test design assumption by stressing them
• Don’t wait for random failure. Remove its uncertainty by forcing it periodically.
• Microservices are a natural design approach for Antifragile
• Gives you the full power of embracing change as you are able to evolve parts of your
application that change at similar rates, typically microservices, at the rate at which they
need to evolve at.
• Enable clients must respond gracefully to provider failure
• Devops and Aggressive monitoring
• Try to break your IT systems using techniques such as game days and systems
like chaos monkey.
Copyright © William El Kaim 2015 107
Netflix Chaos Monkey
• “One of the first systems our
engineers built in AWS is called
the Chaos Monkey.
• The Chaos Monkey’s job is to
randomly kill instances and
services within our architecture.
• If we aren’t constantly testing our
ability to succeed despite failure,
then it isn’t likely to work when it
matters most – in the event of an
unexpected outage.”http://luckyrobot.com/netflix-chaos-monkey-keeps-movies-streaming/http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html
Copyright © William El Kaim 2015 108
Then, Netflix Simian Army
• Simian Army consists of services (Monkeys)
in the cloud for generating various kinds of
failures, detecting abnormal conditions, and
testing our ability to survive them.
• The goal is to keep our cloud safe, secure,
and highly available. More details can be
found at this blog.
• Currently the simians include Chaos
Monkey, Janitor Monkey, and Conformity
Monkey.
• Refer to the Quick start guide to get started
setting up and using the Monkeys.
https://github.com/Netflix/SimianArmyCopyright © William El Kaim 2015 109
MicroService Architecture
Copyright © William El Kaim 2015 110
Conway’s Law
• “Organizations which design systems ... are constrained to produce designs
which are copies of the communication structures of these organizations”
Melvin Conway
• Siloed functional teams lead to siloed application architectures
• Objective: Do the opposite
• Create cross-functional team organized around capabilities and responsibility
• Fight against the monolith …
Source: Neal Ford, ThoughtWorksCopyright © William El Kaim 2015 111
Software Monolith
• A Software Monolith
• One build and deployment unit
• One code base
• One technology stack (Linux, JVM, Tomcat, Libraries)
• Benefits
• Simple mental model for developers
• one unit of access for coding, building, and deploying
• Simple scaling model for operations
• just run multiple copies behind a load balancer
Source: Andreas SchroederCopyright © William El Kaim 2015 112
Software Monolith Issues
• Huge and intimidating code base for developers
• Development tools get overburdened
• refactorings take minutes
• builds take hours
• testing in continuous integration takes days
• Scaling is limited
• Running a copy of the whole system is resource-intense
• It doesn’t scale with the data volume out-of-the-box
• Deployment frequency is limited
• Re-deploying means halting the whole system
• Re-deployments will fail and increase the perceived risk of deployment
Source: Andreas SchroederCopyright © William El Kaim 2015 113
What is a Microservice Architecture?
• Definitions
• On the logical level, MicroService Architectures (also called MSA) are defined by a
functional system decomposition into manageable and independently deployable
components.
• Loosely coupled service oriented architecture with bounded context.
• The term “micro” refers to the sizing: a microservice must be manageable by
a single development team (5-9 developers)
• Functional system decomposition means vertical slicing (in contrast to
horizontal slicing through layers)
• Independent deployability implies no shared state and inter-process
communication (often via HTTP REST-ish interfaces)
• Enables separation and independent evolution of code base, technology stacks, scaling
and, features.
• Microservice is the first architectural style developed post-Continuous Delivery
Copyright © William El Kaim 2015 114
Principles of Microservice
Source: Andreas Schroeder
Source: PWC
Decentralized Governance: Enterprise
Architects suffer from less pressure to
make the correct choice(s) in microservice
architectures.
Copyright © William El Kaim 2015 115
Monolithic vs. MicroServices Architecture
Source: PWCCopyright © William El Kaim 2015 116
Monolithic vs. MicroServices Architecture
Source: Martin FowlerCopyright © William El Kaim 2015 117
MSA = SnowMan Architecture
From Horizontal to Vertical decomposition
Source: The Snowman architectureCopyright © William El Kaim 2015 118
Monolithic vs. MicroServices
Source: Neal Ford, ThoughtWorks
Small enough to fit in your head
Rewrite over maintain
(10—1000 LOC)-ish / service
Copyright © William El Kaim 2015 119
Examples
• eBay
• 5th generation today
• Monolithic Perl Monolithic C++ Java microservices
• 3rd generation today
• Monolithic Rails JS / Rails / Scala microservices
• Amazon
• Nth generation today
• Monolithic Perl / C++ Java / Scala microservices
Re-architecting is a sign of success!
If you never need to, either you
overbuilt or nobody cares.
Source: Randy SoupCopyright © William El Kaim 2015 120
MSA vs. SOA
Source: PWC
Source: Neal Ford, ThoughtWorks
Copyright © William El Kaim 2015 121
MSA: Deployed in Containers
Source: PWCCopyright © William El Kaim 2015 122
Microservice: Frameworks
• DropWizard (Java)
• Google gRPC: alternative to REST for microservices
• Kite (Go)
• NancyFX (.net and Mono)
• Playframework (Java & Scala)
• Qbit (Java)
• Rodent (Ruby)
• Seneca: micro-services toolkit for Node.js
• ServiceStack (F#)
• Spring Boot (Java)
• Vert.x (Java, JavaScript, CoffeeScript, Ruby, Python or Groovy)
Copyright © William El Kaim 2015 123
Microservice: Platform and Automation
• Platform
• Netflix OSS
• Gilliam - A platform for micro services
• Colossus - framework developed at Tumblr
• Silex: PHP micro-framework
• Deployment automation
• Jenkins, uDeploy, Capistrano, Chief, Puppet or custom scripts.
• Monitoring distributed systems in real-time
• Nexflix Suro, Riemann.io, Sensu, Circonus
• Latency and Fault Tolerance
• Hystrix
• Security
• Open Source Identity & Access Management OSIAM
• VAddy (http://vaddy.net): Continuous web security testing service connected with CI tools.
Copyright © William El Kaim 2015 124
Netflix OSS
Source: Adrian Cockcroft
http://netflix.github.io/
Copyright © William El Kaim 2015 125
Microservice Platforms
Source: Adrian Cockcroft
Twitter Microservice Platform
Gilt Microservice Platform
Copyright © William El Kaim 2015 126
Microservice: Container
• Container
• Apache Karaf : OSGi based runtime providing a lightweight container
• Docker
• Other container tools
• Deis: open source PaaS that makes it easy to deploy and manage applications on your
own servers.
• Packer: tool for creating identical machine images for multiple platforms from a single
source configuration.
• Terraform: common configuration to launch infrastructure
• Google Kubernetes: open source orchestration system for Docker containers
• Container specific OS
• Snappy Ubuntu Core (Snappy)
• RedHat Atomic
Copyright © William El Kaim 2015 127
References: Microservice Introduction
• Martin Fowler Articles
• InfoQ series of articles
• David Morgantini series of Blog Posts
• Microservice Architectures, Dr. Andreas Schroeder
• High Scalability Article
• Microservices: The resurgence of SOA principles and an alternative to the
monolith, PWC
• State of the Art in Microservices, Adrian Cockcroft - Battery Ventures
Copyright © William El Kaim 2015 128
Best Practices & Lessons Learned
• Minimimum Viable Architecture in startup, Randy Soup
• It’s Time to Move to a Four-Tier Application Architecture, Nginx
• Microservice Patterns
• Developing applications with a microservice architecture, Chris Richardson
• Sam Newman’s 14 Tips For Microservices, ThoughtWorks
• Building SaaS enabled architecture, Twelve Factor App
• Building PaaS enabled architecture, ActiveState Blog
• Failing at Microservice by Richard Clayton
• Microservices Reality Check, CapGemini
• Migrating to Microservices and Slides, Adrian Cockcroft
• Microservices - 72 resources, Pawel Pacana
• Microservices - Not A Free Lunch!
• Seven microservices architecture problems and solutions, Eugene Dvorkin
Copyright © William El Kaim 2015 129
Microservice: Case Studies & Books
• Case Studies
• Microservice Architecture, Netflix
• Using Services to Break Down Monoliths, Yelp
• Adopting Microservices at Netflix
• Effectively Monitor Your Micro-Service Architectures, wheretoe.at
• A Journey into Microservices, Hailo
• Scaling Gilt: from Monolithic Ruby Application to Distributed Scala Micro-Services
Architecture, Gilt
• I-Tier: Breaking Up the Monolith, Groupon
• What has Soundcloud learned about Microservices?, Soundcloud
• Books
• “Antifragile Software: Building Adaptable Software with Microservices”, Russ Miles
• Building Microservices: Designing Fine-Grained Systems, Sam Newman
Copyright © William El Kaim 2015 130
3rd Generation Mobile Architecture
Copyright © William El Kaim 2015 131
Copyright © William El Kaim 2015 132
Mobile App Ecosystems
Source: Mobile Megatrends 2014Copyright © William El Kaim 2015 133
Enterprise Mobile Ecosystem Map
Source: KinveyCopyright © William El Kaim 2015 134
3rd Generation Mobile App
• First generation: ‘information appliance’ model.
• Using software, you transformed your phone into a mostly mono-purpose device just like
it said on the tin. Now it’s a phone. Now it’s a calculator. Now it’s a messaging tool.
• The second generation: the ‘home screen’ era
• The prevailing wisdom was that you had to cram everything your service offered into
mobile, using a form of design-driven gavage to stuff your app until it was positively
groaning with tabs and gutters and drawers.
• 3rd Generation : The Age Of Apps As Service Layers
• They aren’t for ‘idle browsing’, they’re purpose built and informed by contextual signals
like hardware sensors, location, history of use and predictive computation.
• These ‘invisible apps’ are less about the way they look or how many features they cram
in and more about maximizing their usefulness to you without monopolizing your
attention
Source: TechCrunchCopyright © William El Kaim 2015 135
The Age Of Apps As Service Layers
Source: GLASSEFFECTCopyright © William El Kaim 2015 136
The Age Of Apps As Service Layers
Source: GLASSEFFECTCopyright © William El Kaim 2015 137
Mobile needs multiple API types
Copyright © William El Kaim 2015 138
The Age Of Apps As Service Layers
http://applinks.org/
Apple shows off iOS app extensibility at WWDC 2014.Extensibility opens the door for developers to hand off data between apps and work with content between apps.
Copyright © William El Kaim 2015 139
Hailo New Google Now Card
• Hailo has teamed up with Google to introduce a Now card that will help make the commute a little bit easier for its customers.
• Instead of poking around in apps and web pages to find what you need, Now cards in the Google app can give you the right information at exactly the right time.
• For people who have opted in to Google Now and have downloaded the Hailo app, the HailoNow card will send an alert to anyone who has booked a journey from outer London zones in to Central London between 7-10am in the morning an offer of a cab home if the passenger is still in the same London location after 5pm.
https://blog.hailoapp.com/2015/02/02/hailo-google-now/Copyright © William El Kaim 2015 140
From AdWords to AppWords
• New technology that lets mobile apps
reach outside of their respective walled
gardens so that users can search and
navigate between specific places within
them.
• Israel’s Deeplink.me launched AppWords,
a mobile search and ad platform that uses
keywords to trigger relevant content
between one app and another.
• Installed Apps will bid for displaying the
ads and link to them.
• Several Taxi Apps can bid for an ad on an
Itinerary
http://deeplink.me/appwordsCopyright © William El Kaim 2015 143
Calendar42: Mobility Attached to Agenda
http://site.calendar42.com/Copyright © William El Kaim 2015 144
“Intelligent Email”
http://www.slidemailapp.com/Copyright © William El Kaim 2015 145
http://www.twitter.com/welkaim
SlideShare
http://www.slideshare.net/welkaim
http://fr.linkedin.com/in/williamelkaim
La Revue Du Digital
http://www.larevuedudigital.com/william-el-kaim/
Copyright © William El Kaim 2015 146