© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 1Cisco Public© 2013 Cisco and/or its affiliates. All rights reserved. 1

Creating Business Levers with Smart Analytics @ CiscoAaron LanzenSolutions Architect Business Rules and Analyticsalanzen@cisco.comalanzen@hotmail.com

2014 Loras CollegeData Analytics Symposium

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 2

Where Cisco Services started• 1 Retired Apple server

• Exports of Business Objects

• MySQL, PHP, Apache

• 2 people

• 8 years ago

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 3

True Story• Executive Offsite

• Problem: Customer Satisfaction

• Issue: 298 of 500

• Finding:“Customers want better engineers to solve their cases faster”

• Action:Get the highest skilled engineer to the customer ASAP

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 4

The BIG Idea

“We need to find the highest skilled Engineer ASAP”

What do you do 10 minutes after shift starts?

What is highest skilled ?

How can you staff for that?

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 5

Analytics answer many needs

Leadership Needs

• Stay high level

• Broad agreement is goal

• Generalize to stay nimble

• Power Point Solutions

Staff Needs

• Details matter

• Deal with specific objections

• Define with models

• Business Requirements

Analytics => Glue

Not Equal

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 6

Level 1 Analytics• Based on historical data

• Model your business problems

• Provide insight

• Your Foundation

• Decision Trees / Linear models

• Capacity / Handle Time

• Forecasted Work

• High, Medium, Low -> Skill

Vision #1:“Find the highest skilled Engineer ASAP”

How do I define skill classifications?How many high skill engineers do we have?When do they work?How much work can I give them?

Skill Classification

Business Meaning(Percentile of peers)

High >= 85th percentile

Normal Between 40th and 84th

Low Below the 39th

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 7

• Not Expected• Deep stakeholder engagement

• Stakeholders combining

• Conversations are “Fact Based”

• Funded roadmap

Benefits of Level 1

• Expected• Reuse across problems

• Refinement / Clarification

• Ideas shaped by objections

• Leverage by Planning & Ops

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 8

Why Move past Level 1 Analytics?

Visionary Decision

Strategic #2

Strategic#3

Strategic#1

James Taylor’s Slidehttp://www.decisionmanagementsolutions.com

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 9

How do you get to Level 2?

• Cisco picked a rule engine

• RedHat (https://www.jboss.org/drools/)“The Business Logic integration Platform”

• Implementation:Long running Stateful Knowledge Session

Hold state on 10-15 GB of items

Feed in another 15-20 GB of Level 1

• Core Capabilities:Business-Owned Platform

Global Governance

Robust Simulation

Direct Authoring of Business Rules

Flexible Logging of Decisions

Leverage “Ops Data” Within Platform

Stateful Session

Combine Live data & Level 1 Analytics

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 10

Level 2 Analytics• Real time information

• Based on Events

• “Current” value of Level 1

• You may not be making decisions

• Java Classes hold state

• Rules interpret & manage state

• Decisions can leverage this state

• Actions can be taken on change

Vision #1:“Find the highest skilled Engineer ASAP”

Vision #2“Lets ‘prefer’ currently available engineers with a high skill level”

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 11

Take a Safe and Slow approach.

Incredible source of data

Source

Log it React onChange

Hold State

AnalyzeResults

Use in Decision

Lifecycle of an Analytic

Taking live action as this

value changes

Log and Learn.

More logging and analysis

Make sure there is value to this data

Correlate with any

other Analytics

Stream the Events

Be sure of your data and then use in your warehouse

Rules simply see event and store it

in memory

You can make decisions but not follow the path.

Perfect AB testing.

Phased in as slow or fast as you can:

“DO NO HARM”

Stay Safe: Know the impacts before

you take action

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 12

Level 3 Analytics• Build on Level 2

• Strategy applied to Tactical decisions

• Blends ideas or strategies

• Back to higher level ideas

• Champion/Challenger testing

• Collaboration on ideas

• Validation of theories

• Scale the decision makers

Vision # 2:“Lets ‘prefer’ currently available engineers with a high skill level”

Vision # 3“Lets find the right engineer by balancingtheir skill and availability to the work”

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 13

Analytics Ecosystem

• Start safe and small

• Integrate where possible

• Governance is our innovation engine

• Everyone is Accountable

Engineer

Customer

Development Process

Governance

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 14

Level 4 Analytics• Live feedback loops

• System “snaps the line”

• People defineGoals and outcomes

Constraints and tradeoffs

• Live optimization per transaction

• Quantifiable costs on constraints

• Validation of theories

• Scale the decision makers

Vision 3:“Lets find the right engineer by balancingTheir skill and availability with the work”

Actionable Vision #4“Lets maximize the number of times we create a perfect match between an engineer and the work”

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 15

Benefits by group

• Stakeholders

• Projects

• Data Gathering

• Operations

• Less resistance to change

• Trust : Simulation = Production

• Ideas Multiply

• Feel Listened to / Empowered

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 16

Benefits by group

• Stakeholders

• Projects

• Data Gathering

• Operations

• Metrics are Automatic

• Extremely Low Bug Rates

• Business Impacts are Known

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 17

• Stakeholders

• Projects

• Data Gathering

• Operations

Benefits by group

• Rich Data Sets

• Produce Unique Data

• Everyone is Accountable

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 18

• Stakeholders

• Projects

• Data Gathering

• Operations

Benefits by group

• Ties Planning to Execution

• Impact Assessment Easier

• Governance Covers Both

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 19

• Create a vision for a multi-year Analytics program

• Use Analytics as to create common ground

• Start small and safe. Go for Value

• Governance / Communications will become important

• Successful Analytics programs usually become Decision Management programs

Adivce:

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 20

Thank you

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 21

Backup after this

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 22

Data problems

\ Some hints on how you can get here:Challenge 1: You have unfettered access to the raw data long before you will have Smart Data.Cisco faced two major sourcing challenges:Much of the important data is simply not stored anywhere. Our “systems” simply were not created in a way that valued or captured the data they contained. An engineer’s behavior was reduced to a few specific about the cases they took on a given day. (do a decision tree and the model just gives up)Of the data that was captured it suffered from a DWH “interpretation” of what happened vs an accurate record that crisply described what happened. ( subset of fields, snapshots in time, aggregate data only)If you take anything away from this presentation today please understand that the existence of and access to meaningful raw data is the foundation of ANY analytic journey.Cisco Response: Data lifecycle ( send to log, store a bit and log, reference it to decide , react when it changes

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 23

Routing Operations: Business Rules Engine

• Business-Owned Platform• Global Governance• Robust Simulation• Direct Authoring of Business Rules• Flexible Logging of Decisions• Leverage “Ops Data” Within Platform

Core Capabilities

© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 24

Leverage Agile development methodology to quickly iterate and compress design/dev cycle significantly.

Once design/dev is complete, it’s time to simulate various scenarios to understand impacts to customers/ partners, users and finances.

Model

SimulateMeasureResults

Accept Revise Discard

AnalyzeResults

Implement

Simulation Process

Ensure that success

measures are clearly

understood.

Roll out changes to production, either as a

pilot, LA or GA

Desired outcomes achieved?

Key to understanding impacts and ensuring “Do

No Harm”!

Prepare data, events and

scenarios, and then start to

simulate

Leverage test-driven development!

Ideally, you will leverage automated parsing and run comparison to facilitate rapid analysis of outcomes.

What outcomes are observed? Do they align w/expectations, or are further tweaks necessary? Or is it time to go back to the drawing board?

Once desired outcomes are achieved, it’s time to deploy changes to prod. Plan should account for TOI sessions w/stakeholders and roll-back options.

Measure impacts to customers/ partners, users and finances to look for improvement opportunities.

Give leaders ability to say YES based on real data and validated outcomes.