afsug cafe bi - charles de jager

Data CategoriesData Categories

Supports automated processingC f ith d t d l i t d ith d t b d–Conforms with data models associated with databases and spreadsheets

–Granular data stored in fields

Structured

Generally does not support automated processing–No data model or not easily understood–Insufficient metadata–Noisy data communications such as an email message, blog or

document

Unstructured

document

High Volume of small data bits–Huge volumeHuge volume–Only act on exceptions–Captured at source

Event

© 2011 SAP AG. All rights reserved. 1

Common Structured DataCommon Structured Data

© 2011 SAP AG. All rights reserved. 2

Data CategoriesData Categories

Supports automated processingC f ith d t d l i t d ith d t b d–Conforms with data models associated with databases and spreadsheets

–Granular data stored in fields

Structured

Generally does not support automated processing–No data model or not easily understood–Insufficient metadata–Noisy data communications such as an email message, blog or

document

Unstructured

document

High Volume of small data bits–Huge volumeHuge volume–Only act on exceptions–Captured at source

Event

© 2011 SAP AG. All rights reserved. 3

Common Unstructured DataCommon Unstructured Data

A press releaserelease communication

© 2011 SAP AG. All rights reserved. 4

Common Unstructured DataCommon Unstructured Data

Forum postingsp g

© 2011 SAP AG. All rights reserved. 5

Data CategoriesData Categories

Supports automated processingC f ith d t d l i t d ith d t b d–Conforms with data models associated with databases and spreadsheets

–Granular data stored in fields

Structured

Generally does not support automated processing–No data model or not easily understood–Insufficient metadata–Noisy data communications such as an email message, blog or

document

Unstructured

document

High Volume of small data–Huge volumeHuge volume–Only act on exceptions–Captured at source

Event

© 2011 SAP AG. All rights reserved. 6

Common Event DataCommon Event Data

© 2011 SAP AG. All rights reserved. 7

What vs Why and WhenWhat vs. Why and When

It’s generally said that…

structured data tells us “what” and t d t t ll “Wh t” d “Wh ”event data tells “What” and “When”and

unstructured data tells us “why”unstructured data tells us why

© 2011 SAP AG. All rights reserved. 8

KnowledgeS

e

Knowledgetrategy

telli

genc

eExternal

Information

Int

n FIPP P

lan

form

atio

n FI HR

COSDIn

f SDPMMM

© 2011 SAP AG. All rights reserved. 9

Operate / Generates Data

Business Intelligence Typically Runs Off Structured DataBusiness Intelligence Typically Runs Off Structured Data

© 2011 SAP AG. All rights reserved. 10

Business Intelligence Reporting off Structured DataBusiness Intelligence Reporting off Structured Data

How can you extend your BI investments to

t t d d tunstructured and event information?

© 2011 SAP AG. All rights reserved. 11

Do you report just for the sake f ti ?of reporting?

Or do you innovate with intelligence?

Workers Lose Productivity from InadequateInformation Access

54%54%Lose Productivity

© 2011 SAP AG. All rights reserved. 14

Source: Economist, ‘Enterprise Knowledge Workers Study

The Goal: Be a Best Run BusinessThe Goal: Be a Best Run Business

77%

“77% of high77% of high performers haveperformers have above average

23%

above average analyticalycapability”

Low High

© 2011 SAP AG. All rights reserved. 15

Source: Competing on Analytics, Thomas Davenport

LowPerformers

HighPerformers

IT Is Looking for Flexibility in Sharing Relevant Information

Organizations require:

• Trusted, consolidated, and, ,actionable information

• From a variety of dataysources

• Self-service access

© 2011 SAP AG. All rights reserved. 16

http://www.twitterfall.com/

http://archivist.visitmix.com/

Technology is only an enablerBut the power is in the patternsp p

http://maps.linkfluence.net/vc/

How do you visualize your information?

http://www.whatdoestheinternetthink.net/

Information is Beautiful

So what can you do for me?

Text Data Processing DefinedText Data Processing Definedd

Text

1.Extract meaning

Structured Database

ruct

ured

Once structured it can be… Integrated

g2.Transform into structured

data for analysis3 Cleanse and match

Uns

tr QueriedAnalyzedVi li d

3.Cleanse and match

VisualizedReported against

Unlocks Key Information from Text Sources to

© 2011 SAP AG. All rights reserved. 25

Drive Business Insight

Automate Research AnalysisAutomate Research Analysis

Text data processing semantically understands the meaning and context of information, not just the words themselves. Applies linguistic and statistical

techniques to extract entities, concepts and sentiments Discerns facts and relationships that

were previously unprocessable Allows you to deal with information

overload by mining very large corpora of words and making sense of it without having to read every sentencehaving to read every sentence

© 2011 SAP AG. All rights reserved. 26

SAP BusinessObjects Data Services Data integration, data quality, data profiling, and text data processing

ata Business UI

(InformationTechnical UI(Data Services)

SAP BusinessObjects Data Services 4.0ru

ctur

ed D

a (InformationSteward)

U ifi d M t d t

(Data Services)

Str

One Runtime Architecture &

Services

Unified Metadata

ETL

uctu

red Data Quality

Profiling

Uns

tru

Dat

a Text Analytics

One Administration Environment (S h d li S it U M t)

Provides access to all critical business data (regardless of data source, type,

(Scheduling, Security, User Management) One Set of Source/Target Connectors

© 2011 SAP AG. All rights reserved. 27

( g , yp ,or domain) enabling greater business insights and operational effectiveness

Text Data Processing on the Data Services PlatformText Data Processing on the Data Services Platform

Native Text Data Processing on the Data Services platformg pwith the Entity Extraction transform to extract : Predefined entities (like company, person, firm, city, country, …) Sentiment Analysis (e.g. Strong positive, Weak positive,Sentiment Analysis (e.g. Strong positive, Weak positive,

Neutral, Weak Negative, Strong Negative) Custom entities (customized via dictionaries)

Languages supported (for version 4.0) English German French Spanish JapaneseJapa ese Simplified Chinese …

(expanding to 31 languages in next releases)(expanding to 31 languages in next releases)

© 2011 SAP AG. All rights reserved. 28

Supported Entity Types for ExtractionSupported Entity Types for Extraction

Who: people, job title, and national identification numbers

Wh t i i ti fi i l

Where: addresses, cities, states, countries, facilities, internet addresses and phone numbersWhat: companies, organizations, financial

indexes, and productsWhen: dates, days, holidays, months,

addresses, and phone numbersHow much: currencies and units of

measureyears, times, and time periods Generic Concepts: “text data”, “global

piracy”, and so on

Current Languages supported with Data Services 4.0: English, French, German, Simplified Chinese Spanish Japanese (concepts only)Simplified Chinese, Spanish, Japanese (concepts only)

Some of the additional Languages coming: Arabic, Dutch, Farsi, Italian, Korean, Japanese (with concepts), Portuguese, Russian

© 2011 SAP AG. All rights reserved. 29

Pre-defined Extraction of Sentiments, Events, and Relationships

Voice of Customer Public Sector:Voice of CustomerSentiments: strong positive, weak

positive, neutral, weak negative,

Public Sector: Such as person-organization, person-alias, travel events and security

strong negative, problemsRequests: customer requests Enterprise:

M d i iti llMergers and acquisitions, as well as executive job changes

L S t E li h F h L S t E li hLanguage Support: English, French, German, Spanish

Language Support: English, Simplified Chinese

These are starter packs that can be built upon for a specific deployment

© 2011 SAP AG. All rights reserved. 30

ExampleExample

Web Intelligence reports in the BI Launch PadWeb Intelligence reports in the BI Launch Pad

© 2011 SAP AG. All rights reserved. 32

Opened WebI reportOpened WebI report

© 2011 SAP AG. All rights reserved. 33

Searching on “computer”Searching on computer

© 2011 SAP AG. All rights reserved. 34

“Computer” in the Most Mentions Concepts reportComputer in the Most Mentions Concepts report

© 2011 SAP AG. All rights reserved. 35

“Enjoy” stance in the Positive SentimentsEnjoy stance in the Positive Sentiments

© 2011 SAP AG. All rights reserved. 36

“False” and “Issue” stances in the Negative SentimentsFalse and Issue stances in the Negative Sentiments

© 2011 SAP AG. All rights reserved. 37

Drilling down to further understand the complete contextDrilling down to further understand the complete context

© 2011 SAP AG. All rights reserved. 38

The data flow in the Data Services DesignerThe data flow in the Data Services Designer

© 2011 SAP AG. All rights reserved. 39