sap%20 high...

SAP High-Performance AnalyticAppliance 1.0 (SAP HANA)A First Look At The System Architecture

Marc BernardSAP Technology Regional Implementation Group

February 2011

© 2011 SAP AG. All rights reserved. /

Disclaimer

This presentation outlines our general product direction and should not be relied on inmaking a purchase decision. This presentation is not subject to your licenseagreement or any other agreement with SAP.

SAP has no obligation to pursue any course of business outlined in this presentationor to develop or release any functionality mentioned in this presentation. Thispresentation and SAP's strategy and possible future developments are subject tochange and may be changed by SAP at any time for any reason without notice.

This document is provided without a warranty of any kind, either express or implied,including but not limited to, the implied warranties of merchantability, fitness for aparticular purpose, or non-infringement. SAP assumes no responsibility for errors oromissions in this document, except if such damages were caused by SAPintentionally or grossly negligent.


Agenda

1. Architecture Overview2. Row Store3. Column Store4. Persistency Layer5. Modeling6. Q&A


ERP

Architecture OverviewIn-Memory Computing Engine and Surroundings

ERP DB

In-Memory Computing Engine

Clients (planned, e.g.) BI4 Explorer

DashboardDesign

SAP BI4 universes(WebI,...)

Request Processing / Execution Control

MS Excel

BI4 Analysis

SQL Parser MDXSQL Script Calc Engine

TransactionManager

Session Management

Relational EnginesRow Store Column Store

Persistence LayerPage Management Logger

Disk StorageLog VolumesData Volumes

AuthorizationManager

MetadataManager

In-Memory Computing Studio

Administration Modeling

LoadController

ReplicationAgent

ReplicationServer

SAP Business Objects BI4

DataServicesDesigner

SBO BI4servers

( programfor client)

SBO BI4InformationDesign Tool

Other Source Systems

SAPNetWeaver

BW3rd Party

DataServices


ERP

Architecture OverviewThe Engine

LogERP DB

Clients (planned, e.g.) SBOP Explorer 4.0

Xcelsius SAP BI universes (WebI,...)

MS Excel

SBOP Analysis

IMC Studio


LoadController

ReplicationAgent

Business Objects Enterprise


SBO serverprogramsfor clients

SBOInformationDesign Tool


SAPNetWeaver

BW3rd Party

DataServices


Request Processing / Execution ControlSQL Parser MDXSQL Script Calc Engine

TransactionManager

Session Management





MetadataManager

ReplicationServer


ERP

Architecture OverviewLoading Data into SAP HANA

ERP DB



TransactionManager

Session Management





MetadataManager



LoadController

ReplicationAgent

ReplicationServer



SBO BI4servers




SAPNetWeaver

BW3rd Party

DataServices


DashboardDesign


MS Excel

BI4 Analysis


ERP

Architecture OverviewData Modeling

ERP DB



TransactionManager

Session Management





MetadataManager



LoadController

ReplicationAgent

ReplicationServer



SBO BI4servers




SAPNetWeaver

BW3rd Party

DataServices


DashboardDesign


MS Excel

BI4 Analysis


Clients (planned, e.g.)

ERP

Architecture OverviewReporting

ERP DB



TransactionManager

Session Management





MetadataManager



LoadController

ReplicationAgent

ReplicationServer



SBO BI4servers




SAPNetWeaver

BW3rd Party

DataServices

BI4 Explorer

DashboardDesign


MS Excel

BI4 Analysis


ERP

Architecture OverviewAdministration

ERP DB



TransactionManager

Session Management





MetadataManager



LoadController

ReplicationAgent

ReplicationServer



SBO BI4servers




SAPNetWeaver

BW3rd Party

DataServices


DashboardDesign


MS Excel

BI4 Analysis


DB Server

SAP High-Performance Analytic Appliance 1.0

SAP HANA

JDBC ODBC ODBO SQLDBC

SAP In-MemoryComputing Engine

ReplicationServer

SAP In-Memory Computing Studio

SAP BusinessApplication

ReplicationAgent

SAP BusinessObjectsData Services 4.0

Anysource

SAPBusinessObjects

BI 4.0

Repository

SAP BusinessObjects BI clients

SQ

L

MD

X

BIC

S

Auth

entic

atio

nC

onte

nt m

gmt

sync

Adm

in &

mod

el

load (optional)

(optional)

(optional)

(existing)


Request Processing and Execution ControlConceptual View

Standard SQLProcessed directly by DB engine

SQL Script, MDX and planning engineinterface

Domain-specific programminglanguages or modelsConverted into calculation models

Calc EngineCreate logical execution plan forcalculation modelsExecute user defined functions

Relational EngineDB optimizer produces physicalexecuting planAccess to row and column store


Calc Engine for Dummies

The easiest way to think of Calculation Models is to see them as dataflow graphs,where the modeler can define data sources as inputs and different operations (join,aggregation, projection,…) on top of them for data manipulations.

The Calculation Engine will break up a model, for example some SQL Script, intooperations that can be processed in parallel (rule based model optimizer). Then theseoperations will be passed to the database optimizer which will determine the bestplan for accessing row or column stores (algebraic transformations and cost basedoptimizations based on database statistics).


Calc Engine for DummiesExample


Agenda



In-Memory Computing EngineHigh Level Architecture

Row StoreOne of therelational enginesInterfaced fromcalculation /execution layerPure in-memorystore

Persistencemanaged inpersistencelayer

SAP in-memorycomputing engine

HANA


Row Store ArchitectureRow Store Block Diagram

Row Store Block DiagramTransactional Version Memory

Contains temporary versionsNeeded for Multi-VersionConcurrency Control (MVCC)

SegmentsContain the actual data (content ofrow-store tables) in pages

Page ManagerMemory allocationKeeping track of free/used pages

Version Memory ConsolidationThink ‘garbage collector for MVCC’

Persistence LayerInvoked in write operations (log)And in performing savepointscheckpoint writer


Row Store ArchitectureHighlights

Write OperationsMainly go into “Transactional VersionMemory”“INSERT” also writes to PersistedSegment

Read Operations

Write Operations

TransactionalVersionMemory

Main Memory

PersistedSegment

Data thatmay be

seen by allactive

transactions

Recentversions ofchangedrecords

Version MemoryConsolidation

Version ConsolidationMoves “visible version”from Transaction VersionMemory into PersistedSegment (based onCommit ID)Clears “outdated” recordversions from TransactionalVersion Memory

Memory HandlingRow store tables arelinked list of memorypagesPages are grouped insegmentsPage size: 16 KB

Persisted SegmentContains data that may be seen by anyongoing transactionData that has been committed beforeany active transaction was started)


Indexes for Row Store TablesPrimary Index / Row ID / Index Persistence

Each row-store table has a primary indexPrimary index maps ROW ID primary key of table

ROW ID: a number specifying for each record its memory segment and page

How to find the memory page for a table record?A structure called “ROW ID” contains the segment and the page for the recordThe page can then be searched for the records based on primary keyROW ID is part of the primary index of the table

Secondary indexes can be created if needed

Persistence of indexes in row storeIndexes in row store only exist in memory

No persistence of index dataIndex definition stored with table metadataIndexes filled on-the-fly when system loads tables into memory on system start-up


Agenda



In-Memory Computing EngineHigh Level Architecture

Column StoreOne of the relationalenginesInterfaced fromcalculation / executionlayerPure in-memory store

Persistencemanaged inpersistence layer

Optimized for highperformance of readoperationGood performance ofwrite operationsEfficient datacompression

SAP in-memorycomputing engine

HANA


Column Store ArchitectureColumn Store Block Diagram

Column Store Block DiagramOptimizer and Executor

Handles queries andexecution plan

Main and Delta StorageCompressed data for fast readDelta data for fast writeAsynchronous delta merge

Consistent View Manager

Transaction Manager

Persistence Layer


Column StoreHighlights

Storage Separation (Main & Delta)Enables high compression and high writeperformance at the same time

Delta Merge OperationSee next slide

Read Operations

WriteOperations

Main

Main Memory

Delta

Writeoptimized

Compressedand

Readoptimized

Read OperationsAlways have to read from bothmain & delta storages and mergethe results.Engine uses multi versionconcurrency control (MVCC) toensure consistent read operations.

Data Compression in MainStorage

Compression by creatingdictionary and applying furthercompression methodsSpeed up

Data load into CPU cacheEquality check Search

The compression is computedduring delta merge operation.

Write OperationsOnly in delta storage because write optimized.The update is performed by inserting a newentry into the delta storage.


Column StoreDelta Management

Delta Merge OperationPurpose

To move changes in delta storage into the compressed and read optimized main storageCharacteristics

Happens asynchronouslyEven during merge operation the columnar table will be still available for read and writeoperationsTo fulfil this requirement, a second delta and main storage are used internally

Read Operations

WriteOperations

Main

Before Merge

Delta

Read Operations

WriteOperations

MainNew

After Merge

DeltaNew

Read Operations

WriteOperations

Main

During Merge

MainNew

DeltaNewDelta

Merge Operations


Agenda



Persistence LayerPurpose and Scope

Why Does An In-memory Database Need A Persistence Layer?Main Memory is volatile. What happens upon…

Database restart?Power outage?...

Data needs to be stored in a non-volatile wayBackup and restore

SAP in-memory computing engine offers one persistence layer which is used by row store andcolumn store

Regular “savepoints”full persisted image of DB at time of savepoint

Logs capturing all DB transactions since last savepoint (redo logs and undo logs written)restore DB from latest savepoint onwards

Ability to create "snapshots"used for backups


Persistence LayerSystem Restart and Population of In-memory Stores

Actions During System RestartLast savepoint must be restored plus…

Undo logs must be read for uncommitted transactions saved with last savepointRedo logs for committed transactions since last savepoint

Complete content of row store is loaded into memory

Column store tables may be marked for preload or notOnly tables marked for preloadare loaded into memory duringstartupIf table is marked for loadingon demand, the restoreprocedure is invoked on firstaccess


Agenda



Row Store vs. Column StoreWhen to Use Which Store

Modeling Only Possible For Column TablesThis answers the frequently asked question:"Where should I put a table – row store or column store?"

Information Modeler only works with column tablesReplication server creates tables in column store per defaultData Services creates tables in column store per defaultSQL to create column table: "CREATE COLUMN TABLE ..."Store can be changed with "ALTER TABLE …"

System Tables Are Created Where They Fit BestAdministrative tables in row store:

Schema SYS caches, administrative tables of engineTables from statistics server

Administrative tables in column store:Schema _SYS_BI metadata of created views + master data for MDXSchema _SYS_BIC some generated tables for MDXSchema _SYS_REPO e.g. lists of active/modified versions of models


SAP In-Memory Computing StudioLook and Feel

NavigatorView

Quick LaunchView

PropertiesView


SAP In-Memory Computing StudioFeatures

Information Modeler FeaturesModeling

No materialized aggregatesDatabase viewsChoice to publish and consume at 4 levels of modeling

Attribute View, Analytic View, Analytic View enhanced with Attribute View, Calculation View

Data PreviewPhysical tablesInformation Models

Import/ExportModelsData Source schemas (metadata) – mass and selective loadLandscapes

Data Provisioning for SAP Business Applications (both initial load and replication)Analytic Privileges / Security


Modeling Process Flow

Import SourceSystemmetadata• Physical tables

are createddynamically (1:1schema definitionof source systemtables)

ProvisionData• Physical tables

are loaded withcontent.

CreateInformationModels• Database Views

are created• Attribute Views• Analytic Views• Calculation

Views

Deploy• Column views

are created andactivated

Consume• Consume with

choice of clienttools

• BICS, SQL, MDX


SAP In-Memory Computing StudioTerminology

Information Modeler TerminologyData

Attributes – descriptive data (known as Characteristics SAP BW terminology)Measures – data that can be quantified and calculated (known as key figures in SAP BW)

ViewsAttribute Views – i.e. dimensionsAnalytic Views – i.e. cubesCalculation Views – similar to virtual provider with services concept in BW

HierarchiesLeveled – based on multiple attributesParent-child hierarchy

Analytic Privilege – security object


SAP In-Memory Computing StudioNavigator View - Default Catalog

HANA Instance (<USER>)

HANA Server Nameand Instance Number

User Database schema

Schema Content:Column Views,Functions, Tables,Views


SAP In-Memory Computing StudioNavigator View - Information Models

Information Models organizedin Packages

Attribute Views, Analytic Views,Calculation Views, Analytic Privilegesorganised in folders


Attribute Views

Attribute ViewWhat is an Attribute View?

Attributes add context to data.Attributes are modeled using Attribute Views.Can be regarded as Master Data tablesCan be linked to fact tables in Analytical ViewsA measure e.g. weight can be defined as an attribute.

Table Joins and PropertiesJoin Types

leftOuter, rightOuter,fullOuter, textTable

Cardinality1:1N:11:N

Language Column


Analytical View

Analytical ViewAn Analytical View can be regarded as a “cube”.Analytical Views does not store any data. The data is stored in column store or table viewbased on the Analytical View Structure.Attribute and Measures

Can create Attribute FiltersMust have at least one AttributeMust have at least one MeasureCan create Restricted MeasuresCan create Calculated MeasuresCan rename Attribute andMeasures on the property tab


Analytical View

Analytical View: Data PreviewThere are three main views one can select from when previewing data.

Raw Data – table format of dataDistinct Values – graphical and text format identifying unique valuesAnalysis – select fields (attributes and measures) to display in graphical format.


Calculation View (Scripting)

Calculation ViewDefine Table Output StructureWrite SQL Statement.

Ensure that the selected fields corresponds to previously defined Output table structure of the function.Example :SQL_A = SELECT MATNR, KUNNR, …. FROM<COPA_ACTUAL_ANALYTICAL VIEW 1>

SQL_P = SELECT MATTNR_KUNNR, … FROM<COPA_PROJECTED_ANALYTICAL VIEW 2>

TABLE_OUTPUT_STRUCTURE =SELECT * FROM <SQL_A> UNIONSELECT * FROM <SQL_P>;


SAP In-Memory Computing StudioPre-Delivered Administration Console

NavigatorView

PropertiesView

AdministrationView


Agenda



Thank you!


Further Information onSAP HANA and In-Memory Technologies

In-Memory Computinghttp://www.sap.com/platform/in-memory-computing

Real-Real Time Business with HANAhttp://www.youtube.com/watch?v=uUqtUw-m7mQ

SAP Community Network Topic Pagehttp://www.sdn.sap.com/irj/sdn/in-memory

SAP Community Forumhttp://forums.sdn.sap.com/forum.jspa?forumID=491

The SAP NetWeaver BW – SAP HANA Relationshiphttp://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/21575

SAP HANA Ramp-Up Knowledge Transfer (login required)http://service.sap.com/rkt-hana

SAP HANA Documentation (login required during ramp-up)https://cw.sdn.sap.com/cw/community/docupedia/hana

http://www.sap.com/platform/in-memory-computing

http://www.youtube.com/watch?v=uUqtUw-m7mQ

http://www.sdn.sap.com/irj/sdn/in-memory

http://forums.sdn.sap.com/forum.jspa?forumID=491

http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/21575

http://service.sap.com/rkt-hana

https://cw.sdn.sap.com/cw/community/docupedia/hana


No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained hereinmay be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries,eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+,POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex,MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or othercountries.Oracle is a registered trademark of Oracle Corporation.UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.Java is a registered trademark of Sun Microsystems, Inc.JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, Clear Enterprise, SAP BusinessObjects Explorer and other SAP products and services mentioned herein as wellas their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products andservices mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP France in the United States and in other countries.All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only.National product specifications may vary.The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without theexpress prior written permission of SAP AG.This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies,developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/ordevelopment. Please note that this document is subject to change and may be changed by SAP at any time without notice.SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or otheritems contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties ofmerchantability, fitness for a particular purpose, or non-infringement.SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of thesematerials. This limitation shall not apply in cases of intent or gross negligence.The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot linkscontained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages.

© 2011 SAP AG. All Rights Reserved

sap%20 high...

Technology

sap ag

sap hana erp db

party data services

xcelsius sap bi universes

engine logerp db clients

surroundings erp db

course of business

sbop explorer