Download - 151375083-SAP-HANA-Architecture.pdf
SAP HANA Appliance 1.0
RKT for Applications Consultants
© SAP AG 2011
Agenda
SAP HANA Appliance 1.0Lesson 1: Introductions
Lesson 3: Architecture
Lesson 2: Overview
Lesson 4: Data Provisioning
Lesson 5: Modeling
Lesson 6: Reporting
Lesson 7: User Management
Lesson 8: Scenario: CO-PA
Lesson Objectives
© SAP AG 2011
After completing this lesson, you will be able to understand the:Architecture of the SAP HANA Appliance1.0Concept of MVCC - Multi Version Concurrency ControlConcept of Column Store Concept of Row Store Persistance Layer of SAP HANA Appliance1.0Concept of Backup & Recovery
SAP HANA ApplianceERP
Architecture OverviewSAP HANA Appliance and Surroundings
LogERP DB
Clients BI4 Explorer
Dashboard Design
SAP BI4 universes (WebI,...)
MS Excel
BI4 Analysis
SAP HANA Database
Request Processing / Execution ControlSQL Parser MDXSQL Script Calc Engine
Transaction Manager
Session Management
Relational EnginesRow Store Column Store
Persistence LayerPage Management Logger
Disk StorageLog VolumesData Volumes
Authorization Manager
Metadata Manager
SAP HANA Studio
Administration Modeling
Load Controller
Replication Agent
Replication Server
SAP Business Objects BI4
Data Services Designer
SBO BI4 servers
( program for client)
SBO BI4 Information Design Tool
Other Source Systems
SAP NetWeaver
BW3rd Party
Data Services
SLT Add-on
© 2011 SAP AG. All rights reserved. 13Confidential
SAP HANA Appliance – Real Time Replication
Landscape Option 1: (SAP ERP 4.6c or SAP ECC 6.0 on a NW release below NW ABAP 7.02)
For any customers on SAP ERP 4.6c, utilizing SLT will require setting up an intermediary system with at least NW ABAP 7.02 load controller for replication into HANA
For example, a solution manager system could be used for the SLT Add-on
Supports Non-Unicode or MDMP systems for SAP ERP as long as SLT is installed on a NW 7.02 Unicode system
Landscape Option 2: (SAP ECC 6.0+, running on at least NW ABAP 7.02)
For any customers on SAP ECC 6.0+ running on NW ABAP 7.02 (must be fully Unicode), utilizing SLT can be done directly for replication into HANA
© 2011 SAP AG. All rights reserved. 14Confidential
LT Replication Concept: Trigger-Based ApproachArchitecture and Key Building Blocks
SAP HANA systemLT Replication Server
Application Tables
Source system
Write Modules
Controler Modules
DBConnection
RFCConnection
Read Modules
LoggingTables
Application Tables
Efficient initialization of data replication based on DB trigger
and delta logging concept(as with NearZero downtime approach)
Flexible and reliable replication process, incl. data migration
(as used for TDMS and SAP LT)
Fast data replication via DB connectLT replication functionality is fully integrated with HANA Modeler UI
LT replication server does not have to be a separate SAP system and can run on any SAP system with SAP
NetWeaver 7.02 ABAP stack (Kernel 7.20EXT)
DB Trigger
© 2011 SAP AG. All rights reserved. 15Confidential
SAP HANA Appliance – Data Services
SAPERP
Any Source
BWData Load
Metadata
Repository Server
Open Hub
SAP BusinessObjectsData Services 4.0
HANA
Designer and Management
Console
© 2011 SAP AG. All rights reserved. 16Confidential
SAP ECC
SAP HANA as Accelerator for SAP ApplicationsExample: CO-PA Accelerated by SAP HANA
Aggregation Levels
COPA Application
Redundant copy of ECC tables
Data duplicated into SAP HANARead interfaces accesses SAP HANA if available
Aggregation EngineAnalytic indexes
Calculation Engine
DBMS
SAP ECC on legacy DB with secondary DB Connection into HANA
SAP HANA Database
This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
Read Interface
Primary DB connectionRead / write
Secondary DB connectionRead / write
SAP GUI
Drill-down reporting
BI 4.0 (optional)
Analytics
© 2011 SAP AG. All rights reserved. 17Confidential
SAP HANA as Primary DB in Application ServerFirst example: BW 7.30 running on HANA …expected soon
DBMS
SAP NetWeaver BW
Metadata
DW Management Analytic
Engine
Single Data Management for Row- and Column based storage
Integrated engine for data management and in-memory processing of analytical capabilitiesPure DB conversion. No re-implementation required.
Aggregation Engine
Data Store Objects
Analytic indexes
Master Data
InfoCube Index
Calculation Engine
DBMS
SAP NetWeaver BW as a HANA based system (with “built-in” BWA)
HANA
Incl
udes
SA
P N
etW
eave
r B
W A
ccel
erat
or fu
nctio
nalit
y
This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
SLT Replication Concept: Trigger-Based ApproachArchitecture and Key Building Blocks
SAP HANA systemSLT system (NW7.02)
Application Tables
Source system
Write Modules
Controler Modules
DBConnection
RFCConnection
Read Modules
LoggingTables
Application Tables
DB Trigger
Initialization of data replication based on DB trigger
and delta logging concept(as with NearZero downtime
approach)
Flexible and reliable replication process, incl. data conversion
(as used for TDMS and ByDesign) SLT can also be installed on source
system or Solution Manager
Fast data replication via DB connection
(no other interaction)
© 2011 SAP AG. All rights reserved. 19Confidential
HANA 1.0 Appliance ModelSizing etc
© 2011 SAP AG. All rights reserved. 20Confidential
SAP HANA Appliance ModelHow is HANA delivered
SAP HANA only comes as an applianceCollaboration with strong hardware partners
Partners create servers based on certified hardwareNo other hardware supportedProductive Systems: only one instance of SAP HANA per physical serverSAP Customer Messages are single point of entry for all support questionsSAP support collaborates with Partners’ support organizations
What is installed on the Server?Operating System: SuSE Linux Enterprise Server 11
responsibility of Hardware PartnerSAP HANA Database SoftwareAdditional SAP software, e.g. SAP Host Control3rd party programs/agents, e.g. monitoring agents, backup agents, …
for each hardware partner there is a certified set of such software
© 2011 SAP AG. All rights reserved. 21Confidential
HANA Appliance sizing guidelines(Scale Up configurations - Rack Servers)
CPU core to Memory ratio 16 GB / CPU core
Memory requirements Column store: 2 x data storedRow store: 1.25 x data stored1
Storage requirements(Local storage)
Log volume : 1 x total memory (PCIe-Flash / SSD) 2Data volume: 4 x total memory (SAS drives / SSD) 3Stable Queue for SYRS : 100 – 500 GB on Data volume
Network requirementsDedicated 1 GB networks between HANA & ERP , HANA & BI Platform, HANA & data sources Redundant infrastructure for Fail over
1. 25% extra buffer to manage unicode conversion, initial R3 data loads etc..2. Log volumes need to have 100K i/o per second to prevent slow boot up or writing logs 3. Data volumes need to have at a minimum 10K spindles with sequential read speeds of 800 MB - 1 GB / sec
© 2011 SAP AG. All rights reserved. 22Confidential
HANA Appliance sizing guidelines(Scale-out configurations - Clustered Rack Servers)
CPU core to Memory ratio 16 GB / CPU core
Memory requirements Column store: 2 x data storedRow store: 1.25 x data stored1
Storage requirements (External storage)
Log Volume : 1 x total memory (SSD RAID) 2
Data Volume: 4 x total memory ( SAS / FC drives) 3
Stable Queue for SYRS: 10 x hourly data loads
Network requirements10 GB n/w trunked for providing dedicated n/w as above Dedicated 10 GB n/w between HANA nodes & storage Redundant infrastructure for Failover
1. 25% extra buffer to manage unicode conversion, initial R3 data loads etc..2. Log volumes need to have 100K i/o per second to prevent slow boot up or writing logs 3. Data volumes need to have at a minimum 10K spindles with sequential read speeds of 800 MB - 1 GB / sec
© 2011 SAP AG. All rights reserved. 23Confidential
• 2 x 8 core Intel Nehalem EX ( 2 socket system) • 128 GB Main memory • 160 GB PCIe-Flash / SSD for Log volume• 1 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure
• Uncompressed Data ~ 256 GB to ~500 GB
• Replication Data load5GB / hr
• 60,000 USD to 75,000 USD
• 2 x 8 core Intel Nehalem EX ( 2 or 4 sockets system) • 256 GB Main memory • 320 GB PCIe-Flash / SSD for Log volume• 1 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure
• Uncompressed Data ~ 500 GB to ~1.25TB
• Replication Data load5GB / hr
• 85,000 USD to 125,000 USD
• 4 x 8 core Intel Nehalem EX ( 4 or 8 socket system) • 512 GB Main memory • 640 GB PCIe-Flash / SSD • 2 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure
• Uncompressed Data~1.25TB to ~2.5 TB
• Replication Data load5GB - 20 GB/ hr
• 125,000 USD to 200,000 USD
HANA Appliance T-shirt sizes Specifications & approximate data volumes
M
S
XS
© 2011 SAP AG. All rights reserved. 24Confidential
• 8 x 8 core Intel Nehalem EX • 1 TB Main memory • 1.2 TB PCIe-Flash / SSD • 4 TB SAS / SSD for Data volume• 3 x 1 GB n/w or 1 x 10GB n/w (trunk) • Redundant n/w infrastructure
• Uncompressed Data~ 2.5TB to ~5TB
• Replication Data load5GB – 20 GB / hr
• 235,000 USD to 300,000 USD
• Cluster multiple Large Racks • SSD based shared storage for Log Volume• High speed SAS / FC drives based storage for Data volume • 10 GB n/w for BI clients, Data access & Replication• 10 GB n/w between clustered nodes & storage• Redundant n/w infrastructure
• Uncompressed Data> 5 TB
• Replication Data load> 20 GB / hr
• TBD
HANA Appliance T-shirt sizes Specifications & approximate data volumes
XL
Clus
ter e
nv.
L
MVCC: Multi Version Concurrency Control
© SAP AG 2011
Time
T1 Read D
All transactions want to access the same entry with value D
T5 Read ???
T6 Read A
T2 Read D
T3 Read D
T4 Change D -> A
There must be a mechanism which synchronizes theses accesses
Commit
MVCC: Multi Version Concurrency Control
update personalset name = ‘Jones'
where userid = '4711'and name =
'McEwen'
select *from personal
where name like 'M%'
PNR NAME
T1
T2
MVCC transaction level Snapshot
4711 McEwen 5200 Miller
Version n
MVCC statement level Snapshot
commit
select *from personal
where name like 'M%'
PNR NAME
T3
select *from personal
where name like 'M%'
5200 Miller
PNR NAME
select *from personal
where name like 'M%'
PNR NAME
4711 McEwen 5200 Miller
4711 McEwen 5200 Miller
© SAP AG 2011
MVCC: Multi Version Concurrency Control
Transaction ID (TID)
Commit ID (CID)
• identifies the starting point of write transactions• is increased whenever a write transaction starts• is increased when a read transaction is transformed into a write transaction• is used by COLUMN store for version consolidation
the TID is assigned to write transactions as its unique identifier
• reflects the commit sequence of write transactions• is equivalent to a commit timestamp• maximum CID -> last recent Commit ID• is used by ROW store for version consolidation
When a transaction is committed the max. CID is increased and the new value is assigned to the committed transaction
© SAP AG 2011
© SAP AG 2011
Agenda
Lesson 3:
Architecture
Multi Version Concurrency Control MVCC
Row Store
Column Store
Persistence Layer
Backup & Recovery
Row Store ArchitectureRow store block diagram
Row Store Block DiagramTransactional version memory
Contains temporary versionsNeeded for Multi-Version Concurrency Control (MVCC)
SegmentsContain the actual data (content of row-store tables) in pages
Page ManagerMemory allocationKeeping track of free/used pages
Version Memory ConsolidationThink ‘garbage collector for MVCC’
Persistence LayerInvoked in write operations (log)And in performing savepointscheckpoint writer
© SAP AG 2011
Row Store ArchitectureRow store block diagram
Write operationsMainly go into “Transactional Version Memory”“INSERT” also writes to Persisted Segment
Read Operations
Write Operations
Transactional Version Memory
Main Memory
Persisted Segment
Data that may be
seen by all active
transactions
Recent versions of changed records
Version Memory Consolidation
Version ConsolidationMoves “visible version” from Transaction Version Memory into Persisted Segment (based on Commit ID)Clears “outdated” record versions from Transactional Version Memory
Memory handlingRow store tables are linked list of memory pagesPages are grouped in segmentsPage size: 16 kB
Persisted SegmentContains data that may be seen by any ongoing transaction
© SAP AG 2011
Indexes for Row Store Tables
Each row-store table has a primary indexPrimary index maps the ROW ID and the primary key of a tableROW ID:
Each record has an identifier called ROW IDContains the number of the segment and the slot’s offset in this segment
How to find the memory page for a table record?Access on primary index delivers the Row IDThe ROW ID tells about the Segment and the Page offset where the record is located in memory
Secondary indexes can be created if needed
Persistence of indexes in row storeIndexes in row store only exist in memory
No persistence of index dataIndex definition stored with table metadataIndexes filled on-the-fly when system loads tables into memory on system start-up
Primary Index / Row ID / Index Persistence
© SAP AG 2011
© SAP AG 2011
Agenda
Lesson 3:Lesson 3:
Architecture
Multi Version Concurrency Control MVCC
Row Store
Column Store
Persistence Layer
Backup & Recovery
In-Memory Computing Engine (IMCE)Column Store Block Diagram
© SAP AG 2011
column store optimizer & OLAP optimizer • create physical execution plans• column store optimizer is cost based• OLAP Optimizer is rule based
column store execution control• some column store operation can be parallelized
delta storage• exists only in main memory• change operations are stored here
main storage• data structure that contains the main part of the data• efficient compressed data
history storage • contains data versions which are not current• normal queries don’t access history data
Load/Unload agent• loading data from persistency layer• preloaded on startup or loaded on demand
Column Store
Storage separation (Main & Delta)Enables high compression and high write performance at the same time
High lights
Delta merge operationTo move changes in delta storage into the main storage because main storage is read optimized...Happens asynchronously. (Several events to trigger it.)
Read Operations
Write Operations
Main
Main Memory
Delta
Write optimized
Compressed and
Read optimized
Read operationsAlways have to read from both main & delta storages and merge the results.IMCE uses multi version concurrency control (MVCC) to ensure consistent read operations.
Data compression in Main storage
Compression by creating dictionary and applying further compression methodsSpeed up
Data load into CPU cacheEquality check Search
The compression is computed during delta merge operation.
Write operationsOnly in delta storage because write optimized.
The update is performed by inserting a new entry into the delta storage.
© SAP AG 2011
Row Store vs. Column Store
Modeling only possible for column tablesThis answers the frequently asked question:“where should I put a table – row store or column store”
Modeller only works with column tablesReplication server, Data Services and SLT creates tables in column store per defaultSQL to create column table: “CREATE COLUMN TABLE ...”
(HANA) System tables are created where they fit bestAdministrative tables in Row Store:
Schema SYS Caches, administrative tables of IMCE...Tables from statistics server
Administrative tables in column store:Schema _SYS_BI metadata of created views + Master data for MDXSchema _SYS_BIC Some generated tables for MDXSchema _SYS_REPO e.g. Lists of active/modified versions of models.
When to use which store
© SAP AG 2011
© SAP AG 2011
Agenda
Lesson 3:Lesson 3:
Architecture
Multi Version Concurrency Control MVCC
Row Store
Column Store
Persistence Layer
Backup & Recovery
Persistence Layer in In-memory Comp. Engine
Why does an in-memory database need a persistence layerMain Memory is volatile. What happens upon
Database restart?Power outage?...
Data need to be stored in a non-volatile way
Backup and restore
IMCE offers one persistence layer which is used by row store and column storeRegular “savepoints” full persisted image of DB at time of savepointLogs capturing all DB transactions since last savepoint (redo logs and undo logs written)
restore DB from latest savepoint onwardsCreate Snapshots ( backup)
Purpose and Scope
© SAP AG 2011
SAP In-Memory Computing Engine
Persistence Layer in In-memory Comp. Engine
Memory
Data
Persistent Storage
Regular automatic savepoints
Information aboutdata changes
LogVolume
DataVolumes
© SAP AG 2011
Data is saved to disk in intervals
Persistence Layer in In-memory Comp. EngineSavepoint – writing data in IMCE
© SAP AG 2011
DATA&
Undo
DATA&
Undo
Redo Log
Page Buffer
Log queueData Cache Other
Data PagesData pages of virtual files
Savepoint Coordinator
DATA&
UndoDATA
&Undo
4712
4713
Converter
Persistence Layer in In-memory Comp. EngineSystem Restart
© SAP AG 2011
Reboot or Power failure deletes in-memory dataSystem is normally restarted („lazy“ restart to keep downtime short: tables with preload flag +
subsequently requested tables are loaded first)System is restored to the state just before the failure (except non-committed transactions)
Used for recovery:Last data savepointLog between the last data savepoint and the time of failure(contains the data changes of all commited transactions up to that point)
Time
Data savepointto persistent storage
1Log written
to persistent storage(committed transactions)
2Power failure
3
© SAP AG 2011
Agenda
Lesson 3:Lesson 3:
Architecture
Multi Version Concurrency Control MVCC
Row Store
Column Store
Persistence Layer
Backup & Recovery
Backup & Recovery
Data backup:From persistent storage to external backup destinationsUsing database functions (SAP in-memory computing studio)
Log backup:Not supported in SAP HANA 1.0 SPS2Planned for Hana 1.0 SPS3
Configuration backupManual copy of configuation files to external backup destination
DDData Backup
Persistent Storage
LogVolume
DataVolumes
conf
Configuration Backup
Save to External Backup Destinations
© SAP AG 2011
Backup & Recovery
Disk failure (data volumes are damaged)System is restored to the state just before the failure (except non-committed transactions)Used for recovery:
Last data backupLog since the last data backupAssumption: log area undamaged, all log entries still available (not yet overwritten)
Time
Data backupto external backup
destination
1Log written
to persistent storage(committed transactions)
2Disk failure
(data volumes)
3
Recovery scenario – Disk Failure (Data Volume)
© SAP AG 2011
Data Backup
Log Backup
Recovery to last Data Backup
Point in Time Recovery
Backup & Recovery
SAP HANA 1.0
Recovery to status before crash ( )If log is not damaged
Feature Overview
© SAP AG 2011
© SAP AG 2011
Summary
In this lesson, you learned about the:Architecture of HANA 1.0Concept of MVCCConcept of Column Store Concept of Row Store Persistance Layer of HANA 1.0Concept of Backup & Recovery
© SAP AG 2011
Copyright 2011 SAP AGAll rights reserved
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.
The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence.The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages
Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu Informationszwecken. Produkte können länderspezifische Unterschiede aufweisen.
Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen Vereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmten Geschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht die Richtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, weder ausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck sowie für die Gewährleistung der Nichtverletzung geltenden Rechts.SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendung dieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über Internetseiten Dritter ab.Alle Rechte vorbehalten.