sap hana sps0dfg9 - hana scalability

Upload: renee-williams

Post on 03-Feb-2018

257 views

Category:

Documents


1 download

TRANSCRIPT

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    1/19

    2014 SAP AG or an SAP affiliate company. All rights reserved.

    SAP HANA SPS 09 - Whats NewSAP HANA Scalability

    SAP HANA Product Management (Delta from SPS08 to SPS09)

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    2/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Disclaimer

    This presentation outlines our general product direction and should not be rel

    a purchase decision. This presentation is not subject to your license agreemeagreement with SAP.

    SAP has no obligation to pursue any course of business outlined in this prese

    develop or release any functionality mentioned in this presentation. This prese

    SAPs strategy and possible future developments are subject to change and m

    by SAP at any time for any reason without notice.

    This document is provided without a warranty of any kind, either express or im

    but not limited to, the implied warranties of merchantability, fitness for a partic

    non-infringement. SAP assumes no responsibility for errors or omissions in th

    except if such damages were caused by SAP intentionally or grossly negligent

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    3/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Scalability

    Setting the stage

    Scalability is the ability of a computer hardware/software to take full

    advantage of its changed context in a re-scaled situation, e.g. by adding or

    reducing resources

    Vertical Scalability (Scale-up)

    Adding resources within the same computing unit, e.g. #CPUs, #DRAMs

    Horizontal Scalability (Scale-out)

    Adding multiple computing units and making them work as one logical

    computing unit

    SAP HANA is designed for scale-up and scale-out since the beginning

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    4/19

    Scale-up

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    5/19 2014 SAP SE or an SAP affiliate company. All rights reserved.

    Micro-Architecture Trends

    Core1 Core2

    Core4Core3

    CPU 1

    Core1 Core2

    Core4Core3

    CPU 1

    Core1 Core2

    Core4Core3

    CPU 1

    Core1 Core2

    Core4Core3

    CPU 1

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    RAM

    NUMA (Non Uniform MemoryAccess)

    high-speed interconnect

    each socket has its own memory con

    scales well, but there are NUMA-effe

    memory access is non-uniform -> lo

    local

    remote

    remote

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    6/19 2014 SAP SE or an SAP affiliate company. All rights reserved.

    HANA Parallel Query Processing

    multi-user multi-query (multi-plan)

    Join

    Join

    Core 1:

    Query 2: Aggregate

    Aggregate

    Sort

    Sort

    intra-plan-parallelism (multi-pops)

    JoinQuery 3: Aggregate

    intra-p

    Core 2:

    Core 3:

    Core 4:

    Core 5:

    Core 6:

    Query 1:

    Each execution plan is split into multiple plan

    operators (e.g. join, agg, sort) that are

    executed in parallel

    HANA global optimizer generates

    efficient execution plan for each

    query (rule-based, cost-based)

    Execution plans are cached and re-

    used by HANA

    Plan op

    contain

    Job nod

    availabl

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    7/19 2014 SAP SE or an SAP affiliate company. All rights reserved.

    Summary: Vertical Scalability in HANA (scale-up)

    Parallelism in HANA

    HANA is heavily optimized for state-of-the-art HW architectures

    HANA uses massive intra-plan and intra-operator parallelism for query execution

    HANA tries to use all available resources for maximal parallelization

    Challenge: NUMA-effects on large multi-socket systems

    Remark: Virtualization of CPU resources can reinforce NUMA-effects

    Job Scheduler

    Central place in HANA to execute plan-operator jobs

    Re-adjust dynamically the concurrency level avoiding over-parallelization

    Can consider additional .ini configuration data to limit the concurrency level

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    8/19

    Scale-out

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    9/19 2014 SAP SE or an SAP affiliate company. All rights reserved.

    Horizontal Scalability (scale-out)

    General

    Scale-out means expanding to multiple servers rather than a single, bigger server in one database system Multiple servers (nodes) are switched together to one logical, but physically distributed database system

    When to scale-out ?

    Overcome hardware limitations of one single server (memory, cpu)

    Scale-out aims to distribute data and workload

    Attention

    Scale-out requires deep knowledge about data, application and hardware

    Scale-out increases data center operation costs

    Scale-out affects your scaling factor by inter-node communication

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    10/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    HANAs Shared-Nothing Architecture

    Topology

    Each node runs on its own local data (shared-nothing)

    Standby node(s) without own persistence

    Shared file system for node fail-over and recovery (HW partner

    implementation)

    Nodes communicate via internal HDB net protocol

    All nodes belonging to the same HANA system must have the

    same HW setup

    Data

    Volume

    Log

    Volume

    Data

    Volume

    Log

    Volume

    Auto fail-over

    A shared-nothing architecture helps to minimize the inter-node communication in distributed environm

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    11/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Minimize Inter-Node Communication with Data Distribut

    Data model

    Types of data, e.g. master data, transactional data and

    configuration data

    Referential integrity between tables

    Data usage Data interdependencies, e.g. table groups

    Data access paths

    Workload

    Mixed workloads, i.e. read/ write, analytical/transactional

    Update frequency

    Data distribution aims to find the best balance between data model, data usage and actual workload to

    node communication

    Analyze

    Table co-location

    Locate interdependent tables on th

    referential integrity

    Locate tables within the same state

    node, e.g. for collocated joins or tr

    Horizontal Partitioning

    Locate interdependent table partitio

    Table Replication

    Duplicate tables to multiple nodes,

    referenced by many tables

    Distribute

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    12/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Step 1. Classification

    In HANA it is possible to classify a table by additional metadata, like schema_name, group name, group_type

    The table classification can be set with the create / alter table statement

    The table classifications are stored in the table sys.table_groups_

    Step 2. Table Placement Configuration

    For each table classification you can assign a table placement configuration

    A table placement configuration defines, on which node type, disk-volume (SPS09) a table and its partitions shall be located

    A table placement configuration also defines when a table shall be partitioned (works only if the table has a partition specification), e.g.

    min_rows_partitioning and repartitioning_threshold

    The table placement values are inserted via SQL commands into table _sys_rt.table_placement (HANA studio -> SQL-console)

    When is the Table Placement Configuration evaluated

    During create/alter table and table redistribute operations

    The table placement can be overruled with the location_clause, partition_clause or move_clause information in the DDL statement

    HANA Data DistributionTable Placement

    Table Placement is the definition of generic table distribution patterns to control the physical location of

    and how many partitions of a table are to be created in a distributed HANA system

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    13/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    HANA Data DistributionTable Redistribution

    Collects information about landscape and database

    Evaluates table placement information and .ini-parameter

    Apply heuristic algorithm to optimize table placement

    Save execution plan with execution order

    Execute Plan

    1. Generate Plan 2. Execute Pla

    Typical operations

    Review Plan

    Read plan

    Parallel execution of plan items

    Plan is always executed completely

    No automatic rollback, but restore of SYS.REORG_PLAN_

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    14/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Ma

    OL

    tab

    Sla

    OL

    tab

    Sla

    O

    ta

    Stan

    BW workload characteristics

    Mostly OLAP workload in rather static distribution environment

    OLAP load benefits f rom parallel processing on distributed partitions

    Master Node

    Handles OLTP load and DDL statements: ABAP system tables, meta data,

    operational tables and all row tables are stored

    Slave Nodes

    will handle OLAP load exclusively:

    Master data + cubes/DSOs/PSAs are distributed evenly across all slaves

    Cube, DSO and PSA tables are partitioned dependent on the table placement rules

    Useful information

    global.ini [table_placement] method= 2 (keeps row-store tables on the master

    node)

    note #1908075 (BW scale-out configuration recommendations)

    BW ABAP

    App-Server

    Symmetric scale-

    HANA Scale-out SolutionsBW on HANA

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    15/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Master-N

    Table

    group

    Slave-No

    Slave-No

    Standby-N

    Suite ABAP

    App-Server

    Symmetric hardware a

    Table

    group

    Tabl

    group

    HANA Scale-out SolutionsSuite on HANA

    Suite workload characteristics Mixed OLTP/OLAP workloads across all nodes (2PC, Cross-joins)

    Table access patterns dynamic and differ from customer to customer

    Dynamic Table distribution

    Buffered SQL statements in the statement cache are parsed and prioritized

    based on DB statistics

    Determine disjoint table groups

    Table groups are placed on same node while balancing memory and CPU

    Useful information

    note #1899817 (Suite scale-out configuration recommendations)

    Note #1781986 (Suite on distributed HANA database)

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    16/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Takeaways

    Scale-up first !

    Scale-up Most common and easiest way for databases to scale

    No changes in the database or application required

    HANA parallelism strongly supports latest multi-cpu, multi-core HW architectures

    HANA NUMA-aware scheduling and memory allocation is under development

    Scale-out

    Requires deep knowledge about data and workload to achieve good scaling

    Scale-out works good in static distribution environments, like OLAP

    Scale-out in mixed OLAP/OLTP environments can be challenging

    Optimizing scale-out performance is an iterative task

  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    17/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    How to find SAP HANA documentation on this topic?

    In addition to this learning material, you can find SAP HANA

    platform documentation on SAP Help Portal knowledge center athttp://help.sap.com/hana_platform.

    The knowledge centers are structured according to the product

    lifecycle: installation, security, administration, development:SAP HANA Options

    SAP HANA Advanced Data Process

    SAP HANA Dynamic Tiering

    SAP HANA Enterprise Information M

    SAP HANA Predictive

    SAP HANA Real-Time Replication

    SAP HANA Smart Data Streaming

    SAP HANA Spatial

    Documentation sets for SAP HANA option

    http://help.sap.com/hana_options :

    SAP HANA Platform SPS

    Whats New Release Notes

    Installation

    Administration

    Development

    References

    http://help.sap.com/hana_platformhttp://help.sap.com/hana_optionshttp://help.sap.com/hana_optionshttp://help.sap.com/hana_platform
  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    18/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    Thank you

    Contact information

    Ruediger Karl

    SAP HANA Product Management

    [email protected]

    mailto:[email protected]:[email protected]
  • 7/21/2019 Sap Hana Sps0dfg9 - Hana Scalability

    19/19

    2014 SAP SE or an SAP affiliate company. All rights reserved.

    2014 SAP SE or an SAP affiliate company. All rights

    No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an

    SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SA

    company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epxfor additional tradema

    Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

    National product specifications may vary.

    These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of aaffiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate c

    are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be co

    additional warranty.

    In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related

    release any functionality mentioned therein. This document, or any related presentation, and SAP SEs or its affiliated companies strategy adevelopments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliatedany reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, o

    looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Reade

    undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purcha

    http://global12.sap.com/corporate-en/legal/copyright/index.epxhttp://global12.sap.com/corporate-en/legal/copyright/index.epxhttp://global12.sap.com/corporate-en/legal/copyright/index.epxhttp://global12.sap.com/corporate-en/legal/copyright/index.epx