technical report in-memory precision gaming on sap … · 3 sizing sap hana ... for sap hana as the...

32
Technical Report In-Memory Precision Gaming on SAP HANA with Violin Memory Reference Architecture for Installing and Configuring Violin Memory Arrays with SAP HANA as a Precision Gaming Solution Version 1.0 Abstract SAP HANA enables organizations to optimize their business operations by analyzing large amounts of data in real time. It achieves very high performance without requiring any tuning. This document shows you the benefits of Violin Memory Flash Storage Systems in a HANA environment and guides through the setup of the iSCSI components of this solution.

Upload: vankhanh

Post on 15-May-2018

243 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

Technical Report

In-Memory Precision Gaming on SAP HANA with Violin Memory Reference Architecture for Installing and Configuring Violin Memory Arrays with SAP HANA as a Precision Gaming Solution Version 1.0

Abstract

SAP HANA enables organizations to optimize their business operations by analyzing large amounts of data in real time. It achieves very high performance without requiring any tuning. This document shows you the benefits of Violin Memory Flash Storage Systems in a HANA environment and guides through the setup of the iSCSI components of this solution.

Page 2: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

2

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Contents 1 Introduction .................................................................................................................................................... 3 2 SAP HANA at a Glance.................................................................................................................................. 3

2.1 HANA Database Engines ........................................................................................................................ 4 3 Sizing SAP HANA .......................................................................................................................................... 6

3.1 The Concept of T-shirt Sizes for SAP HANA........................................................................................... 6 3.2 Sizing the RAM Needed .......................................................................................................................... 6 3.3 Sizing the Disks ....................................................................................................................................... 7 3.4 Selecting a T-shirt Size ............................................................................................................................ 7 3.5 Data Replication Methods for SAP HANA ............................................................................................... 7 3.6 SAP HANA Failover and High Availability ............................................................................................... 8

4 SAP HANA Solution Overview for Precision Gaming .................................................................................... 8 4.1 Network and Cabling Overview: Rack 1 .................................................................................................. 9 4.2 Network and Cabling Overview – Rack 2 .............................................................................................. 10 4.3 Network Overview.................................................................................................................................. 11 4.4 Host IP Parameter: Internal Connectivity............................................................................................... 11 4.5 Host IP Parameter: iSCSI Connectivity ................................................................................................. 12

5 The Violin Memory 6000 Storage Array ....................................................................................................... 12 5.1 Why Store SAP HANA Data on Violin Arrays? ...................................................................................... 13 5.2 How Applications Benefit from Violin Technology ................................................................................. 14

6 Setting up the Brocade VDX 6720 10Gb Ethernet Switch ........................................................................... 14 6.1 Configuring a Static IP v4 Ethernet Address ......................................................................................... 14 6.2 Configuring the Ethernet Management Interface ................................................................................... 14 6.3 Enabling LACP on a DCB Interface....................................................................................................... 15 6.4 Setting the MTU for a Port-Channel ...................................................................................................... 16 6.5 Setting the MTU for a Single Interface................................................................................................... 16 6.6 Creating a VLAN.................................................................................................................................... 16 6.7 Creating Bonded Interfaces on the Violin Memory Array....................................................................... 17

7 UDEV Setup................................................................................................................................................. 21 8 Multipathing Setup ....................................................................................................................................... 22 9 iSCSI Initiator Configuration......................................................................................................................... 23 10 Conclusion ................................................................................................................................................. 27 Appendix A: VLAN Overview .............................................................................................................................. 28

Page 3: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

3

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Appendix B: Link Aggregation Overview ............................................................................................................ 29

1 Introduction In today’s highly competitive business climate, immediate access to operational data and real-time analytics ultimately determines the success of an organization. Organizations demand proactive insight into business operations in order to preemptively respond to changing market conditions.

However, the volume of operational data available for proactive business insight often exceeds the amount of data that traditional disk-based systems can process within a reasonable time. Organizations struggle to provide immediate access to the detailed data needed to gain true competitive advantage. They are forced to reduce or summarize data from operational applications and external sources to feed their analytic models. The result is inadequate access to the full spectrum of information needed and unacceptable lag times between data capture and the development of meaningful insights.

Real-time analytics and insights can change the way a business operates. Real-time analytics and insights can change the way a business operates, as the following “what-if” scenarios illustrate:

• What if a retail organization had the power to analyze purchase orders as they are placed, managing production demand and inventory more accurately?

• What if an apparel manufacturer had the ability to know which suppliers provide the highest quality, lowest-cost raw materials when reordering?

• What if a utility company was able to combine internal capacity generation metrics with real-time consumption patterns, linking that to key weather data to ensure business, residential service and rates are continuously optimized?

2 SAP HANA at a Glance SAP HANA is the next generation of SAP computing technology. SAP HANA database software is unique because it resides in-memory. It gives organizations the power of instant insight into business. This solution is possible due to advances in hardware technology, a reduction in cost of memory, and an expansion of capacity. SAP has optimized SAP HANA to take advantage of these technology advances, and you can now evaluate this technology as an option for your information and data needs.

SAP HANA is a flexible, data source agnostic toolset (meaning it does not care where the data comes from) that allows you to hold and analyse large (massive) volumes of data in real time, without the need to aggregate or create highly complex physical data models. The SAP HANA in-memory database solution is a combination of hardware and software that optimizes row-based, column-based, and object-based database technologies to exploit parallel processing capabilities.

We want to say the key part again: SAP HANA is a database. The overall solution requires special hardware and includes software and applications, but at its heart, SAP HANA is a database. Figure 2-1 shows the high-level architecture of the SAP HANA appliance.

Page 4: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

4

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

The following sections explain the most important components of this architecture.

2.1 HANA Database Engines The heart of the SAP HANA database is a pair of relational database engines. There are two engines within the SAP HANA database:

1. The column-based store, storing relational data in columns, optimized holding tables with huge amounts of data, which are aggregated and used in analytical operations.

2. The row-based store, storing relational data in rows, as traditional database systems do. This row store is more optimized for write operation and has a lower compression rate, and query performance is much lower compared to the column-based store.

The engine used to store data can be selected on a per-table basis at the time of creation of a table. Tables in the row-store are loaded at startup time, whereas tables in the column-store can be either loaded at startup or on demand, during normal operation of the SAP HANA database. Both engines share a common persistency layer, which provides data persistency consistent across both engines. There is page management and logging, much like in traditional databases. Changes to in-memory database pages are persisted through save points written to the data volumes on persistent storage, which is usually hard drives.

Figure 2-1. SAP HANA Architectural Overview

Page 5: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

5

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Every transaction committed in the SAP HANA database is persisted by the logger of the persistency layer in a log entry written to the log volumes on persistent storage. The log volumes use flash technology storage - like the Violin Memory 6616 Storage System - for high I/O performance and low latency. The relational engines can be accessed through a variety of interfaces. The SAP HANA database supports SQL (JDBC/ODBC), MDX (ODBO), and BICS (SQL DBC).

The calculation engine allows calculations to be performed in the database, without moving the data into the application layer. It also includes a business functions library that can be called by applications to do business calculations close to the data. The SAP HANA-specific SQL Script language has an extension to SQL that can be used to push down data-intensive application logic into the SAP HANA database.

Figure 2-2. SAP HANA Database Engines

Page 6: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

6

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

3 Sizing SAP HANA This section introduces the concept of T-shirt sizes for SAP HANA and gives a short overview of how to size this dataset for an SAP HANA system.

3.1 The Concept of T-shirt Sizes for SAP HANA SAP defined so-called T-shirt sizes for SAP HANA to both simplify the sizing and to limit the number of hardware configurations to support, thus reducing complexity. SAP’s hardware partners provide configurations for SAP HANA according to one or more of these T-shirt sizes. Table 3-1 lists the T-shirt sizes for SAP HANA.

Table 3-1. SAP HANA T-shirt Sizes

SAP T-Shirt Size XS S and S+ M and M+ L

Compressed Data in memory 64GB 128GB 256GB 512GB

Server Main Memory 128GB 256GB 512GB 1024GB

Number of CPUs 2 2 4 8

In addition to the sizes listed above, you might come across the T-shirt size “XL,” which denotes a scale-out configuration for SAP HANA. The T-shirt sizes S+ and M+ denote upgradable versions of the S and M sizes:

• S+ delivers capacity equivalent to S, but the hardware is upgradable to an M size.

• M+ delivers capacity equivalent to M, but the hardware is upgradable to an L size.

These T-shirt sizes are used when relevant growth of the data size is expected.

3.2 Sizing the RAM Needed Sizing an SAP HANA system is mainly based on the amount of data to be loaded into the SAP HANA database, because this determines the amount of main memory (or RAM) needed in an SAP HANA system. To size the RAM, the following steps have to be performed:

• Determine the information that has to be transferred (either by replication or extraction) to the SAP HANA database. Note that typically customers will only select a sub-set of information from their ERP or CRM database, so this has to be done at the table level.

The sizing methodology is based on uncompressed source data size, so in case compression is used in the source database, this has to be taken into account as well. The information required for this step can be acquired with database tools. SAP Note 1514966 contains a script supporting this process for several database systems, for example, DB2 LUW and Oracle. The current size of all the tables (without DB indexes) storing the required information in the source database is denoted as A.

• Although the compression ratio achieved by SAP HANA can vary depending on the data distribution, a working assumption is that, in general, a compression factor of 7 can be achieved:

B = ( A / 7 )

B is the amount of RAM required to store the data in the SAP HANA database.

• Only 50% of the total RAM should be used for the in-memory database. The other 50% is needed for temporary objects (for example, intermediate results), the operating system, and application code:

Page 7: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

7

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

C = B * 2

C is the total amount of RAM required.

The total amount of RAM should be rounded up to the next T-shirt configuration size, as described in “The concept of T-shirt sizes for SAP HANA”, to get the correct T-shirt size needed.

3.3 Sizing the Disks The capacity of the disks is based on the total amount of RAM. There are two types of storage in SAP HANA:

• Diskpersistence – The persistence layer writes snapshots of the database in HANA to disk in regular intervals. These are usually written to an array of SAS drives. The capacity for this storage is calculated based on the total amount of RAM:

Diskpersistence = 4 * C

• Disklog – This contains the database logs, written to Violin Memory flash technology storage arrays. The capacity for this storage is calculated based on the total amount of RAM:

Disklog = 1 * C

3.4 Selecting a T-shirt Size According to the sizing results, select a SAP HANA T-shirt size that satisfies the sizing requirements in terms of main memory, and possibly CPU capabilities. For example, a sizing result of 400 GB for the main memory (C) suggests a T-shirt size of M.

The sizing methodology described above is valid for SAP HANA in a side-car approach. Other use cases might require another sizing methodology, for example, for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which might affect the sizing methodology. Consult SAP documentation regarding other use cases and up-to-date sizing information.

Note: The sizing approach described here is simplified and can only provide a rough idea of the sizing process for the actual sizing for SAP HANA. Consult the SAP sizing documentation for SAP HANA when performing an actual sizing.

3.5 Data Replication Methods for SAP HANA When SAP HANA is used in a side-car data mart approach (that is, not as a database server), data has to be fed from the source system into SAP HANA by replication. There are several replication methods available to accomplish this:

• Log-based replication: This method is based on reading the transaction logs from the source database and re-applying them to the SAP HANA database.

• Trigger-based replication: This method is based on database triggers on the source system to record changes to the source database and replicate these changes to the SAP HANA database.

• ETL-based replication: This method employs an Extract, Transform, and Load (ETL) process to extract data from the data source, transform it to meet the business or technical needs, and then load it into the SAP HANA database.

Figure 3-1 illustrates these replication methods.

Page 8: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

8

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

3.6 SAP HANA Failover and High Availability With SAP HANA, you can use additional hardware to be in a standby mode for failover in case the system experienced issues. You can keep the server running and be ready to take over at any time. However, in this cold standby mode, no SQL is executed on the system, and it may take a few minutes to restore if the primary system goes down.

Currently, you can assign up to three master servers as the nameservers and select one of these as the active server. The active server is assigned a master index server and a statistics server, while those in standby mode have none. If the active master nameserver fails, the system restores itself to any available standby masters and uses this system instead.

The index server keeps tracks of the directories in a volume where the files are located. If the index server fails, the nameserver will automatically try to restart it. This can take up to two to three minutes. If that fails, it looks for any standby index servers. The restore occurs by having the nameserver assign the volume to the standby index server, and after the process is complete, the system becomes available again. Because the statistics server is stored on the nameserver, any failure of the statistics server triggers a nameserver failover. This will create a new statistics server on the new active nameserver, and the old statistics server is removed automatically.

Groups of failover systems can also be defined for each of the hosts to assure that preferred standby servers on the same hardware are used first. This helps restore the servers faster because less data and configurations may need to physically be moved on the network.

Currently the first master server exports all Violin LUNs, using the NFS protocol to the other two master servers for HA. At a later stage of the project, a NFS Proxy will be added. LUNs from the Violin Storage will all be exported to the NFS proxy and exported to the three master servers using the NFS protocol.

4 SAP HANA Solution Overview for Precision Gaming This Section gives you a brief overview of the hardware components used for the Precision Gaming Solution.

Figure 3-1. SAP HANA Replication Methods

Page 9: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

9

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

4.1 Network and Cabling Overview: Rack 1 Figure 4-2 shows the network connections of Rack 1.

Figure 4-1. SAP HANA Precision Gaming Architecture

Figure 4-2. Network and Cabling for Rack 1

Page 10: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

10

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

4.2 Network and Cabling Overview – Rack 2 The following figure shows the network connections of Rack 2.

Figure 4-3. Network and Cabling for Rack 2

Page 11: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

11

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

4.3 Network Overview Figure 4-4 shows an overall view of all network connections used for the SAP HANA solution.

4.4 Host IP Parameter: Internal Connectivity IP Addresses for the control network were configured as shown in the next figure.

Figure 4-4. Network Overview

Figure 4-5. Internal Connectivity

Page 12: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

12

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

4.5 Host IP Parameter: iSCSI Connectivity IP Addresses for the iSCSI storage network were configured as shown in the next figure.

5 The Violin Memory 6000 Storage Array The Violin 6000 Memory Array represents the consolidation of a rack of data center storage equipment into a single 3U platform and provides the benefits of “a rack in a box”. This consolidation is made possible through the ongoing development of flash memory and Violin’s specific flash memory technologies including both aggregation and vRAID.

With no single point of failure, the Violin 6000 Memory Array is suitable for use as mission-critical primary flash storage. By moving from HDD storage to Violin 6000 Memory Arrays, the primary benefits are:

• 80% reduction in power and rack space

• 90% reduction in latency

• 80% reduction in Cost per IOP

• 50x increase in IOPS per shelf

Here is a concise list of the key elements of the Violin 6000 flash Memory Array Platform Architecture and capability along with a supporting graphical description.

• Violin switched memory (VXM) interconnect provides Performance, Scalability, Density, Availability

• vRAID controllers for non-blocking erasure, massively parallel reads/writes, and sustained performance

• Redundant active/active gateways

• Non-disruptive firmware upgrade (2QCY2012)

• Fail-in-place Flash Memory on VIMMs

Figure 4-6. iSCSI Connectivity

Figure 5-1. Violin Memory 6000 Series Array Architecture

Page 13: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

13

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Figure 5-1 provides a photographic illustration of these hardware components.

Additional architectural elements include the following:

• Redundant power supplies

• Passive Memory Plane – Provides interconnection between modules and controllers

• Redundant Power Controllers (PCs) – Power management controllers, modules and fans

• Redundant Array Controllers (ACs) – Provide PCIe interfaces, switching & management services

• Network Interfaces (NIs) – The Network Interfaces connect via PCIe to the Memory Gateways and provide the physical connectivity to the SAN or LAN

All active modules listed above are hot swappable.

5.1 Why Store SAP HANA Data on Violin Arrays? The SAP HANA database resides in memory in a columnar architecture. All read/write activity is occurring in memory. But the system also writes to a traditional hard drive based database, which resides in fact inside the Violin 6616 storage array.

The real purpose of this traditional database is for system restarts. By using the Violin Memory 6000 Series Array, SAP HANA is now able to recover back from Tier 1 storage into the in-memory database much faster compared to a traditional disk based storage system. The main goal is to reduce network latency when serving up data from a storage array. With flash memory, Violin is also getting rid of disk latency.

Page 14: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

14

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

5.2 How Applications Benefit from Violin Technology The Violin 6000 Series is ideal for applications with large amounts of data that require active processing or rapid access. These applications include:

• Transaction processing (e.g. Oracle, DB2, SQL Server, Sybase)

• Data Warehousing and Business Intelligence

• Extract, Transform, Load (ETL) for Data Warehouses

• Real-time Analytics

• System Integration and Test

• Active file storage and metadata for file systems (e.g. GPFS, Lustre)

• Tier 1 storage for High Volume Messaging systems (e.g. Email)

• Software Configuration Management

• Multi-user software development environments

• Multi-user CAD/CAM environments

Each of these applications scales more easily with storage that has both high IOPS and low latency. The increase in storage performance leads to both improved user response times and more efficient use of server, CPU, and software resources.

6 Setting up the Brocade VDX 6720 10Gb Ethernet Switch The following sections guides through the configuration of the Brocade VDX Ethernet switch.

6.1 Configuring a Static IP v4 Ethernet Address 1. Connect to the switch through the serial console.

2. In privileged EXEC mode, issue the configure terminal command to enter global configuration mode.

3. Enter the interface Management rbridge-id/port command to configure the management port.

This command enters a management interface configuration mode where you can choose configuration parameters for IPv4 and IPv6 addresses. A compact switch has a single management port, and the port number for the management port is always 0.

6.2 Configuring the Ethernet Management Interface Follow these instructions to setup the management interface IP Address of the Brocade VDX Ethernet switch.

1. Enter the no ip address dhcp command to disable DHCP.

2. Enter the ip address IPv4_address/prefix_length command.

3. Enter the ip gateway-address IP_address command to configure the gateway address.

4. Verify the configuration with the do show running-config interface Management command.

Example:

<Switchname># configure terminal

Entering configuration mode terminal

<Switchname> (config)# interface Management 1/0

Page 15: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

15

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

<Switchname> (config-Management-1/0)# no ip address dhcp

<Switchname> (config-Management-1/0)# ip address 10.20.192.5/24

<Switchname> (config-Management-1/0)# ip gateway-address 10.20.192.1

<Switchname> (config-Management-1/0)# do show running-config interface Management

interface Management 1/0

no ip address dhcp

ip address 10.20.192.5/24

ip gateway-address 10.20.192.1

no ipv6 address autoconfig

6.3 Enabling LACP on a DCB Interface To add additional interfaces to an existing LAG, repeat this procedure using the same LAG group number for the new interfaces.

To enable LACP on a DCB interface, perform the following steps from privileged EXEC mode:

1. Enter the configure terminal command to access global configuration mode.

2. Enter the interface command to specify the DCB interface type and slot/port number.

<Switchname> (config)# interface tengigabitethernet 4/0/1

3. Enter the no shutdown command to enable the DCB interface.

4. Enter the channel-group command to configure the LACP for the DCB interface.

<Switchname> (conf-if)# channel-group 1 mode active type standard

Repeat steps 2-4 for all Ports that need to be in the same Port-Channel.

In our scenario, we are going to use Switch Ports one to eight to connect our Storage Array. We created two separate Port-Channels, one for Memory Gateway A and a second one for Memory Gateway B.

<Switchname># show port-channel summary

LACP Aggregator: Po 1

Aggregator type: Standard

Ignore-split is enabled

Admin Key: 0001 - Oper Key 0001

Member ports on rbridge-id 4:

Link: Te 4/0/1 (0x418010000) sync: 0

Link: Te 4/0/2 (0x418020001) sync: 0

Link: Te 4/0/3 (0x418030002) sync: 0

Link: Te 4/0/4 (0x418040003) sync: 0

LACP Aggregator: Po 2

Aggregator type: Standard

Ignore-split is enabled

Page 16: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

16

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Admin Key: 0002 - Oper Key 0002

Member ports on rbridge-id 4:

Link: Te 4/0/5 (0x418050004) sync: 0

Link: Te 4/0/6 (0x418060005) sync: 0

Link: Te 4/0/7 (0x418070006) sync: 0

Link: Te 4/0/8 (0x418080007) sync: 0

Note on configuring the MTU on an interface port: The entire fabric acts like a single switch. Therefore, MTU is applicable only on the edge-ports, and not on ISL. To configure the maximum transmission unit (MTU) on an interface port, perform the following steps from privileged EXEC mode.

6.4 Setting the MTU for a Port-Channel Because we set up LACP, the MTU must be set on the Port-Channel. If an Interface does not belong to a Port-Channel, the MTU must then be set on the Interface itself. The Brocade Switch supports a maximum MTU of 9128 bytes.

<Switchname>(config)# interface Port-channel 1

<Switchname>(config-Port-channel-1)# no shutdown

<Switchname>(config-Port-channel-1)# mtu 9128

<Switchname>(config-Port-channel-1)#

In our scenario, we are going to use Switch Ports nine and ten to connect our Server. The next Step describes how to setup these two ports for iSCSI. No Port-Channel is created to connect the server.

6.5 Setting the MTU for a Single Interface <Switchname>(config)# interface TenGigabitEthernet 4/0/9

<Switchname>(conf-if-te-4/0/9)# mtu 9128

<Switchname>(conf-if-te-4/0/9)# exit

<Switchname>(config)# interface TenGigabitEthernet 4/0/10

<Switchname>(conf-if-te-4/0/10)# mtu 9128

<Switchname>(conf-if-te-4/0/10)# do copy running-config startup-config

This operation will modify your startup configuration. Do you want to continue? [y/n]:y

<Switchname>(conf-if-te-4/0/10)#

6.6 Creating a VLAN On Brocade VDX hardware, VLANs are treated as interfaces from a configuration point of view. By default, all the DCB ports are assigned to VLAN 1 (VLAN ID equals 1).

The vlan_ID value can be 1 through 3963. VLAN IDs 3964 through 4090 are internally-reserved VLAN IDs. However, the reserved-vlan command can modify this range. VLANs above 4090 are not configurable. Refer to the Brocade Network OS Command Reference.

To create a VLAN interface, perform the following steps from privileged EXEC mode:

1. Enter the configure terminal command to access global configuration mode.

Page 17: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

17

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

2. Enter the interface vlan command to assign the VLAN interface number.

switch(config)# interface vlan 1

6.7 Creating Bonded Interfaces on the Violin Memory Array First of all, it’s always a good Idea to backup the current configuration before making major changes. This can be done by issuing the following command:

array4-mg-b [array4: master] (config) # configuration write to NAME

We are going to use four Interfaces to create a bond, eth4 to eth7.

Check the settings of the interfaces and what the current MTU size is.

array4-mg-b [array4: master] (config) # show interfaces eth4

Set the MTU values on each of the interfaces, and then create a network bond on the master gateway. This will create an interface called eth0 that is a bond of all iSCSI interfaces on both gateways. This is a global command so it only needs to be run on the master gateway. Note that this will switch the master role to the other gateway, and change the cluster interface to eth0. We will have to restore the cluster interface to eth1 before completing the iSCSI configuration.

array4-mg-b [array4: master] (config) # no network global mtu enable

*array4-mg-a [array4: master] (config) # interface eth4 mtu 9000

*array4-mg-a [array4: master] (config) # interface eth5 mtu 9000

*array4-mg-a [array4: master] (config) # interface eth6 mtu 9000

*array4-mg-a [array4: master] (config) # interface eth7 mtu 9000

*array4-mg-a [array4: master] (config) #write memory

array4-mg-a [array4: master] (config) #slogin mg-b

array4-mg-b [array4: master] (config) # interface eth4 mtu 9000

*array4-mg-b [array4: master] (config) # interface eth5 mtu 9000

*array4-mg-b [array4: master] (config) # interface eth6 mtu 9000

*array4-mg-b [array4: master] (config) # interface eth7 mtu 9000

Note: The next command will terminate your SSH session if you are connected over Ethernet.

Connect now to the MG with a serial connection.

array4-mg-b [array4: master] (config) # network bond eth0 interface eth4 interface eth5

interface eth6 interface eth7 mode link-agg-layer2+3

Are you sure you wish to change networking modes? All cached content will be

lost, the cache modules will leave the cluster to set up networking, and remote

cli connections may be terminated. Continue? [no] yes

*array4-mg-a [array4: master] (config) # write memory

array4-mg-a [array4: master] (config) #

Notice that the cluster interface moved to eth0. We need to move that back to eth1. But before we complete that, we need to assign a new IP address to eth0, which frees up the management address to be re-assigned

Page 18: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

18

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

to eth1. Then we can move the management IP back to eth1, on both gateways. The new IP Address for eth0 is the one we are going to use later for the iSCSI connections to the Storage Array.

array4-mg-a [array4: master] (config) # interface eth0 ip address 192.168.200.1

255.255.255.0

*array4-mg-a [array4: master] (config) #interface eth1 ip address 10.20.192.194

255.255.255.0

*array4-mg-a [array4: master] (config) #cluster master interface eth1

Configure the IP Gateway Address for eth1:

*array4-mg-a [array4: master] (config) #ip route 0.0.0.0 0.0.0.0 10.20.192.1 eth1

*array4-mg-a [array4: master] (config) #write memory

array4-mg-a [array4: master] (config) #exit

Connection to 169.254.1.101 closed

Do the same for MG-B:

array4-mg-b [array4: standby] > enable

array4-mg-b [array4: standby] # configure terminal

array4-mg-b [array4: standby (config) # interface eth0 ip address 192.168.201.1

255.255.255.0

*array4-mg-b [array4: standby] (config) #interface eth1 ip address 10.20.192.195

255.255.255.0

*array4-mg-b [array4: standby] (config) #cluster master interface eth1

*array4-mg-b [array4: standby] (config) #ip route 0.0.0.0 0.0.0.0 10.20.192.1 eth1

*array4-mg-b [array4: standby] (config) #write memory

array4-mg-b [array4: standby] (config) #exit

Connection to 169.254.1.102 closed

At this point, the LACP bond is created, it has an IP address and route, the MTU is set on each port for maximum performance, and the cluster management interface has been restored to what it was originally. The preliminary setup is complete.

Next, enable iSCSI on the Violin array:

array4-mg-b [array4: master] (config) # iscsi enable global

Create the iSCSI targets and bind them to the address created above. Create four iSCSI targets on the master gateway, and then bind them to the IP of eth0 to all targets on each of the gateways. Having four paths per gateway seems to give the best performance.

array4-mg-a [array4: master] (config) # iscsi target create iscsi1

array4-mg-a [array4: master] (config) # iscsi target create iscsi2

*array4-mg-a [array4: master] (config) # iscsi target create iscsi3

*array4-mg-a [array4: master] (config) # iscsi target create iscsi4

array4-mg-a [array4: master] (config) # iscsi target bind iscsi1 to 192.168.200.1

Page 19: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

19

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

*array4-mg-a [array4: master] (config) # iscsi target bind iscsi2 to 192.168. 200.1

*array4-mg-a [array4: master] (config) # iscsi target bind iscsi3 to 192.168. 200.1

*array4-mg-a [array4: master] (config) # iscsi target bind iscsi4 to 192.168. 200.1

*array4-mg-a [array4: master] (config) # write memory

Double check if the MTU on the bonded interface (eth0) is set to 9000, in our case it is set to 1500 and we have to change it to 9000.

array4-mg-a [array4: master] (config) # show interfaces eth0

Interface eth0 status:

Comment:

Admin up: yes

Link up: yes

IP address: 192.168.200.1

Netmask: 255.255.255.0

Speed: N/A

Duplex: N/A

Interface type: ethernet

Interface ifindex: 14

Interface source: bond

MTU: 1500

HW address: 00:13:CC:01:C5:BC

RX bytes: 982080 TX bytes: 981158

RX packets: 7920 TX packets: 7941

RX mcast packets: 7920 TX discards: 0

RX discards: 0 TX errors: 0

RX errors: 0 TX overruns: 0

RX overruns: 0 TX carrier: 0

RX frame: 0 TX collisions: 0

TX queue len: 0

*array4-mg-a [array4: master] (config) # interface eth0 mtu 9000

*array4-mg-a [array4: master] (config) # write memory

array4-mg-a [array4: master] (config) #

Do the same for MG-B:

array4-mg-a [array4: master] (config) # slogin mg-b

array4-mg-b [array4: standby] (config) # iscsi target bind iscsi1 to 192.168.201.1

*array4-mg-b [array4: standby] (config) # iscsi target bind iscsi2 to 192.168.201.1

Page 20: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

20

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

*array4-mg-b [array4: standby] (config) # iscsi target bind iscsi3 to 192.168.201.1

*array4-mg-b [array4: standby] (config) # iscsi target bind iscsi4 to 192.168.201.1

*array4-mg-b [array4: standby] (config) # interface eth0 mtu 9000

*array4-mg-b [array4: standby] (config) # write memory

array4-mg-b [array4: standby] (config) # show iscsi

iSCSI is enabled

iSCSI Target: iscsi1 #Bindings: 1 192.168.201.1 (eth0)

iSCSI Target: iscsi2 #Bindings: 1 192.168.201.1 (eth0)

iSCSI Target: iscsi3 #Bindings: 1 192.168.201.1 (eth0)

iSCSI Target: iscsi4 #Bindings: 1 192.168.201.1 (eth0)

array4-mg-b [array4: standby] (config) #

You can verify the targets and bindings we have just created in the GUI. Therefore, in the top pane of the GUI, click on “Manage” and select “Manage Initiators.”

This can also be done on the command line:

array4-mg-a [array4: master] (config) # show targets

Node Hostname Proto Target State Address

------------------------------------------------------------------------------

4 array4-mg-a iscsi iscsi1 Active iqn.2004-02.com.vmem:array4-mg-a:iscsi1

4 array4-mg-a iscsi iscsi2 Active iqn.2004-02.com.vmem:array4-mg-a:iscsi2

4 array4-mg-a iscsi iscsi3 Active iqn.2004-02.com.vmem:array4-mg-a:iscsi3

4 array4-mg-a iscsi iscsi4 Active iqn.2004-02.com.vmem:array4-mg-a:iscsi4

6 array4-mg-b iscsi iscsi1 Active iqn.2004-02.com.vmem:array4-mg-b:iscsi1

6 array4-mg-b iscsi iscsi2 Active iqn.2004-02.com.vmem:array4-mg-b:iscsi2

Figure 6-1. Verifying Available Targets

Page 21: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

21

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

6 array4-mg-b iscsi iscsi3 Active iqn.2004-02.com.vmem:array4-mg-b:iscsi3

6 array4-mg-b iscsi iscsi4 Active iqn.2004-02.com.vmem:array4-mg-b:iscsi4

array4-mg-a [array4: master] (config) #

The following command creates a 120GB LUN on the Violin Storage Array:

memgate-1605ac [samara-default-cluster: master] (config) # lun create container 41146F00101 name HANA size 120

Note: This 120GB LUN was just used for testing. In the “real” environment” three LUNs are used, each 2,8TB in size. As our client is SUSE SLES and there are two 10Gb interfaces, you will also need to configure bonding on the client side. Otherwise you will run into the same problem where traffic only flows through one interface.

7 UDEV Setup Follow the steps below to set up UDEV on SUSE Linux:

1. Create a new file in /etc/udev/rules.d directory named 12-violin.rules:

# touch /etc/udev/rules.d/12-violin.rules

2. Edit the file /etc/udev/rules.d/12-violin.rules and add the following content:

KERNEL=="sd*[!0-9]|sg*", BUS=="scsi", SYSFS{vendor}=="VIOLIN", RUN+="/bin/sh -c 'echo

noop > /sys/$devpath/queue/scheduler'"

KERNEL=="sd*[!0-9]|sg*", BUS=="scsi", SYSFS{vendor}=="VIOLIN", RUN+="/bin/sh -c 'echo

1024 > /sys/$devpath/queue/nr_requests'"

KERNEL=="sd*[!0-9]|sg*", BUS=="scsi", SYSFS{vendor}=="VIOLIN", RUN+="/bin/sh -c 'echo 1 >

/sys/$devpath/queue/rq_affinity'"

KERNEL=="sd*[!0-9]|sg*", BUS=="scsi", SYSFS{vendor}=="VIOLIN", RUN+="/bin/sh -c 'echo 0 >

/sys/$devpath/queue/rotational'"

KERNEL=="sd*[!0-9]|sg*", BUS=="scsi", SYSFS{vendor}=="VIOLIN", RUN+="/bin/sh -c 'echo 1 >

/sys/$devpath/queue/nomerges'"

KERNEL=="sd*[!0-9]|sg*", BUS=="scsi", SYSFS{vendor}=="VIOLIN", RUN+="/bin/sh -c 'echo

4096 > /sys/$devpath/queue/max_sectors_kb'"

ENV{DM_UUID}=="mpath-SVIOLIN_SAN_ARRAY_?*", RUN+="/bin/sh -c 'echo noop >

/sys/$devpath/queue/scheduler'"

ENV{DM_UUID}=="mpath-SVIOLIN_SAN_ARRAY_?*", RUN+="/bin/sh -c 'echo 1024 >

/sys/$devpath/queue/nr_requests'"

ENV{DM_UUID}=="mpath-SVIOLIN_SAN_ARRAY_?*", RUN+="/bin/sh -c 'echo 1 >

/sys/$devpath/queue/rq_affinity'"

ENV{DM_UUID}=="mpath-SVIOLIN_SAN_ARRAY_?*", RUN+="/bin/sh -c 'echo 0 >

/sys/$devpath/queue/rotational'"

ENV{DM_UUID}=="mpath-SVIOLIN_SAN_ARRAY_?*", RUN+="/bin/sh -c 'echo 1 >

/sys/$devpath/queue/nomerges'"

ENV{DM_UUID}=="mpath-SVIOLIN_SAN_ARRAY_?*", RUN+="/bin/sh -c 'echo 4096 >

/sys/$devpath/queue/max_sectors_kb'"

Set the correct permissions on the file:

server1:~ # chmod 0644 /etc/udev/rules.d/12-violin.rules

Re-load udev rules and process them:

server1:~ # udevadm control --reload-rules

Page 22: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

22

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

server1:~ # udevadm trigger

8 Multipathing Setup The enable multipathing, a Violin-specific section must be created in /etc/multipath.conf so that round robin kicks in. If the file does not yet exist, create it (use the same method as for the udev file) using the code example below as a foundation.

blacklist {

devnode "*"

}

blacklist_exceptions {

devnode "sd*"

}

defaults {

user_friendly_names yes

}

devices {

device {

vendor "VIOLIN"

product "SAN ARRAY"

path_grouping_policy group_by_serial

# Ubuntu getuid_callout "/lib/udev/scsi_id -p 0x80 -g -u --whitelisted --replace-whitespace /dev/%n"

# RHEL5 getuid_callout "/sbin/scsi_id -p 0x80 -g -u -s /block/%n"

# RHEL6 getuid_callout "/sbin/scsi_id --whitelisted --replace-whitespace --page=0x80 --device=/dev/%n"

# SLES getuid_callout "/lib/udev/scsi_id --page=0x83 --whitelisted --replace-whitespace --device=/dev/%n"

# For iSCSI features "1 queue_if_no_path"

path_checker tur

path_selector "round-robin 0"

hardware_handler "0"

failback immediate

rr_weight uniform

no_path_retry fail

rr_min_io 8

}

}

multipaths {

Page 23: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

23

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

multipath {

# Get the wwid with "multipath -ll" and "multipath -v3" to refresh after altering this file.

# wwid list-your-serial-#-and-alias-associations here

wwid 0x70fc037c02048024

alias violin

# ...

}

}

9 iSCSI Initiator Configuration Follow the steps below for iSCSI initiator configuration:

1. Check the installation status of the open-iscsi package. Install the package if it is not already installed:

# rpm –q open-iscsi

2. Check the status of the service:

# service open-iscsi status

Checking for iSCSI initiator service: running

Iscsiadm: No active sessions.

3. Verify the service is enabled for boot:

# chkconfig --list open-iscsi

iscsi 0:off 1:off 2:off 3:on 4:on 5:on 6:off

In this test, configuration of the iSCSI service was already enabled to execute on boot. If the system displays “off” for runlevels 3 and 5, use the following command to enable the service for boot:

# chkconfig open-iscsi on

The Hosts iscsi initiator name is stored in /etc/iscsi/initiatorname.iscsi

server1:/etc/iscsi # cat initiatorname.iscsi

##

## /etc/iscsi/iscsi.initiatorname

##

## Default iSCSI Initiatorname.

##

## DO NOT EDIT OR REMOVE THIS FILE!

## If you remove this file, the iSCSI daemon will not start.

## If you change the InitiatorName, existing access control lists

## may reject this initiator. The InitiatorName must be unique

## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.

Page 24: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

24

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

InitiatorName=iqn.1996-04.de.suse:01:67016efebd3

server1:/etc/iscsi #

We use two 10Gb interfaces to connect to the Storage Array. Figure 9-1 shows that we have to configure them first to use them for iSCSI traffic. Start YAST and navigate to Network Devices->Network Settings to configure the network interfaces:

server1:~ # yast

Configure the IP addresses to match the ones we setup earlier for the bonded eth0 interfaces on the memory gateways. Press ALT plus n on your keyboard to continue and select the second interface. Assign IP Address 192.168.200.10/24 to the first interface, 192.168.201.10/24 to the second one. After setting up the IP Addresses it should look like the image in Figure 9-2.

Press ALT plus O to save the changes. Now we have to export the LUN to the SUSE SLES host. Export the LUN to the IQN we identified from reading the /etc/iscsi/initiatorname.iscsi file.

# lun export container 41146F00101 name HANA to iqn.1996-04.de.suse:01:67016efebd3

Discover the newly added volume from the server:

# iscsiadm -m discovery -t sendtargets –p 192.168.200.1

# iscsiadm -m discovery -t sendtargets –p 192.168.201.1

The paths will show similar to the below as one group in multipath –ll output.

Figure 9-1. Network Settings

Figure 9-2. Example IP Address

Page 25: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

25

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

multipath -ll

mpatha (SVIOLIN_SAN_ARRAY_8BC4FEEDBB1CEC41) dm-0 VIOLIN,SAN ARRAY

size=120G features='0' hwhandler='0' wp=rw

`-+- policy='round-robin 0' prio=1 status=active

|- 3:0:0:6 sdb 8:96 active ready running

|- 5:0:0:6 sdc 8:240 active ready running

|- 3:0:1:6 sdd 65:128 active ready running

|- 7:0:0:6 sde 66:16 active ready running

|- 5:0:1:6 sdf 66:160 active ready running

|- 4:0:0:6 sdg 67:48 active ready running

|- 7:0:1:6 sdh 67:192 active ready running

|- 4:0:1:6 sdi 68:80 active ready running

|- 7:0:3:6 sdj 68:224 active ready running

|- 4:0:3:6 sdk 69:112 active ready running

|- 7:0:4:6 sdl 70:0 active ready running

|- 4:0:4:6 sdm 70:144 active ready running

|- 3:0:2:6 sdn 71:112 active ready running

|- 5:0:2:6 sdo 71:128 active ready running

|- 3:0:3:6 sdp 128:64 active ready running

`- 5:0:3:6 sdq 128:208 active ready running

Create the mount point for the LUN:

server1:~ # mkdir /hana/data

Create a file system, in this example we have used EXT3 to format the LUN:

server1:~ # mke2fs -j /dev/mapper/mpatha

mke2fs 1.41.9 (22-Aug-2009)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

7864320 inodes, 31457280 blocks

1572864 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=4294967296

960 block groups

Page 26: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

26

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,

4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or

180 days, whichever comes first. Use tune2fs -c or -i to override.

Mount the LUN to the directory you created earlier:

server1:~ # mount /dev/mapper/mpatha /hana/data/

server1:~ # df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda2 72G 4.8G 63G 8% /

devtmpfs 48G 296K 48G 1% /dev

tmpfs 48G 6.6M 48G 1% /dev/shm

/dev/mapper/mpatha 119G 188M 112G 1% /hana/data

server1:~ #

Test if you can write data to the LUN. We have used dd - which is not a tool to measure performance to the Array - but it´s perfect to create some data:

server1:~ # dd if=/dev/zero of=/hana/data/testfile.iso bs=4096 count=1000000

1000000+0 records in

1000000+0 records out

4096000000 bytes (4.1 GB) copied, 10.2386 s, 400 MB/s

server1:~ #

Page 27: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

27

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

10 Conclusion SAP HANA is an amazingly fast database platform - it can do many tasks thousands of times faster than other RDBMS platforms like Oracle. However, bad designs run slow on SAP HANA, just like on any other platform.

SAP HANA has opened up new opportunities for companies to compete smarter using the latest advanced business applications and analytics required by employees to make better, timely decisions.

Whether you're a CIO or an IT manager leading the deployment of SAP HANA, you should be aware of the most common challenges other organizations have faced when deploying one of the most exciting new enterprise technologies to come onto the market in the past few years.

Page 28: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

28

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Appendix A: VLAN Overview IEEE 802.1Q Virtual LANs (VLANs) provide the capability to overlay the physical network with multiple virtual networks. VLANs allow you to isolate network traffic between virtual networks and reduce the size of administrative and broadcast domains.

A VLAN contains end stations that have a common set of requirements that are independent of physical location. You can group end stations in a VLAN even if they are not physically located in the same LAN segment. VLANs are typically associated with IP subnetworks and all the end stations in a particular IP subnet belong to the same VLAN. Traffic between VLANs must be routed. VLAN membership is configurable on a per interface basis.

VLAN Configuration Guidelines and Restrictions Follow these guidelines and restrictions when configuring VLANs:

1. In an active topology, MAC addresses can be learned, per VLAN, using Independent VLAN Learning (IVL) only.

2. A MAC address ACL always overrides a static MAC address entry. In this case, the MAC address is the forwarding address and the forwarding entry can be overwritten by the ACL.

3. The Brocade DCB switch supports Ethernet DIX frames and 802.2 LLC SNAP encapsulated frames only.

4. You must configure the same native VLAN on both ends of an 802.1q trunk link. Failure to do so can cause bridging loops and VLAN leaks.

5. All switches in a Brocade VCS Fabric cluster must be configured with the same VLAN number.

Page 29: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

29

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

Appendix B: Link Aggregation Overview Link aggregation allows you to bundle multiple physical Ethernet links to form a single logical trunk providing enhanced performance and redundancy. The aggregated trunk is referred to as a Link Aggregation Group (LAG). The LAG is viewed as a single link by connected devices, the Spanning Tree Protocol, IEEE 802.1Q VLANs, and so on.

When one physical link in the LAG fails, the other links stay up and there is no disruption to traffic.

To configure links to form a LAG, the physical links must be the same speed and all links must go to the same neighboring device. Link aggregation can be done by manually configuring the LAG or by dynamically configuring the LAG using the IEEE 802.3ad Link Aggregation Control Protocol (LACP).

When queuing traffic from multiple input sources to the same output port, all input sources are given the same weight, regardless of whether the input source is a single physical link or a trunk with multiple member links.

The LAG or LAG interface is also referred to as a port-channel.

The benefits of link aggregation are summarized as follows:

1. Increased bandwidth (The logical bandwidth can be dynamically changed as the demand changes.)

2. Increased availability

3. Load sharing

4. Rapid configuration and reconfiguration

The Brocade VDX family of switches supports the following trunk types:

1. Static, standards-based LAG

2. Dynamic, standards-based LAG using LACP

3. Static, Brocade-proprietary LAG

4. Dynamic, Brocade-proprietary LAG using proprietary enhancements to LACP

You can configure a maximum of 24 Link Aggregation Groups (LAGs) with up to 16 links per standard LAG, or four links per Brocade-proprietary LAG. Each LAG is associated with an aggregator.

The aggregator manages the Ethernet frame collection and distribution functions.

On each port, link aggregation control:

1. Maintains configuration information to control port aggregation.

2. Exchanges configuration information with other devices to form LAGs.

3. Attaches ports to and detaches ports from the aggregator when they join or leave a LAG.

4. Enables or disables an aggregator’s frame collection and distribution functions.

Each link in the Brocade VDX hardware can be associated with a LAG; a link cannot be associated with more than one LAG. The process of adding and removing links to and from a LAG is controlled statically, dynamically, or through LACP.

Each LAG consists of the following components:

1. A MAC address that is different from the MAC addresses of the LAG’s individual member links.

Page 30: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

30

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

2. An interface index for each link to identify the link to neighboring devices.

3. An administrative key for each link. Only links having the same administrative key value can be aggregated into a LAG. On each link configured to use LACP, LACP automatically configures an administrative key value equal to the port-channel identification number.

Dynamic Link Aggregation Dynamic link aggregation uses LACP to negotiate which links can be added and removed from a LAG. Typically, two partner systems sharing multiple physical Ethernet links can aggregate a number of those physical links using LACP. LACP creates a LAG on both partner systems and identifies the LAG by the LAG ID. All links with the same administrative key and all links that are connected to the same partner switch become members of the LAG. LACP continuously exchanges LACPDUs to monitor the health of each member link.

Static Link Aggregation In static link aggregation, links are added into a LAG without exchanging LACPDUs between the partner systems. The distribution and collection of frames on static links is determined by the operational status and administrative state of the link.

Brocade-Proprietary Aggregation Brocade-proprietary aggregation is similar to standards-based link aggregation but differs in how the traffic is distributed.

It also has additional rules that member links must meet before they are aggregated:

1. The most important rule requires that there is not a significant difference in the length of the fiber between the member links, and that all member links are part of the same port-group. The Brocade VDX 6720-24 has two port groups; te0/1 to te0/12 and te0/13 to te0/24. The Brocade VDX 6720-60 has six port groups; te0/1 to te0/10, te0/11 to te0/20, and so on.

2. A maximum of four Brocade LAGs can be created per port-group.

LAG Distribution Process The LAG aggregator is associated with the collection and distribution of Ethernet frames. The collection and distribution process is required to guarantee the following:

1. Inserting and capturing control PDUs.

2. Restricting the traffic of a given conversation to a specific link.

3. Load balancing between individual links.

4. Handling dynamic changes in LAG membership.

Note: Interfaces configured as “switchport” interfaces cannot be aggregated into a LAG. However, a LAG can be configured as a switchport.

LACP Overview Link Aggregation Control Protocol (LACP) is an IEEE 802.3ad standards-based protocol that allows two partner systems to dynamically negotiate attributes of physical links between them to form logical trunks. LACP determines whether a link can be aggregated into a LAG. If a link can be aggregated into a LAG, LACP puts the link into the LAG. All links in a LAG inherit the same administrative characteristics.

Page 31: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

31

Reference Architecture: Precision Gaming With Violin and SAP HANA

www.vmem.com

LACP Configuration Guidelines and Restrictions This section applies to standards-based and Brocade-proprietary LAG configurations, except where specifically noted otherwise.

Follow these LACP configuration guidelines and restrictions when configuring LACP:

1. All ports on the Brocade VDX hardware can operate only in full-duplex mode.

2. Brocade-proprietary LAGs only—All LAG member links must be part of the same port-group.

LACP operates in two modes:

1. Passive mode—LACP responds to Link Aggregation Control Protocol Data Units (LACPDUs) initiated by its partner system but does not initiate the LACPDU exchange.

2. Active mode—LACP initiates the LACPDU exchange regardless of whether the partner system sends LACPDUs.

Note: In our scenario, we must set up the Port-Channels in Active Mode. If the LAG ports don´t go to the “in-sync” state on the Brocade switch, try to randomly unplug the cables from the Memory Gateway and plug them back in to a different port on the same Memory Gateway.

Page 32: Technical Report In-Memory Precision Gaming on SAP … · 3 Sizing SAP HANA ... for SAP HANA as the database for an SAP BW system. Also, SAP HANA is constantly being optimized, which

Reference Architecture: Precision Gaming With Violin and SAP HANA

About Violin Memory Violin Memory is pioneering a new class of high-performance flash-based storage systems that are designed to bring storage performance in-line with high-speed applications, servers and networks. Violin Flash Memory Arrays are specifically designed at each level of the system architecture starting with memory and optimized through the array to leverage the inherent capabilities of flash memory and meet the sustained high-performance requirements of business critical applications, virtualized environments and Big Data solutions in enterprise data centers. Specifically designed for sustained performance with high reliability, Violin’s Flash Memory Arrays can scale to hundreds of terabytes and millions of IOPS with low, predictable latency. Founded in 2005, Violin Memory is headquartered in Mountain View, California.

For more information about Violin Memory products, visit www.vmem.com.

© 2013 Violin Memory. All rights reserved. All other trademarks and copyrights are property of their respective owners. Information provided in this paper may be subject to change. For more information, visit www.vmem.com.

vmem-13q1-tr-docname-uslet-en-r0