mwoug-whats up asmcoug.us/wp/wp-content/uploads/2019/08/mwoug-whats-up-asm.pdf · •oracle cloud...

@ViscosityNAwww.viscosityna.com

@ViscosityNAviscosityna.com

Viscosity is a team of experts specializing in Oracle Cloud and Engineered Systems.


Nitin Vengurlekar• Chief Architect at Viscosity

– Responsible for Service Delivery

– Focus on Virtualization, Engineered & Converged Systems

– Developed white papers and Best Practices for Application/Database High Availability & Consolidation

• Oracle ACE Director and Vmware vExpert

• Oracle Cloud Architect Certified / AWS Cloud Architect Certified

• Infrastructure and Zero Downtime Systems Migration specialist

– Oracle Applications Migration and Certification Engineer

– Sought after High Availability Specialist

• Database Cloud Architect/Evangelist

• 18 Years with Oracle

– Oracle Support, RAC/ASM Product Management,“Private Database Cloud” Evangelist

3

@dbcloudshifu


We’ve written 18 books on Data, Cloud, and Oracle…

Industry Experts

o Viscosity founders hold 25+ years each in the Oracle space

o Authors of 18 books in the Oracle space

o 3 ACE Directors, only 36 in the United States

o Direct connections to Oracle support and Product Managers support

o Expertise in Oracle 11g, 12c, 18c, ASM, Data Guard, IaaS and PaaS, and much more

o Hire only best of the best


PerformanceHealth ChecksHow’s it running?

AppsSaaS/PaaS, Mobility, Application Development

DBA ServicesRemote and On-site

On-Call Support

ManagedServices

ZERO DOWNTIME

MigrationsProfessionalServicesWhere you need it most

Staff Aug Workforce Capacity on Demand

Oracle License ManagementGet the most out of your Oracle investment

Text “12C” to 444999 for the session slides! 5

6

Viscosity Pillars And Delivery Models

Database

InfrastructureAppsData

Data Integration

IAAS / PAAS

Data WarehousingAnalytics

GoldenGate

SAAS / PAASCX ERP SCM HCM

PAAS

Web Applications

Mobility

EBS / JDE / PS

IAASPublic Cloud

Bare Metal Cloud

AWS Azure Bluemix

Engineered SystemsExa- ZFS @Customer

X86 / Sparc

Workshops Assessments Proof of Concepts

Training Turnkey Projects

Managed Services

Program Agenda

Introduction – How Did We Get Here?

Best Practices That Every Admin Should Know

ASM in Engineered Systems

Q & A

1

2

3

4

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 7

Program Agenda

Introduction – How Did We Get Here?


Best Practices for Engineered Systems

Q & A

1

2

3

4


ASM Architecture - Review

• Automatic Storage Management (ASM) instance• Built on the Oracle instance architecture• Instance that manages the diskgroup metadata• ASM along with ACFS define the Oracle Storage Stack

• Disk Groups • Logical grouping of disks• Determines file mirroring options• Provides an integrated cluster volume manager and file system• Stripes and mirrors files across disks in ASM disk groups• Automatic rebalances after storage configuration changes

• ASM Disks• LUNs presented to ASM

• ASM Files• Files that are stored in ASM disk groups are called ASM files, this includes database files

ASM History 101Before Oracle 12c

Shared Disk Groups

Wide File Striping

1-1 ASM to Server

Diskgroup BDiskgroup A

Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7 Disk 8

ASM Storage Pool

Database Database Database

File 1File 3

File 4File 2

Oracle Cluster

Database Instance


Database Instance

Database Instance

Database Instance

Database Instance

ASM Instance ASM Instance ASM Instance ASM Instance ASM Instance

Disk Group A Disk Group B

Flex ASMIntroduced in Oracle 12c Release 1

• Eliminates requirement for an ASM instance on every server- Database instances connect to local ASM

instance if available or a remote instanceotherwise

-Database instances can failover to a secondary ASM instance

- Administrators specify the cardinality of ASM instances (default is 3)

- Clusterware ensures ASM cardinality is maintained

- Enables online patching of Grid Infrastructure

- Required starting Oracle 12c Release 2

Database Database Database

Database Instance

Database Instance

Database Instance

Database Instance

Database Instance

1 2 3 4 5

ASM Instance

11

ASM Instance ASM Instance

Disk Group-Oriented Storage Management

• Disk groups contain files striped across disks and optionally mirrored

• No distinction between individual databases

• It was a one-size fits approach

Pre-12.2 Disk Group Organization

Disk Group

DB3 : File 1

12

DB2 : File 2 DB1 : File 3

DB3 : File 3

DB2 : File 1

DB1 : File 1

DB1 : File 2

DB2 : File 3DB3 : File 2

DB2 : File 4

Database-Oriented Storage Management (12.2 and forward)

• New disk group type: Flex disk groups• Flex disk groups provide file groups• A file group is the collection of files

belonging to individual databases or PDBs• A file group’s name defaults to the

database or PDB name

Flex Disk Group OrganizationFile Group

Flex Disk Group

DB1 DB2 DB3File 1 File 1 File 1

File 2 File 2 File 2

File 3 File 3 File 3

File 4

File Group


ASMFlex Disk Groups and File Groups

• Flex Disk groups are the foundation to support new features such as File groups

• File group is a group of files that share the same set of properties and characteristics

• This set of properties includes redundancy, rebalance rebuild priority, rebalance power limit, client compatibility, striping, quota group, and the access control list.

• A major benefit is the ability to have a different availability specification for each database that shares the same disk group. File groups are also useful for point-in-time database clones.

ASM File Group Flexibility and Availability

• Flex disk groups enable-Quota Management - limit the space

databases can allocate in a disk group and thereby improve the customers’ ability to consolidate databases into fewer disk groups

- Redundancy Change – utilize lower redundancy for less critical databases

-ASM Database Clones to easily anddynamically create database clonesfor test/dev databases

File 1

File 2

File 3

File 1

File 2

File 3

File 4

File 1

File 2

File 3

Flex Disk Group Quota Management

File 1

File 2

File 3

File 1

File 2

File 3

File 4

Flex Disk Group

DDBB11 DDBB22 DDBB33File 1

File 2

File 3

QQuuoottaaGGrroouuppBBQQuuoottaaGGrroouuppAA

DB3a

15

File 1File 2File 3


• Disk group contains at least one file group, the default file group.• Disk group can contain multiple file groups.• Disk group must have FLEX redundancy to contain a file group.

• File group can belong to only one disk group.• File group can describe only one database, PDB, CDB, volume, or cluster.• File group can belong to only one quota group

• Database can span multiple disk groups with multiple file groups in different disk groups.

CREATE DISKGROUP data FLEX REDUNDANCY DISK '/dev/asmdata1','/dev/asmdata2','/dev/asmdata3' ATTRIBUTE

'compatible.asm' = ‘18.3.0.0.0',

'compatible.rdbms' = ‘18.3.0.0.0',

'compatible.advm' = ‘18.3.0.0.0';

OR

SQL> ALTER DISKGROUP DATA MOUNT RESTRICTED;

SQL> ALTER DISKGROUP DATA CONVERT REDUNDANCY TO FLEX;

Create CDB called NISHAN

$ASMCMD> lsfg

File Group Disk Group Quota Group Used Quota MB Client Name Client Type

DEFAULT_FILEGROUP DATA GENERIC 0

NISHAN_CDB$ROOT DATA GENERIC 4624 NISHAN_CDB$ROOT DATABASE NISHAN_PDB$SEED DATA GENERIC 1552 NISHAN_PDB$SEED DATABASE


CREATE A PDB called OBIWAN. ASM automagically builds an OBIWAN file group.

ASMCMD> lsfg



NISHAN_CDB$ROOT DATA GENERIC 4624 NISHAN_CDB$ROOT DATABASE NISHAN_PDB$SEED DATA GENERIC 1552 NISHAN_PDB$SEED DATABASE

OBIWAN DATA GENERIC 1760 OBIWAN DATABASE

Note, There’s currently no QuotaGroup For Filegroup OBIWAN.

ASMCMD> lsqg

Group_Num Quotagroup_Num Quotagroup_Name Incarnation Used_Quota_MB Quota_Limit_MB

2 1 GENERIC 1 7936 0


ALTER DISKGROUP DATA ADD QUOTAGROUP QGNishan SET quota = 2000m

ALTER DISKGROUP DATA MODIFY FILEGROUP OBIWAN SET 'quota_group' = 'QGNishan’;

$ asmcmd lsfg



NISHAN_CDB$ROOT DATA GENERIC 4624 NISHAN_CDB$ROOT DATABASE

NISHAN_PDB$SEED DATA GENERIC 1552 NISHAN_PDB$SEED DATABASE

OBIWAN DATA QGNISHAN 1760 OBIWAN DATABASE

$ asmcmd lsqg


2 1 GENERIC 1 6176 0

2 3 QGNISHAN 3 1760 2000


ASMCMD> mkqg -G data Clonewars quota 10g

ASMCMD> lsqg


2 1 GENERIC 1 11360 0

2 3 QGNISHAN 3 1760 100

2 4 CLONEWARS 1 0 10240

Create a new non-CDB called Hansolo

ASMCMD> lsfg



HANSOLO DATA GENERIC 5384 HANSOLO DATABASE



OBIWAN DATA QGNISHAN 1760 OBIWAN DATABASE


ASMCMD> mvfg -G data --filegroup HANSOLO CLONEWARS

ASMCMD> lsfg



HANSOLO DATA CLONEWARS 5384 HANSOLO DATABASE



OBIWAN DATA QGNISHAN 1760 OBIWAN DATABASE ASMCMD>

ASMCMD> lsqg


2 1 GENERIC 1 6232 0

2 2 NISHAN 1 0 6144

2 3 QGNISHAN 3 1760 100

2 4 CLONEWARS 1 5384 10240


ASMCMD> lsqg

Group_Num Quotagroup_Num Quotagroup_Name Incarnation Used_Quota_MB Quota_Limit_MB2 1 GENERIC 1 6232 02 2 NISHAN 1 0 61442 3 QGNISHAN 3 1760 1002 4 CLONEWARS 1 5384 10240


ASMCMD> lsfg -G DATA --filegroup obiwan

File Group Disk Group Property Value File Type

OBIWAN DATA PRIORITY MEDIUM

OBIWAN DATA STRIPING COARSE CONTAINER

OBIWAN DATA STRIPING FINE CONTROLFILE

OBIWAN DATA REDUNDANCY MIRROR DATAFILEOBIWAN DATA STRIPING COARSE DATAFILE

OBIWAN DATA REDUNDANCY MIRROR ONLINELOG

OBIWAN DATA STRIPING COARSE ONLINELOG

OBIWAN DATA REDUNDANCY MIRROR ARCHIVELOGOBIWAN DATA STRIPING COARSE ARCHIVELOG

OBIWAN DATA REDUNDANCY MIRROR TEMPFILE

OBIWAN DATA STRIPING COARSE TEMPFILE

….…..

Z


Program Agenda

Introduction – How did we get here?


Best Practices for Engineered Systems

Q & A

1

2

3

4


Support for Pre-12c Databases

Flex ASM cluster supporting pre-12c databases

– Set cardinality to ALL

– ASM instance on every server

– Only 12c or later database instances can reconnect to a surviving ASM instance after an instance failure

DB DB DB

ASM

Server

DB DB DB

Server

DB

ASM

Server

DB DB DB

Server

ASM

Server

DB DB DB

Server

DB DB

DB DB DB

ASM

ASM

ASM

PPrree1122cc

P1o2sct

27

12c

Top ASM Questions

Q. How does the database interact with the ASM instance and how do I make ASM go faster? Are there parameters that will make ASM faster?

A. ASM is not in the I/O path so ASM does not impede the database file access. Since the RDBMS instance is performing raw I/O, the I/O is as fast as possible.

RDBMS and ASM Instance Interaction

Server

Operating System

DATABASE ASM(1) Database opens file

(1A) OPEN

(1B) Extent Map

(2) Database Reads file

(2A) R

EAD

(2B) I/O

Com

pletes(3) Database Creates file

(3A) CREATE(3B

) Allocates file

(3C) Extent Map

(3D) Initializes file (3D) Commit

Deploying Flex ASMMemory and Process Count Considerations for the ASM Instance

• Use Automatic Memory Management (AMM) “MEMORY_TARGET”– Defaults to 1 GB – adequate for most configurations– Exadata uses a custom memory configuration

• Process Count “PROCESSES”– For #_DBs < 10, PROCESSES = 50* #_DBs + 50– For #_DBs >= 10, PROCESSES = 10* #_DBs + 450– Oracle Exadata environment, the MAX(450 + 10* #_DBs, 1024)

Top 10 ASM Questions

Q. Why is the best practices to have 2 diskgroups and What is the best LUN size for ASM

A. There is no best size! In most cases the storage team will dictate based on their standardized LUN size. ASM admin merely has to communicate the ASM Best Practices and application characteristics to storage folks :• Need equally sized / performance LUNs • Minimum of 4 LUNs• The capacity requirement• The workload characteristic (random r/w, sequential r/w) & any response time SLA

Using this info, and their standards, the storage folks should build a nice LUN group set for you.

ASM Disk Group Configuration

33

How many Disk Groups?

• Best practice is DATA, RECO, GRID– DATA disk group for data files and online redo logs– RECO disk group for backups, archive logs, and online redo logs– GRID disk group for OCR, Voting Files, SPFILE (optional) -> Operational Diskgroup

• Separating files into DATA and RECO disk groups allows the database to survive the loss of either disk group

• Separating Grid Infrastructure data (OCR, Voting Files) from DATA/RECO disk groups provides greater management flexibility for these disk groups

• Separate disk groups for different storage tiers, e.g. hard disk or flash

Exadata X8 ASM Disk Group Configuration


• All non-external redundancy disk groups must be balanced– All disks should have same size and similar performance (enforced in 12.1.0.2+)– Same number of disks in each failure group (enforced in 12.2+)

• While not required for external redundancy disk groups, it is a best practice to have disks of similar size and performance

• Minimum disks: 4 times the number of paths for each disk group– Normal redundancy disk group with 2-way multipathing >= 8 disks

• Maximum: < 1000 disks in a disk group– Long disk discovery times and frequent capacity additions with too many disks

ASM Disk Group ConfigurationHow Big and How Many Disks in a Disk Group?

ASM Disk Group Configuration (Non-Engineered Systems)

• Use external redundancy if you have complete confidence in your storage– A small percentage of customer use normal redundancy with high-end storage arrays

• Mirror data across storage arrays• Need to provide quorum failure group

• Always use multi-pathing– Provide ASM the multi-path O/S device path name– Set multi-path timeout to less than clusterware heartbeat timeout

• Advance COMPATIBLE.ASM to current release– Provides access to latest features on availability and performance

Top 10 ASM QuestionsQ. I’m going to do add disks to my ASM diskgroup, how long will this rebalance take?

A. Rebalance time is heavily driven by the three items:• Amount of data currently in the diskgroup• IO bandwidth available on the server• ASM_POWER_LIMIT or Rebalance Power Level

ASM Disk Group Reconfiguration


ASM Power

• Power determines the number of concurrent I/O operations– Highly configuration dependent, but values greater than 32 often have declining

benefit with respect to rebalance performance– Can be dynamically changed to manage performance impact– Use ALTER DISKGROUP REBALANCE MODIFY POWER to change rebalance power

setting without restarting rebalance operation (12.2)– Engineered systems cannot set asm_power_limit to 0 starting 18c

• Power setting can be used for disk resync (disk online) starting 12c• Administrators can now replace a disk as a fast and efficient operation

– Disk group reorganization is not required– Replacement disk is populated with copies of ASM extents from online disks

ASM Disk Group Reconfiguration

41

V$ASM_ESTIMATE provides information for adjusting ASM_POWER_LIMIT and the resulting power of rebalance operations.

The EST_WORK column provides an estimate of the number of allocation units that have to be moved by the rebalance operation to complete.

EXPLAIN WORK FOR ALTER DISKGROUP datadb1 DROP DISK datadb1_0001;

Explained.

SELECT est_work FROM V$ASM_ESTIMATE; EST_WORK

--------

544


Repair Times WhenMirroring - Disk Failure

• Disk and failure group repair timers– disk_repair_time default value is 12 hours (18c)– failgroup_repair_time provides extra opportunity to avoid unnecessary rebalance –

default value is 24 hours– ALTER DISKGROUP <NAME> SET ATTRIBUTE 'DISK_REPAIR_TIME' = ‘24H';– DROP AFTER clause of "OFFLINE DISK"|"OFFLINE DISKS IN FAILGROUP“ can be used to

reset active timer– Timer runs only while disk group mounted– REPAIR_TIMER column in V$ASM_DISK reflects remaining time– When timer expires and the disk is force dropped, you cannot use ONLINE DISK or

REPLACE DISK

Preventing Accidental Corruption

43

• The most common cause of corruption is accidental administrative action made to the wrong disk– Overwriting ASM disk with a File System– Assigning an ASM disk to an LVM

• ASM Filter Driver – prevents overwriting ASM disks• Employ operational procedures that established hard separation

between ASM disks and all others. A few ideas:– Unique ASM disk partitions e.g. /dev/sdu2

• First partition is small partition that aligns second partition to 1 MB boundary• ASM is assigned second partition• NEVER use second partitions elsewhere

Checking for Corruption

44

• Silent data corruption is a fact of life in today’s storage world• The database checks for corruption when reading data

– If physical corruption is detected then automatic recovery can be performed using the ASM mirror copies

– For seldom accessed data, over time all mirror copies of data could be corrupted

• ASM data can be proactively scrubbed:– Scrubbing occurs automatically during rebalance operations– Scrubbing of disk groups, individual files, or individual disks– Check alert log for results– ALTER DISKGROUP <NAME> SCRUB [REPAIR];

What Are the different Snapshots Technologies in the ASM Space

45

• Point-In-Time Database Cloning With Oracle ASM Flex Disk Groups

• Snapshots using ASM Sparse Diskgroups

• Snapshots using ACFS snapclone

• 3rd party Snapclone feature/functionality

Program Agenda



ASM in Engineered Systems

Q & A

1

2

3

4


47

ASM in Exadata

• Storage technology is the foundation for Exadata Database Machine• Each Exadata Storage Cell is an ASM Failgroup• ASM automates disk failure management and remediation

• Exadata Automation Manager (XDMG) initiates automation tasks. • Monitors all configured storage cells for state changes. • Exadata Automation Worker (XDWK) performs automation tasks

requested by XDMG.• Applies to transient and physical disk failures• Pro-Active Disk Quarantine - ASM doesn’t wait for disk_repair_time, drops

the disk and rebalance • When disk is replaced - grid disks are created & rebalance starts

automatically

48

ASM in Exadata

• Disk failure - MS detects failed disk à notifies CELLSRV • If there are griddisks on the failed disk, the CELLSRV notifies ASM about the event. ASM

then drops all griddisks from the corresponding disk groups.• Replacement disk inserted into storage cell - MS detects new disk -> checks cell

configuration file to see if celldisk and griddisks need to be created on it. • If yes, notifies the CELLSRV to do so. Once finished, CELLSRV notifies ASM about new

griddisks and ASM adds them to the corresponding disk groups.• Poorly performing disk - CELLSRV first notifies ASM to offline the disk.

• If possible, ASM then offlines the disk. • Once disk is offlined by ASM, notifies CELLSRV to execute performance • Once completed, CELLSRV will either notify ASM to drop disk (if tests failed) or online (if

tests passed).

49

ASM in Exadata – ASM-CellSrv-XDMG (“the semi-holy trinity”)

Exadata Disk Groups- High or Normal Redundancy

51

• For Exadata, high redundancy is the best practice recommendation for disk groups.

• However,– For test and development workloads utilizing separate disk group, normal redundancy

may be acceptable– When the standby database utilizing Data Guard is deployed, utilizing normal

redundancy may also be acceptable

• Consider using Flex disk groups to provide for different database protection schemes and space consumption management through quota groups

• ASM and ACFS are tightly integrated into ODA architecture.

• Option to select FLEX disk group redundancy during ODA provisioning • Flex disk groups are available with Oracle Database 12.2 or higher

• ODA storage management all managed by odacli dbstorage and disk commands

53

ASM/ACFS in ODA

• ODACLI commands consolidate ASM/ACFS storage utilization information across disk groups, file systems, and databases.

• Provides simple overview of used and available storage information

54

ASM/ACFS in ODA - Streamlined Disk storage reporting

• Disk replacement

• Disk expansion -• Sets of drives in 5 packs • Entire array expansion unit

• Complete automation of disk expansion via OAKD• Creates OS disk devices• Builds /dev/mapper devices and AFD devices• Modifies kernel parameters for optimization and IO scheduling• Adds to [AFD] disks to ASM diskgroups appropriately (splits between DATA

and RECO)

55

ASM/ACFS in ODA – Disk and Storage Management

ASM New Features in ODA

56

• New command: odacli list-dgstorage – shows free space in disk group taking into account space required to restore redundancy

• Even read – improves performance by distributing reads evenly across all disks, not just the primaries (ODA Release 12.2.1.2+)

• ASM Filter Driver (AFD) – protects ASM devices from being accidentally overwritten

• ASM Flex disk group (ODA Release 18.3+)– Client-side storage management via file groups– Capabilities include redundancy change, quota enforcement, and Point-in-time Database Clones

• Rolling Firmware Patching for storage on ODA HA (ODA Release 18.3+)

ODA Best Practices

57

• Don’t break it! ODA is a system optimized for simplicity of operation• Best practices are already implemented as part of ODA deployment• However, a few thoughts…

Summary and Conclusion

58

• ASM12c – 19c is a single integrated solution, designed to manage database files and general-purpose data

• The adoption of ASM and ACFS not only eliminates the need for third party volume managers and file systems, but also simplifies storage management, offering best I/O performance whilst enforcing Oracle best practices.

• Flex ASM removes previous important architecture limitations:• Availability: removing hard dependency between local ASM and database instance,

eliminating single point of failure.

• Performance: Flex ASM reduces the network traffic generated among the ASM instances, leveraging the architecture scalability; and it is easier and faster to keep the ASM metadata synchronized across large clusters.

Program Agenda


Best Practices That Every Admin Should Know Best

Practices for Engineered Systems

ASM and Engineered Systems – Where is it Embedded

Q & A

1

2

3

4

59

Contact Viscosity @ViscosityNAviscosityna.com

Facebook.com/ViscosityNA

Linkedin.com/company/Viscosity-North-America

@ViscosityNA

Viscosity North America

@Viscosity_NA

Text “12C” to 444999 to be sent session slides and updates

Follow Us Online!

60

mwoug-whats up asmcoug.us/wp/wp-content/uploads/2019/08/mwoug-whats-up-asm.pdf · •oracle cloud...

Documents