calpont infinidb administrator guide (for version 1.0.3)

3011 Internet Blvd., Suite 100 : Frisco, Texas 75034 : 214.618.9500 : fax [email protected] : www.calpont.com

Calpont InfiniDBTM

Administrator’s GuideRelease 1.0.3

Document Version 1.0.3-1March 2010

xii

Copyright © 2010 Calpont Corporation. All rights reserved.

InfiniDB and Calpont product names are trademarks of Calpont. References to other companies and their products use trademarks owned by the respective companies and are for reference purposes only.

The information in this document is subject to change without notice. Calpont assumes no responsibility for any inaccuracies in this document.

SOFTWARE LICENSE

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

Table of Contents

Audience ...................................................................................... iDocument standards .................................................................... iList of documentation ................................................................. iiiObtaining documentation ........................................................... iiiDocumentation feedback ........................................................... iiiAdditional resources .................................................................. iii

Chapter 1 Calpont InfiniDB Console Overview 1

Logging on and off the Calpont InfiniDB Console .......................1Calpont InfiniDB Console tips .....................................................2

Help command ......................................................................2Case sensitivity ......................................................................3

Recall command history ...................................................4Command repeat option ...................................................4

Linux and Calpont InfiniDB Console command prompts .......4Complete list of Calpont commands ...........................................5

Chapter 2 System Maintenance 9

Upgrading Calpont InfiniDB ........................................................9Calpont software ....................................................................9Upgrading Calpont InfiniDB ...................................................9

Upgrading Storage ....................................................................10System and Module Operations ...............................................10

Stopping the system or modules .........................................11Starting the system or modules ...........................................11Restarting the system or modules .......................................11Shutting down the system or modules .................................12Altering the System Modules ...............................................12

Table of Contents xiii

Chapter 3 System Status 15

Viewing system status .............................................................. 15Viewing process status ............................................................ 16Viewing network configuration ................................................. 17Viewing module information ..................................................... 18Monitoring module usage and threshholds .............................. 19

Module disk usage ......................................................... 21Memory usage ............................................................... 21Swap space usage ........................................................ 22Disk thresholds .............................................................. 22CPU thresholds .............................................................. 23

Monitoring alarms ..................................................................... 24Viewing active alarms ......................................................... 24Viewing archived alarms ..................................................... 25Viewing top CPU users ....................................................... 28

Monitoring resource usage ........................................... 28

Chapter 4 Configuring System Status Monitoring 31

Management console overview ................................................ 31Calpont configuration files ................................................... 31Linux shell ........................................................................... 31

Configuring system .................................................................. 32Setting server heartbeats .................................................... 34Configuring Disk thresholds ................................................ 34Configuring archived transaction log periods ...................... 35

Configuring modules ................................................................ 36Configuring module CPU thresholds ................................... 36Configuring module disk usage ........................................... 37Configuring swap space usage ........................................... 37

Configuring alarms ................................................................... 38Steps to configure alarms ................................................... 38

Configuring logging .................................................................. 38Enabling and disabling logging ........................................... 40

Chapter 5 Importing Data 41

Import overview ........................................................................ 41

xiv Table of Contents

Estimating load sizes ...........................................................41Import Tasks ........................................................................41Import process .....................................................................41Directory structure ...............................................................42

Copy source files ......................................................................42Source file formats ...............................................................42Source file names ................................................................42Import directory ....................................................................43

Generate job file .......................................................................43Import data ................................................................................44Troubleshooting import .............................................................45

Number of error rows exceeds threshhold ..........................45Data mismatch .....................................................................45

Chapter 6 Transactions 47

Transaction overview ................................................................47Transaction Maintenance .........................................................48ACID Compliance .....................................................................48

Chapter 7 Performing Backup and Recovery 51

Backing up data ........................................................................51Suspending database writes ...............................................52Backing up front-end database ............................................52Backing up Calpont InfiniDB database ................................53Resuming database writes ..................................................53

Recovering from a backup ........................................................53Restoring front-end database ..............................................54Restoring Calpont InfiniDB database ..................................54

Transactions .............................................................................55Saving Calpont Transaction Logs ........................................55Replaying Transaction Logs ................................................55

Dropping the Calpont InfiniDB database ..................................55

Chapter 8 Performance Tuning 57

Query Tuning Guidelines - Data Access Operations ................57Operations used by Calpont InfiniDB include: .....................57

Table of Contents xv

Gathering Performance Metrics - calgetstats() ........................ 58Gathering Performance Metrics ............................................... 59

Using /var/log/Calpont/debug.log ........................................ 59Using calsettrace(1); ........................................................... 59Using /var/log/mysqld.log .................................................... 60

Understanding Extent Map Range Partitioning ........................ 60Scaling ..................................................................................... 61

For performance ................................................................. 61For concurrency .................................................................. 61For data capacity ................................................................ 61

Chapter 9 Configuring Modules 63

Before configuring modules ..................................................... 63Module IDs .......................................................................... 63Automatic configuration ...................................................... 64Manual configuration ........................................................... 64

Adding modules ....................................................................... 64Removing modules .................................................................. 66Reconfiguring Modules ............................................................ 67Starting modules ...................................................................... 68

Chapter 10 Operating Mode 69

Chapter 11 Decimal Scale 71

Enable/Disable Decimal Scale ................................................. 71Set Decimal Scale Level .......................................................... 72

Chapter 12 System Utilities 73

configxml.sh ........................................................................ 73colxml .................................................................................. 74cpimport .............................................................................. 74viewtablelock ....................................................................... 74cleartablelock ...................................................................... 74

xvi Table of Contents

Chapter 13 Troubleshooting 75

UM memory exceeded error / distributed join performance 75Error in TupleAggregateStep ...............................................75Circular join detected error: .................................................75

Appendix A GNU Free Documentation License 79

Table of Contents xvii

xviii Table of Contents

Introduction

This guide includes information for installing, configuring, maintaining, performance tuning, and troubleshooting Calpont InfiniDB.

Some of the content in this guide is only available with the Calpont InfiniDB Enterprise Edition.

Audience

This guide is intended for database administrators and IT administrators who are responsible for setting up and maintaining Calpont InfiniDB.

Document standards

The following typographical conventions and user alerts are used throughout this guide:

Table 1: Typographical Conventions

Item Description

Bold Typeface Characters you type exactly as shown.

For example: Type getLogInfoYou would type: getLogInfo

Bold Italic Typeface

Used as a variable or placeholder. Type the appropriate replacement text. Variables identified with more than one word are connected by underscores.

Examples: Type ID You would type the ID number: 34878Type IP_address You would type the IP address: 110.68.52.01

Introduction i

Table 2: User Alerts

Item Description

Note: Informs you of helpful information.

Warning: Alerts you to possible hardware and/or software failures that can lead to loss of data or corruption.

ii Introduction

List of documentation

The Calpont Database Platform documentation consists of several guides intended for different audiences. The documentation is described in the following table:

Obtaining documentation

These guides reside on our http://www.infinidb.org/ and http://www.calpont.com websites. Contact [email protected] for any additional assistance.

Documentation feedback

We encourage feedback, comments, and suggestions so that we can improve our documentation. Send comments to [email protected] along with the document name, version, comments, and page numbers.

Additional resources

If you need help installing, tuning, or querying your data with Calpont InfiniDB, you can contact [email protected].

Table 3: Documentation

Document Description

Calpont InfiniDB Minimum Recommended Technical Specifications

Lists the minimum recommended hardware and software specifications for implementing Calpont InfiniDB.

Calpont InfiniDB JumpStart Guide

Provides quick setup tasks for installing and configuring software.

Calpont InfiniDB Installation Guide

Contains a summary of steps needed to perform an install of Calpont InfiniDB in a distributed configuration

Calpont InfiniDB SQL Syntax Guide

Provides syntax native to Calpont InfiniDB.

Calpont InfiniDB Concepts Guide Introduction to the Calpont InfiniDB analytic database.

Performance Tuning for the InfiniDB Analytics Database

Provides help for tuning the InfiniDB analytic database for parallelization and scalability.

Introduction iii

iv Introduction

Chapter 1

Calpont InfiniDB Console Overview

The Calpont InfiniDB Console allows you to configure, monitor, and manage the Calpont InfiniDB system and servers. This chapter explains how to use the Calpont InfiniDB Console and gives a complete listing of Calpont commands. For more detailed information, see the following chapters: Configuring Calpont InfiniDB, Monitoring Calpont InfiniDB, and Managing Calpont InfiniDB System and Servers.

Logging on and off the Calpont InfiniDB Console

You can log on to the Calpont InfiniDB Console using a SSH client. This chapter describes both processes.

If you use a SSH client to connection to a Linux shell, you run the Calpont InfiniDB Console in the Linux shell. Many of the Linux-shell features are available to the Calpont InfiniDB Console such as:

Entering partial command names to execute commands

Recalling previous commands using the Up and Down arrows

Using hot keys to edit and navigate commands

The following procedures explain how to log on and off of the Calpont InfiniDB Console. You can choose to execute Calpont commands from a Linux prompt. See “Linux and Calpont InfiniDB Console command prompts” on page 4.

To log on to the Management Console with SSH client:

1. Connect to Linux using your SSH client.

You will be prompted to enter a user name and password.

2. Type your Linux user name and password and press Enter.

The server factory defaults are:

User Name: root

Password: Calpont1

3. At the Linux prompt, type cmconsole

Calpont InfiniDB Console Overview 1

The cmconsole prompt appears.

To log off of the Calpont InfiniDB Console:

1. At the cmconsole prompt, type exit or quit

The Linux prompt appears.

Calpont InfiniDB Console tips

The following tips and tricks help you use the Calpont InfiniDB Console.

Help commandThe help command displays supported commands. You can view brief help definitions or verbose definitions. You can also enter partial command names with the help command to view verbose definitions.

To view HELP commands:

1. At the Linux prompt, type cmconsole

2. Type help

An alphabetical list of support commands appears with brief descriptions.

2 Calpont InfiniDB Console Overview

You can enter the help command follow by a command name or partial command name to display the verbose description of the command.

For example, type help restartModule to get the verbose definition of the restart server command as shown in the figure below.

Case sensitivityCommands are not case sensitive; however parameters and device names, like server and processes, are case sensitive. For example, the GetAlarmLog command can be entered as getalarmlog.

Below are examples of the setsystemconfig command followed by the ProcessHeartbeatPeriod parameter.

Correct:

setsystemconfig ProcessHeartbeatPeriod 5SetSystemConfig ProcessHeartbeatPeriod 5


Recall command historyTo browse the history of commands entered, use the Up and Down arrow keys on your keyboard to scroll through commands.

Command repeat optionYou can run a command continuously using the “-r” option. This is useful to check status in real-time mode.

The repeat option repeats a command every 5 seconds. You can change the repeat interval to be between 1 and 60 seconds by adding the number of seconds after the command.

For example to repeat the GetProcess Status command every 2 seconds, type: GetProcessStatus –r2

NOTE:To exit out of a repeating command, press Ctrl-C to exit back to a Linux prompt, or Ctrl-D to return to the Calpont InfiniDB Console.

Linux and Calpont InfiniDB Console command prompts For easier command execution without going back and forth between Linux and the Management Console, you can choose to execute Linux commands from the Management Console or execute Calpont commands from a Linux prompt.

To execute a Linux system command from Calpont InfiniDB Console:

Type system (and the Linux command name) and press Enter. This processes the Linux command and displays output to the console.

For example, to view the Linux manual for the copy command from the Calpont prompt, type: system man cp

To execute a Calpont InfiniDB Console from the Linux prompt:

Type cmconsole Calpont_command

For example, to view the system status from the Linux command prompt, type


cmconsole getSystemStatus

Complete list of Calpont commands

Below are the commands you can run from the Management Console to configure, manage, and operate the Calpont System.

Table 1: Calpont Commands

Command Description

? Help

addExternalDevice Add External Device to Configuration file

addModule1 Add a Module within the Calpont System

alterSystem-disableModule1

Disable a Module and Alter the Calpont System

alterSystem-enableModule1

Enable a Module and Alter the Calpont System

alterSystem-reconfigureModule1

Reconfigure a Module and Alter the Calpont System

disableCoreDump Disable Core File Dumping on the system

disableLog Disable the levels of process and debug logging

enableCoreDump Enable Core File Dumping on the system

enableLog Enable the levels of process and debug logging

exit Exit from the Console tool

getActiveAlarms Get Active Alarm list

getActiveSQLStatements Get List Active SQL Statements within the System

getAlarmConfig Get Alarm Configuration Information

getAlarmHistory Get system alarms

getAlarmSummary Get Summary counts of Active Alarm


getCalpontSoftwareInfo Get the Calpont RPM detailed information

getCoreDumpConfig Get the System core file dump configuration

getExternalDeviceConfig Get External Device Configuration Information

getLogConfig Get the System log file configuration

getModuleConfig Get Module Name Configuration Information

getModuleCpu Get a Module CPU usage

getModuleCpuUsers Get a Module Top Processes utilizing CPU

getModuleDisk Get a Module Disk usage

getModuleMemory Get a Module Memory usage

getModuleMemoryUsers Get a Module Top Processes utilizing Memory

getModuleResourceUsage Get a Module Resource usage

getModuleTypeConfig Get Module Type Configuration Information

getProcessConfig Get Process Configuration Information

getProcessStatus Get Calpont Process Statuses

getSystemConfig Get System Configuration Information

getSystemCpu Get System CPU usage on all modules

getSystemCpuUsers Get System Top Processes utilizing CPU

getSystemDisk Get System Disk usage on all modules

getSystemInfo Get the Over-all System Statuses

getSystemMemory Get System Memory usage on all modules

getSystemMemoryUsers Get System Top Processes utilizing Memory

getSystemNetworkConfig Get System Network Configuration Information

getSystemResourceUsage Get System Resource usage on all modules


Command Description


getSystemStatus Get System and Modules Status

help Get help on the Console Commands

monitorAlarms Monitor alarms in realtime mode

quit Exit from the Console tool

reconfigureModule1 Reconfigure a Module within the Calpont System

removeExternalDevice Remove External Device to Configuration file

help Get help on the Console Commands

removeModule1 Remove a Module within the Calpont System

resetAlarm Resets an Active Alarm

restartModule Restarts the Processes within the Module

restartSystem Restarts the Processes within the Calpont System

resumeDatabaseWrites Resume performing writes to the Calpont Database

setAlarmConfig Set a Alarm Configuration parameter

setExternalDeviceConfig Set a External Device Configuration parameter

setModuleTypeConfig Set a Module Type Configuration parameter

setProcessConfig Set a Process Configuration parameter

setSystemConfig Set a System Configuration parameter

shutdownNode Shutdowns a Node within the Calpont System

shutdownSystem Shutdowns the Calpont System

startModule Starts a stopped Module within the Calpont System

startSystem Starts a stopped Calpont System

stopModule Stops the processing of a Module within the Calpont Sys-tem

stopSystem Stops the processing of the Calpont System


Command Description


suspendDatabaseWrites Suspend performing writes to the Calpont Database

system Execute a system shell command

1. These commands are not supported in a Shared-Nothing configuration. A message will be issued to the console stating “command not supported in a shared-nothing configura-tion.


Command Description


Chapter 2

System Maintenance

This chapters describes how to upgrade Calpont InfiniDB software and how to use the Calpont InfiniDB Console for server operations.

Using the Calpont InfiniDB Console, you can perform the following system and server operations: stop, start, restart, and shut down.

Upgrading Calpont InfiniDB

Calpont issues new releases and software patches as needed to fix issues or provide better performance. You should only upgrade Calpont InfiniDB when you are directed to do so by technical support.

Calpont softwareThere are two Calpont software packages that you can patch:

Calpont Platform Software - This software configures the modules and provides the Calpont performance capabilities and Calpont InfiniDB Console interface.

When advised by Calpont Technical Support, you should apply recommended patches.

Upgrading Calpont InfiniDBThe Calpont System should be taken offline. The front-end database and Calpont database should both be backed up before you upgrade Calpont InfiniDB. During an upgrade, Calpont InfiniDB keeps and uses the existing Calpont.xml file.

1. Stop Calpont InfiniDB. See “Stopping the system or modules” on page 11.

2. Back up front-end database and Calpont database. See “Performing Backup and Recovery” on page 51.

3. In a browser address window, go to ftp://ftp.calpont.com and press Enter.

4. Download the RPM(s).

5. Unextract the RPM zip file.

System Maintenance 9

The install program launches and places a shortcut on the desktop.

6. Install the upgrade. Please see the “Calpont InfiniDB Installation Guide” for installation/upgrade instruction.

Upgrading Storage

The Calpont System is typically configured with RAID 0+1 or commonly called RAID10, which is a stripe of mirrors and the system is partitioned with logical unit numbers (LUN). To better understand when additional storage capacity is needed, “Performance Tuning” on page 57

System and Module Operations

You can stop, start, restart or shut down the system and individual modules. In most cases Calpont recommends shutting down the system as opposed to modules. Additionally, you can disable and enable modules.

When you stop a module and do a system restart, the stopped module is started. If you disable a module and then restart the system, the disabled module does not restart, but stays offline until you enable it.

In the case that you want to disable or enable a module and bring the system back online quickly, you can use the AlterSystem command.

There following options are available when stopping the system or servers:

Graceful - Processes perform any gradual shutdown processes they might need.

Forceful (Default option) - Processes are stopped no matter what state they are in.

Install - This option is only used when stopping the system so that new Calpont Software can be installed. This option brings down all processes except the Management Console.

Acknowledgement Yes - The system informs you when the operation has been successfully or unsuccessfully performed.

Acknowledgement No (Default option) - The system returns a success message when the command is received, but the operation might still be processing.

10 System Maintenance

Stopping the system or modulesStopping the system or modules does not shutdown the hardware, but only stops the application processes. The platform process that supports the Management Console and System Alarms remain active. If you want to stop the system or server and immediately start the processes again (typically if the system or a server hangs), you can restart the system or servers as shown in the section “Restarting the system or modules” on page 11.

To stop the system:

1. From the Calpont InfiniDB Console, type stopSystem

2. Press y

The system processes stop.

To stop a server (you must specify the server you want to stop):

1. From the Calpont InfiniDB Console, type stopModule server_name

Example: stopModule UM2

2. Press y

The server processes stop.

Starting the system or modulesYou can start the system or module application processes with the following commands:

To start the system:

From the Calpont InfiniDB Console, type startSystem

The system processes start.

To start a module (you must specify the server you want to start):

From the Calpont InfiniDB Console,type startModule server_name

Example: startModule UM3

The server processes start.

Restarting the system or modulesWhen you want to stop and immediately start application processes, you can perform a restart. You restart the system or modules application processes with the following commands:

To restart the system:


From the Calpont InfiniDB Console, type restartSystem

The system processes stop and restart.

To restart a module (you must specify the module you want to start):

From the Calpont InfiniDB Console, type restartModule server_name

Example: restartModule UM3

The module processes stop and restart.

Shutting down the system or modulesWhen you perform a shutdown, all Calpont processes are stopped and the modules in the system are powered off. You should shutdown the system and/or modules when hardware maintenance is required.

To shutdown the system:

1. From the Calpont InfiniDB Console, type shutdownSystem

2. Press y

The system is shutdown.

To shutdown a module (you must specificy the module you want to shut down):

1. From the Calpont InfiniDB Console, type shutdownNode module_id

Example: shutdownNode PM3

2. Press y

The module processes stop and the module is shutdown.

Altering the System ModulesWhen you want to quickly disable or enable a module and restart the system, you can do it with the following commands:

To disable a module and restart system:

1. From the Calpont InfiniDB Console, type alterSystem-disableModule module_id

Example: alterSystem-disablemodule PM2, PM3

2. Press y

The module is disabled and the system is restarted.


To enable a module and restart system:

1. From the Calpont InfiniDB Console, type alterSystem-enableModule module_id

Example: altersSystem-enableModule PM@, PM3

2. Press y

The module is enabled and started as part of the system.


Chapter 3

System Status

The Calpont InfiniDB Console allows you to monitor the system without using a third-party Network Monitoring System (NMS). Calpont recommends you use a third-party NMS for more detailed system monitoring. The system is set up with factory defaults. To configure system monitoring, see “Configuring System Status Monitoring” on page 31.

Viewing system status

The system status shows the status of the system and all equipped servers. The table below shows the available system and server statuses.

To view the system status:

Type getSystemStatus

The system and server status appears.

Table 1: System and Module Statuses

Status Definition

Active The system, server, or Network Interface Card (NIC) is available to process database requests.

Auto Disabled Disabled as a result of a server failure.

Auto Init Auto initilization mode during a fault recovery.

Auto Offline The system or server is offline due to a fault.

Degraded The server is active, but the performance is degraded. A server is degraded when a NIC is not working.

Down Failing to communication via ping command.

Failed A stop/start/restart request for the system or a server failed.

System Status 15

When all servers are active, then the system status is active. If one server is man offline and the others are active, the system is man offline. All equipped servers must be active before the system is shown as active.

Viewing process status

The Process Configuration data is stored in the ProcessConfig.xml file in the /usr/local/Calpont/etc directory.

To view process status:

Type getProcessStatus

The processes status appears.

The table below shows the supported process states.

Initial Initial state after a system reboot or install and before any action is taken.

Man Disabled Disabled as a result of executing the altersystem-disableModule command.

Man Init Manual initialization mode during a start or restart command.

Man Offline The system or server was taken offline with the stop or shutdown command.

Up Successfully communication via ping command.

Table 2: Process Statuses

Status Definition

Active The process is functional and ready to process.

Auto Init Auto initialization mode during a fault recovery

Auto Offline The process is offline due to a fault.

Table 1: System and Module Statuses

Status Definition

16 System Status

Viewing network configuration

The network configuration lists the servers and status of the NIC cards.

To view network system configuration:

Type getSystemNetworkConfig

The system and server status appears.

Busy Init The process is performing an initialization task at startup time before going to the ACTIVE state.

Failed A stop/start/restart request for a process failed.

Hot Standby The Process is functional in a standby/ready state in case a failover occurs.

Initial State after a system reboot or install and before any action is taken

Man Init Manual initialization mode during a start or restart command

Man Offline The process was taken offline with the stop or shutdown command.

Standby Init Manual initialization mode during a start or restart command of a Hot Standby process.

Term Definition

Device Name The server configuration type. Example: DMs, UMs, and PMs.

Device Description

The server configuration type and unique numbering scheme. Exam-ple: Director Module #1.

NIC ID This field shows the available NICs on each server.

Host Name Host name of each server.

IP Address IP Address of each server.

NIC State A NIC is either UP or DOWN.

Table 2: Process Statuses

Status Definition

System Status 17

Viewing module information

The module configuration provides the server names and IP addresses for all modules.

To view the module configuration:

Type getModuleConfig

18 System Status

Monitoring module usage and threshholds

This section explains the module thresholds that are monitored by Calpont InfiniDB and how to view them.

To view module usage and threshholds:

Type getmoduletype

System Status 19

20 System Status

Module disk usageThe file systems on each local server can be monitored for the percentage used. There are three thresholds you can set for the disk usage for each server: Critical, Major, and Minor. When a threshold is crossed, an alarm is issued.

The alarms that are issued when the configured disk file system usage crosses each threshold are shown below:

DISK_USAGE_LOW is set when usage percentage goes above threshold ServerDiskMinorThreshold

DISK_USAGE_MED is set when usage percentage goes above threshold ServerDiskMajorThreshold

DISK_USAGE_HIGH is set when usage percentage goes above threshold ServerDiskCriticalThreshold

These same alarms are cleared as the usage percentage starts going down. The clearing is done in this manner:

DISK_USAGE_HIGH is cleared when the usage percentage usage goes below the threshold ServerDiskCriticalThreshold

DISK_USAGE_MED is cleared when the usage percentage usage goes below the threshold ServerDiskMajorThreshold

DISK_USAGE_LOW is cleared when the usage percentage usage goes below the threshold ServerDiskMinorThreshold

See “To configure the module disk usage:” on page 37.

Memory usageThe memory usage can be monitored for the percentage used. There are three thresholds you can set for the disk usage for each server: Critical, Major and Minor. When a threshold is crossed, an alarm is issued.

The alarms that are issued when the configured disk file system usage crosses each threshold are shown below:

MEMORY_USAGE_LOW is set when usage percentage goes above threshold ServerMemMinorThreshold

MEMORY_USAGE_MED is set when usage percentage goes above threshold ServerMemMajorThreshold

MEMORY_USAGE_HIGH is set when usage percentage goes above threshold ServerMemCriticalThreshold

System Status 21


MEMORY_USAGE_HIGH is cleared when the usage percentage usage goes below the threshold ServerMemCriticalThreshold

MEMORY_USAGE_MED is cleared when the usage percentage usage goes below the threshold ServerMemMajorThreshold

MEMORY_USAGE_LOW is cleared when the usage percentage usage goes below the threshold ServerMemMinorThreshold

Swap space usageThe swap space usage can be monitored for the percentage used. There are three thresholds you can set for the swap space usage for each server: Critical, Major, and Minor. When a threshold is crossed, an alarm is issued.

The alarms that are issued when the Configured Disk File System usage crosses each threshold are shown below:

SWAP_USAGE_LOW is set when usage percentage goes above threshold ServerSwapMinorThreshold

SWAP_USAGE_MED is set when usage percentage goes above threshold ServerSwapMajorThreshold

SWAP_USAGE_HIGH is set when usage percentage goes above threshold ServerSwapCriticalThreshold


SWAP_USAGE_HIGH is cleared when the usage percentage usage goes below the threshold ServerSwapCriticalThreshold

SWAP_USAGE_MED is cleared when the usage percentage usage goes below the threshold ServerSwapMajorThreshold

SWAP_USAGE_LOW is cleared when the usage percentage usage goes below the threshold ServerSwapMinorThreshold.

Disk thresholdsYou can set thresholds to alert you when a specified percentage of a file system is being used on a server.

There are three thresholds you can set for the disk usage for each server: Critical, Major, and Minor. When a threshold is crossed, an alarm is issued.

22 System Status

The alarms that are issued when the Disk File System usage crosses each threshold are shown below:

DISK_USAGE_LOW is set when usage percentage goes above threshold RAIDMinorThreshold.

DISK_USAGE_MED is set when usage percentage goes above threshold RAIDMajorThreshold.

DISK_USAGE_HIGH is set when usage percentage goes above threshold RAIDCriticalThreshold.

These same alarms are cleared as the usage percentage starts going down as shown below:

DISK_USAGE_HIGH is cleared when the usage percentage usage goes below the threshold RAIDCriticalThreshold.

DISK_USAGE_MED is cleared when the usage percentage usage goes below the threshold RAIDMajorThreshold.

DISK_USAGE_LOW is cleared when the usage percentage usage goes below the threshold RAIDMinorThreshold.

CPU thresholdsYou can set thresholds to alert you when the CPU usage on a server reaches a specific percentage.

There are four thresholds for the server CPU usage: Critical, Major, Minor and MinorClear. When a threshold is crossed, an alarm is issued. To disable thresholds, set the thresholds to 0.

The alarms that are issued when the Server total CPU usage crosses each threshold are shown below:

CPU_USAGE_LOW is set when CPU goes above threshold ServerCPUMinorThreshold

CPU_USAGE_MED is set when CPU goes above threshold ServerCPUMajorThreshold

CPU_USAGE_HIGH is set when CPU goes above threshold ServerCPUCriticalThreshold

These same alarms are cleared as the CPU usage starts going down. To prevent alarm thrashing, the clearing is done in this manner:

CPU_USAGE_HIGH is cleared when the CPU usage goes below the threshold ServerCPUMajorThreshold

System Status 23

CPU_USAGE_MED is cleared when the CPU usage goes below the threshold ServerCPUMinorThreshold

CPU_USAGE_LOW is cleared when the CPU usage goes below the threshold ServerCPUMinorClearThreshold

Monitoring alarms

Alarms are triggered when system, servers, processes, or hardware are down, power on test has warning, process fail or time out, or when threshold percentages are met. These thresholds are set for the following:

Module disk usage

Memory usage

Swap space usage

RAID usage

CPU usage

This section explains how to view alarms, and describes the alarm definitions. To change default thresholds, see “Configuring System Status Monitoring” on page 31.

Viewing active alarmsTo view active alarms:

Type getActiveAlarms

Below is a sample alarm file.

24 System Status

Viewing archived alarmsAlarms are archived each day at midnight. The Calpont system saves seven days of alarm history.

To view historitcal alarms:

Type getAlarmHistory (yyyymmdd)

In place of the date, you can type today to get the alarm log for the current day.Example statements: getAlarmHistory today getAlarmHistory 20070923

Table 3: Alarm Definitions

Description Definition

Alarm ID#1 CPU_USAGE_HIGH The usage on the indicated CPU has exceeded its high threshold

System Status 25

Alarm ID#2 CPU_USAGE_MED The usage on the indicated CPU has exceeded its medium threshold

Alarm ID#3 CPU_USAGE_LOW The usage on the indicated CPU has exceeded its low threshold

Alarm ID#4 DISK_USAGE_HIGH The usage on the indicated disk drive has exceeded its high threshold

Alarm ID#5 DISK_USAGE_MED The usage on the indicated disk drive has exceeded its medium threshold

Alarm ID#6 DISK_USAGE_LOW The usage on the indicated disk dr ive has exceeded its low threshold

Alarm ID#7 MEMORY_USAGE_HIGH The usage on the indicated memory has exceeded its high threshold

Alarm ID#8 MEMORY_USAGE_MED The usage on the indicated memory has exceeded its medium threshold

Alarm ID#9 MEMORY_USAGE_LOW The usage on the indicated memory has exceeded its low threshold

Alarm ID #10 SWAP_USAGE_HIGH

The usage on the indicated swap has exceeded its high threshold

Alarm ID #11 SWAP_USAGE_MED

The usage on the indicated swap has exceeded its medium threshold

Alarm ID #12 SWAP_USAGE_LOW

The usage on the indicated swap has exceeded its low threshold

Alarm ID #13 PROCESS_DOWN_AUTO Process is auto out-of-service

Alarm ID #14 MODULE_DOWN_AUTO Module is auto out-of-service

Alarm ID #15 SYSTEM_DOWN_AUTO System is auto out-of-service

Alarm ID #16 POWERON_TEST_SEVERE A Module Power-On Severe Warning error occurred. Test failures indicate that the sys-tem is running in a degraded state

Alarm ID #17 POWERON_TEST_WARNING Warning error occurred. Test failures indicate that the system is running with little or no degradation in performance


26 System Status

Alarm ID #18 HARDWARE_HIGH A hardware device's resource has exceeded its high threshold

Alarm ID #19 HARDWARE_MED A hardware device's resource has exceeded its medium threshold

Alarm ID #20 HARDWARE_LOW A hardware device's resource has exceeded its low threshold

Alarm ID #21 PROCESS_DOWN_MANUAL Process is manually out-of-service

Alarm ID #22 MODULE_DOWN_MANUAL Module is manually out-of-service

Alarm ID #23 SYSTEM_DOWN_MANUAL System is manually out-of-service

Alarm ID #24 SWITCH_DOWN_AUTO Switch is auto out-of-service

Alarm ID #25 STORAGE_DOWN_AUTO Storage Unit is auto out-of-service

Alarm ID #26 PROCESS_INIT_FAILURE Process Initialzation failed or timed-out

Alarm ID #27 NIC_DOWN_AUTO NIC Interface is auto out-of-service

Alarm ID #28 DBRM_LOAD_DATA_ERROR Error occurred during loading DBRM data

Alarm ID #29 INVALID_SW_VERSION Invalid software version found on a module which doesn’t match the version running on the Director Module

Alarm ID #30 STARTUP_DIAGNOSTICS_FAILURE A failure occurred when the system startup diagnostics was executed on a module. Log files can be viewed for details on the failure

Alarm ID #31 CONN_FAILURE A connection failure occurred that could affect the performance of system processing.

Table 4: Alarm Severity

Severity Description

Critical A condition occurred that requires immediate attention and could result in loss of data.


System Status 27

Viewing top CPU usersYou can view the top users on a single module or the top users on all modules in Calpont InfiniDB. The default is to display the top five users, but you can specify to display a range from 1-10 processes. Processes that use less than 1% CPU are not displayed.

To monitor top users on a single module:

Type getmodulecpuusers module_ID

For example, to monitor to the top CPU users on Director Module 1, the command is:

getmodulecpuusers dm1

To monitor top users on Calpont InfiniDB:

Type getsystemcpuusers

Monitoring resource usageYou can monitor the system CPU usage, memory, cache and swap space for the system by module.

To monitor system resources:

Type getsystemresourceusage

Major A condition occurred that impairs service and needs attention.

Minor A condition occurred that hasn’t impaired ser-vice, but needs attentions so that it doesn’t impair service in the future.

Warning A condition has occurred that ......

Info Information only.

Table 4: Alarm Severity

Severity Description

28 System Status

System Status 29

30 System Status

Chapter 4

Configuring System Status Monitoring

This chapter describes how to configure the system, processes, servers, CPU usage, disk usage, RAID, memory usage, and swap space.

For more information about monitoring the system, “System Status” on page 15, and for a complete list of Calpont Commands, see page page 5.

Management console overview

This chapter explains how to configure the following:

System

Modules

Processes

Alarms

Logging

Calpont configuration filesWhen you configure the system using the Management Console, the changes are stored in XML files that reside in the “etc” directory. Some of these configuration items are updated by Calpont Processes and the Calpont postConfig script at installation time.

AlarmConfig.xml - Alarm configuration file.

Calpont.xml - System and module configuration file.

ConsoleCmds.xml - Management Console tool help file.

ProcessConfig.xml - Process configuration file.

Linux shellYou need a SSH client to connect to the Linux shell. Once you are connected to the Linux shell, you enter the command to run the Management Console and it runs in the

Configuring System Status Monitoring 31

Linux shell. Many of the Linux-shell features are available to the Management Console such as:

Entering partial command names to execute commands

Recalling previous commands using the Up and Down arrows

Using hot keys to edit and navigate commands

For tips on using the Management Console, “Calpont InfiniDB Console tips” on page 2.

Configuring system

You can modify the following system settings: the heartbeat period and counts, RAID thresholds, monitoring, alarm, and logging configurations. The system configuration file also displays the current software and hardware versions used with the Calpont Database Platform. This can be useful troubleshooting information.

Most of the data in the configuration file is stored in the Calpont.xml file located in /usr/local/Calpont/etc directory. Some data is extracted from other sources when the getSystemConfig command is executed.

To view the system configuration:

1. At the Linux prompt, type cmconsole.

2. Type getSystemConfig and press Enter.

32 Configuring System Status Monitoring

3. The system configuration appears.


Setting server heartbeatsHeartbeat monitoring occurs between servers in the Calpont Database Platform. The server heartbeat settings are the same for all servers in the Calpont Database Platform.

The server heartbeat period refers to how often the heartbeat test is performed. For example, if you set the period to 5, then the heartbeat test is performed every 5 seconds.

The server heartbeat count refers to how many failures in a row must take place before a fault is processed. To disable heartbeat monitoring, set the period to -1.

To set the server heartbeat period and count:


2. Type setSystemConfig ServerHeartbeatPeriod n (where n= number of seconds) and press Enter.

A message appears when the period is successfully set.

3. Type setSystemConfig ServerHeartbeatCount n (where n= number of failures) and press Enter.

A message appears when the count is successfully set.

Configuring Disk thresholdsYou can set thresholds to alert you when a specified percentage of a file system is being used on a server.

There are three thresholds you can set for the disk usage for each server: Critical, Major and Minor. When a threshold is crossed, an alarm is issued. To disable thresholds, set the threshold to 0.

The alarms that are issued when the Disk File System usage crosses each threshold are shown below:

DISK_USAGE_LOW is set when usage percentage goes above threshold RAIDMinorThreshold.

DISK_USAGE_MED is set when usage percentage goes above threshold RAIDMajorThreshold.

DISK_USAGE_HIGH is set when usage percentage goes above threshold RAIDCriticalThreshold.

These same alarms are cleared as the usage percentage starts going down as shown below:


DISK_USAGE_HIGH is cleared when the usage percentage usage goes below the threshold RAIDCriticalThreshold.

DISK_USAGE_MED is cleared when the usage percentage usage goes below the threshold RAIDMajorThreshold.

DISK_USAGE_LOW is cleared when the usage percentage usage goes below the threshold RAIDMinorThreshold.

The system configuration parameters are:

RAIDMinorThreshold

RAIDMajorThreshold

RAIDCriticalThreshold

To configure Disk thresholds:


2. Type setSystemConfig RAID(Minor/Major/Critical)Threshold n (where n= percentage of file system used) and press Enter.

A message appears when the threshold is successfully set.

For example, to modules to issue a disk usage medium alarm when the file system is 25% full, enter the following command:

setSystemConfig RAIDMajorThreshold 25

TIP: When setting thresholds, minor should be less than major, and major should be less than critical.

Configuring archived transaction log periodsYou can change how often the transaction log file is archived by configuring the systemconfiguration file. The default period is set to 10 minutes. You can select between 0 and 60 minutes. For more information about transaction logs, see “/usr/local/Calpont/datan where n represents unique directories such as data1, data2, etc. Copy all the files and directories in each numbered data directory.” on page 54.

To modify archived transaction log period:

1. Connect to Linux using your SSH client.

You will be prompted to enter a user name and password.


2. Type your Linux user name and password and press Enter.

A Linux prompt appears.


The cmconsole prompt appears.

4. Type setsystemconfig TransactionArchivePeriod (log period) and press Enter.

Configuring modules

You can configure the following module items:

CPU thresholds

Disk monitor file system usage

Disk thresholds

Module memory thresholds

Module swap thresholds

Configuring module CPU thresholdsYou can set thresholds to alert you when the CPU usage on a module reaches a specific percentage. For more information about CPU Thresholds, see “CPU thresholds” on page 23.

There are four thresholds for the module CPU usage: Critical, Major, Minor and MinorClear. When a threshold is crossed, an alarm is issued. To disable thresholds, set the thresholds to 0.

To configure the CPU thresholds:


2. Type setServerConfig (server name) ServerCPU(Clear/Minor/Major/Critical)Threshold n (where n= percentage of CPU usage) and press Enter.

The CPU threshold is set and a message appears.

For example, to set a server alert to Minor at 50%, type the following:setServerConfig CD1C ServerCPUMinorThreshold 50.



Configuring module disk usageThe file systems on each local server can be monitored for the percentage used. For more information about server disk usage, see “Module disk usage” on page 21.

To configure the module disk usage:


2. Type setServerConfig (server name) ServerDisk(Minor/Major/Critical)Threshold n (where n= percentage of disk system used) and press Enter.

The server disk usage threshold is set and a message appears.


Configuring swap space usageThe swap space usage can be monitored for the percentage used. There are three thresholds you can set for the swap space usage for each server: Critical, Major and Minor. When a threshold is crossed, an alarm is issued. To disable thresholds, set the thresholds to 0.For more information about swap space usage, see “Swap space usage” on page 22.

To configure the server swap space usage:


2. Type setServerConfig (server name) ServerSwap(Minor/Major/Crictical)Threshold n (where n= percentage of swap space used) and press Enter.

The server swap space usage threshold is set and a message appears.



Configuring alarms

In this chapter, you learned how to configure the system and servers to trigger alarms for specific events or conditions. You can also set the total number of alarms that can be triggered in a 30-minute period. The Calpont system comes with a pre-configured alarm file, stored in the CalpontAlarm.xml file, that has the threshold setting for all alarms at 100. This means that an alarm can be triggered a maximum of 100 times in 30 minutes before the alarm quits being triggered.

To disable a specific alarm from being reported, set the alarm threshold to 0.

Steps to configure alarmsTo configure alarms, you use the setalarmconfig command with the Alarm ID#, and the parameter to be changed.


2. Type setAlarmConfig (alarmID#) Threshold n (where n= maximum number of times an alarm can be triggered in 30 minutes), and press Enter.

For example, if you want to change Alarm ID#23 to have a threshold of 25 you would type the following at the Calpont prompt: setAlarmConfig 23 Threshold 25.

Configuring logging

The Log Configuration can be displayed using the following command: getLogConfig. There are 5 levels of logging:

Critical

Error


Warning

Info

Debug

Each server within the Calpont System can have a different levels configured. These five levels are supported through the Syslog functionality. So this means that the Syslog service should always be running on the servers.

The Log Configuration can be enabled and disabled using the following commands: EnableLog, DisableLog. These five logging levels can be individually enable and disabled or you can enable and disable all five by entering "all" for the log level.

Messages are identified by a unique number. In the following syslog message, items 1-4 are generated automatically by syslog and items 5-8 come from the message object itself.

1. Date & time

2. Host of origin

3. Subsystem name

4. PID

5. Seconds and microseconds of message

6. Session ID, transaction ID & thread ID

7. Message severity level - A single character message-class prefix:

D - debug

I - info

W - warning

E - error

C - critical

8. Subsystem ID - A 2-digit subsystem ID.

9. Message number - A 4-digit error number. Error message ID's (and thus error message texts) can be shared across subsystem boundaries. This allows reuse of common error messages.

10. Message text - formatted error text after all positional arguments have been substituted.


These log file are stored on each server in /var/log/Calpont directory.

Enabling and disabling loggingTo enable logging:


2. Type enableLog (module_ID)(log_type) and press Enter.

For example, to enable Crictical logging on server PM1, the command is: enableLog PM1 Crictical.

To disable logging:


2. Type disableLog (module_ID)(log type) and press Enter.

For example, to disable Info logging on server PM1, the command is: disableLog PM1 Info.


Chapter 5

Importing Data

This chapter describes how to import data into the Calpont System. This includes data migrations from other systems or periodic updates during the maintenance window.

Import overview

Before importing data, you must complete the Calpont Installation as documented in the Calpont InfiniDB Installation Guide. This includes creating a Calpont system catalog and creating the tables in which you want to import data.

Estimating load sizesBusiness requirements for data latency drive the size of data loads. If access to new data is needed every 30 seconds, then data should be loaded in smaller loads so that it can be written to the database quickly and available for queries.

If access to new data is needed within 24 hours, then data can be loaded overnight in larger data loads.

Import TasksPerform the tasks below to import data:

Copy data files to be imported into the correct directory

Generate job file

Import data

Import processCalpont InfiniDB does the following during the import:

Reads data

Transforms the data into a column orientation

Tokenizes data values

Load tokenized data into database

Importing Data 41

Directory structureThe directory structure below is created when Calpont InfiniDB is installed. This structure must be maintained to successfully import data. You perform the bulk load process on the Performance Module with the DDLProc and DMLProc processes active (typically the first Performance Module).

/usr/local/Calpont/data/bulk./job - Contains the job description XML files that map tables and columns. ./data//import - Contains the data files to be imported. ./log - Contains the log file for each executed job and error logs if applicable. The file extensions are .log and .err../process - Contains the temporary process files.

Copy source files

Source files need to be formatted, named correctly, and placed in the Calpont import folder for Calpont InfiniDB to import the data.

Source file formatsThe default delimiter is the ‘|’ (pipe) character.

You can configure the system to accept other formats by providing an optional delimiter paramenter during the import. For most character delimiters, use -d and the delimiter character.

Examples:-d ,-d ~

For special characters such as tab, single quote, or tilde, add a backwards slash “\” and then the special character. Note: If you use the backwards slash for special characters, it must be wrapped in a single quotes. The examples for each are as follows:

-d ‘\t’-d ‘\~’

Source file namesThe source files must be named with the table name that the data should be imported into. Example: nation.tbl

42 Importing Data

If source data resides in a different directory, you can use the ln command tp create a symbolic link “import” instead. For example:

ln -s /mnt/source /usr/local/Calpont/data/bulk/data/import

Import directoryYou must place source data to imported into the following directory:

/usr/local/Calpont/data/bulk/data/import

Generate job file

You must create an XML job file for tables that you are loading before you can import data. You should create a new job file before each bulk load.

To generate the job file for all tables:

At the Linux prompt, type: /usr/local/Calpont/bin/colxml schema_name

Use the -t option for specific tables.

The job file is named Job_(id#).xml and placed in the job directory. The default name is: Job_299.xml.

The table below lists additional parameters that you can use when creating the job file. For example, to change the delimiter from the default | Pipe Bar to a tilde:

/usr/local/Calpont/bin/colxml schema_name -d \~

Table 1: Optional job file parameters

Parameter Definition

-d Delimter type. See “Source file formats” on page 42.

-e Max errors allowed before aborting

-j Job ID

-l Load table name

-n Name in quotes

-p Path

Importing Data 43

Import data

Calpont InfiniDB uses the cpimport utility to import data. This file is located in the following directory:

/usr/local/Calpont/bin

You must specify the job ID when importing data.

To import data:

At the Linux prompt, type cpimport -j jobID#

where jobID is the number of the job number from the colxml execution

and press Enter.

Example: cpimport -j 299

Normally you will not need to specify any other parameters to cpimport, as the XML job file contains the details of the job. In some cases you may see an incremental performance increase by raising the number of parsers threads (the –w option). Users are strongly encouraged to perform a full backup of the database before experimenting with any of the cpimport options.

You can get a list of the other optional parameters by typing ‘cpimport –h’. Note that some options are not implemented on the 32-bit architecture builds.

-s Description in quotes

-t Table Name

-u User

-b Debug level

Table 1: Optional job file parameters

Parameter Definition

44 Importing Data

Troubleshooting import

If the import terminates abnormally, an error code and a brief description are displayed. Go to the log directory and open the error log file in a text editor to view more detailed information.

The error log file be logged to a file named Job_(id#).err. Example: Job_299.err.

Successful imports will be logged to a file named Job_(id #).log. Example: Job_299.log.

In both files, the job ID is the same as the job ID of the XML job file.

Below are some possible errors that you might encounter.

Number of error rows exceeds threshholdError message “Total number of rows exceeds the maximum error number specified in the job description file”.

The maximum errors defaults to 10. Increase the maximum errors by using the -e parameter followed by the maximum number of rows.

1. At the Linux prompt, type cpimport.sh -e (number of rows) and press Enter.

Data mismatchWhen the data to be imported does not match up correctly with the table in which it is to be imported, the rows are saved into a new file labeled with the table name and .bad. For example: nation.tbl.bad. These tables are saved in the /import directory.

You can review the bad data in the files, make changes to the data to be imported, and re-run the import.

Importing Data 45

46 Importing Data

Chapter 6

Transactions

This chapter describes how transactions are handled in the Calpont System.

Transaction overview

Calpont InfiniDB processes Insert, Update, and Delete statements through transactions such that changes to the database are not made visible to other queries unless a commit statement has been issued.

The commit may be applied automatically or may be explicitly issued depending on client configurations external to Calpont InfiniDB. The current setting can be viewed via the following command:

mysql> show variables like '%autocommit%';

--------------show variables like '%autocommit%'--------------+---------------+-------+| Variable_name | Value |+---------------+-------+| autocommit | OFF |+---------------+-------+1 row in set (0.00 sec)

The current setting can be changed by one of the following commands:

mysql> set autocommit = on;

mysql> set autocommit = off;

Execution of concurrent DML statements is limited in this release of the Calpont InfiniDB product to ensure data consistency under all cases. Additional transactions beyond the first will wait until the first transaction has committed before executing.

Transactions 47

Transaction Maintenance

In the event that you wish to roll back a transaction for another session, the following procedure can be used:

1. Verify that there are outstanding transactions:

/usr/local/Calpont/bin/SesMgrverID = 223TxnID SessionID------- -------223 1

2. Verify that there are outstanding transactions that exceed the time limit set in the Calpont.xml config file:

/usr/local/Calpont/bin/sessionWalker1 timed out transaction(s).sessionid 1 txnid 223 valid TRUE time_t 1246470554 tdiff 30 ctime Wed Jul 1 12:49:14 2009

3. Roll those transactions back:

/usr/local/Calpont/bin/sessionWalker -r1 timed out transactions.sessionid 1 txnid 223 valid TRUE time_t 1246470554 tdiff 30 ctime Wed Jul 1 12:49:14 2009

You can place this command in a cron script to automatically roll back transactions that run longer than you have configured, but you should consider all the effects of such a policy.

ACID Compliance

Database systems deliver ACID behavior:

Atomicity: Either all the tasks in a transaction must be done, or none of them. The transaction must be completed, or else it must be undone (rolled back).

Consistency: Every transaction must preserve the integrity constraints - the declared consistency rules - of the database. It cannot place the data in a contradictory state.

48 Transactions

Isolation: Two simultaneous transactions cannot interfere with one another. Intermediate results within a transaction are not visible to other transactions.

Durability: Completed transactions cannot be aborted later or their results discarded. They must persist through (for instance) restarts of the DBMS after crashes.

http://en.wikipedia.org/wiki/Database

The key to support ACID behavior related to Atomicity and Isolation will be accomplished for the Calpont Engine a custom implementation of concurrency control to support consistent read behavior in support of Isolation and rollback in support of Atomicity. Calpont InfiniDB provides the most commonly use Isolation level of Read Committed which ensures that any query is consistent as of the start of the query and won't see any partially entered data that is added after the query starts.

The most important benefit of Calpont InfiniDB's versioning is that reads are never blocked. That is, one session can read (an older) versioned block while another session is simultaneously updating that block.

Both Rollback (Atomicity) and Consistent Read (Isolation) support are implemented by retaining prior versions of data.

Datatype consistency is supported through the DDL and DML. Constraints are not supported.

Durability is supported through the transaction log and checkpointing (saves to disk at appropriate points).

Transactions 49

50 Transactions

Chapter 7

Performing Backup and Recovery

This chapter explains how to perform a full Calpont InfiniDB data backup and how to perform a recovery in the event of a disk outage or other catastrophe. It explains how to recover data to the point of the last full backup or perform a point-in-time recovery.

This chapter will also describe dropping the Calpont InfiniDB database to start from the beginning.

Backing up data

To create a full data backup, you must back up the front-end database and the Calpont database from the same point in time.

To back up your data, perform the following procedures which are explained in more detail later in this chapter:

Suspend database writes

This process suspends database writes to ensure data consistency during the backup process, but allows users to continue to query the database.

Back up front-end database

This process backs up the users, and their database objects that connect to the Calpont InfiniDB Platform.

Back up Calpont InfiniDB Platform database files

This process backs the Calpont InfiniDB Platform database files.

Resume database writes

This process is done after front-end database and Calpont InfiniDB backups are complete and allows users to write to the database.

Performing Backup and Recovery 51

Suspending database writesYou must suspend database writes when backing up your data to ensure data consistency.

NOTE: If a user attempts to write to the database during a backup, Calpont generates an error message, but the backup process is not affected. Calpont allows queries during the back up process.

To suspend database writes:

1. Connect to the Director Module 1, or your OAM Module using a SSH client.

2. Type your Linux user_name and password.


3. [root@server]# cmconsole

The Calpont InfiniDB Console prompt appears.

4. calpont>suspendDatabaseWrites

A confirmation message appears.

Backing up front-end databaseYou should back up your front-end database using the procedures and instructions for that vendor.

Calpont InfiniDB doesn’t implement anything unique into the MySQL front-end and the following directory may be backed up in lieu of previously established procedures:

/usr/local/Calpont/mysql/db

Example:cp -rp /usr/local/Calpont/mysql/db /mnt/InfiniDB/backup/frontend

Note: The -rp options are for copying directories recursively and saving ownership information.

If selected databases are to be backed up only, then the database directories within the above directory may be backed up instead.

52 Performing Backup and Recovery

Backing up Calpont InfiniDB databaseThis process creates a full Calpont database backup by copying all the DBRM and DB (data) files. Because you can have multiple files being copied from multiple directories, you should label your backups so that you can restore them to the proper directory in the event of a failure.

Copy all files from Performance Module 1.

To backup the Calpont database:

Copy the entire directory structure of each of your DBRoots to backup storage. These directories are listed in the /usr/local/Calpont/etc/Calpont.xml file in the SystemConfig section. Usually they will look like /usr/local/Calpont/datan where n is a number starting at 1. Depending on your configuration you may also have important system data in a directory called simply /usr/local/Calpont/data. You should also backup this directory if it contains any important files for your installation (e.g. bulk load scripts).

Resuming database writesWhen you have finished backing up your MySQL database and the Calpont database, resume database writes.

1. Connect to the Director Module 1, or your OAM Module using a SSH client.

2. Type your Linux user_name and password.


3. [root@server]# cmconsole

The Calpont InfiniDB Console prompt appears.

4. calpont>resumeDatabaseWrites

A confirmation message appears.

Recovering from a backup

If you have a hardware failure and need to recover from a backup, perform the following procedures which are described in more detail later in this chapter:


Restore front-end database

Restore Calpont database

Restoring front-end databaseRestore your front-end database using the recommended procedures.

Calpont InfiniDB doesn’t implement anything unique into the MySQL front-end and the following directory may be restored in lieu of previously established procedures:

/usr/local/Calpont/mysql/db

Example to remove and restore:rm -rf /usr/local/Calpont/mysql/db/*cp -rp /mnt/InfiniDB/backup/frontend/* /usr/local/Calpont/mysql/db/

Note: The -rf options are for removing directories recursively and not prompting for removal. The -rp options are for copying directories recursively and saving ownership information.

If selected databases are to be restored only, then the database directories within the above directory may be restored instead.

Restoring Calpont InfiniDB databaseTo restore the Calpont database, you must copy the following files and directories from the backup directory to the Performance Module in which DDLProc is active.

To restore the Calpont database:

1. Ensure the InfiniDB system is not active.

2. Copy the data files and directories back to their original location.

/usr/local/Calpont/datan where n represents unique directories such as data1, data2, etc. Copy all the files and directories in each numbered data directory.


Transactions

Saving Calpont Transaction Logs

DDL and DML statements are written to transaction logs and written by the currently active Performance Module. These logs allow you to review or replay statements as needed.

Transaction logs can be copied off of the production server as needed.

The following table lists the file names and the directories where the files are located.

Replaying Transaction LogsThe transaction log allows you to review or replay statements.

Transaction logs are archived every 10 minutes. See “Configuring archived transaction log periods” on page 35 for instructions on changing the archive period.

If copying transaction log files back to the source directory, be aware that some transaction log files may be more current on the target directory and should not be over-written.

Dropping the Calpont InfiniDB database

If you need/want to go back to "square one", that is, restore the system to the same condition as when you first installed Calpont InfiniDB, the following steps should be fol-lowed:

Table 1: Transaction Logs

Default Name Default Directory

data_mods.log /usr/local/Calpont/data1/systemFiles/dataTransaction

data_mods.log.timestamp /usr/local/Calpont/data1/systemFiles/dataTransaction/archive


1. In MySQL identify all the Calpont InfiniDB tables. For each of these tables: 'drop <table> restrict;'.

Note: The 'restrict' option will speed up the drop table, but should only be used in anticipation of deleting the Calpont InfiniDB database.

2. Use calpontConsole to stopSystem.

3. Identify each of your DBRoots, possibly by going through the Calpont.xml file. For each of these roots, 'rm -rf <root>/000.dir'.

4. Locate the DBRM persistence files, usually in /mnt/OAM/dbrm and remove all of them.

5. Erase the InfiniDB shared memory, '/usr/local/Calpont/bin/clearShm'.

6. Use calpontConsole to startSystem.

7. When the system is fully up, at the Linux prompt build a new Calpont InfiniDB System Catalog: '/usr/local/Calpont/bin/dbbuilder 7'.

The system is now ready for the creation of tables.


Chapter 8

Performance Tuning

This chapter describes the methods of tuning Calpont InfiniDB performance.

Query Tuning Guidelines - Data Access Operations

Calpont InfiniDB does not use traditional index or table scan operations. Neither operation exists within the Calpont InfiniDB system.

Operations used by Calpont InfiniDB include: Full Column Scan - an operation that scans each entry in a column using all

available threads on the Performance Modules. Speed of operation is generally related to the size of the data type and the total number of rows in the column. The closest analogy for a traditional system is an index scan operation.

Partitioned Column Scan - an operation that uses the Calpont InfiniDB Extent Map to identify that certain portions of the column do not contain any matching values for a given set of filters. The closest analogy for a traditional row based dbms is a partitioned index scan, or partitioned table scan operation.

Column lookup by row offset - once the set of matching filters have been applied and the minimal set of rows have been identified, additional blocks are requested using a calculation that determines exactly which block is required. The closest analogy for a traditional system is a lookup by rowid.

These operations are automatically executed together in order to execute appropriate filters and column lookup by row offset. This set of operations makes up the work issued from the User Module to the set of Performance Modules in support of a given step in a query.

Performance Tuning 57

Gathering Performance Metrics - calgetstats()

The calgetstats() function is used to provide information on the last query executed within a given session.select calgetstats();

mysql> select count(*) from part;+-----------+| count(*) |+-----------+| 200000000 |+-----------+1 row in set (0.48 sec)

mysql> select calgetstats();+----------------------------------------------------------------------------------------| calgestats() +----------------------------------------------------------------------------------------| Query Stats: MaxMemPct-0; NumTempFiles-0; TempFileSpace-0MB; PhyI/O-0; CacheI/O-98039; +----------------------------------------------------------------------------------------

. . . continued:

---------------------------------------------------------------------------------------+BlocksTouched-97658; CasPartBlks-0; MsgBytesIn-2MB; MsgBytesOut-0MB| 1242146662 640516 |---------------------------------------------------------------------------------------+

1 row in set (0.00 sec)

The output contains information on:

MaxMemPct - Peak memory utilization on the User Module, likely in support of a large (User Module) based hash join operation.

NumTempFiles - Report on any temporary files created in support of query operations larger than available memory, typically for unusual join operations where the smaller table join cardinality exceeds some configurable threshold.

TempFileSpace - Report on space used by temporary files created in support of query operations larger than available memory, typically for unusual join operations where the smaller table join cardinality exceeds some configurable threshold.

58 Performance Tuning

PhyI/O - Number of 8k blocks read from disk, SSD, or other persistent storage. In most cases, the quantity of individual I/O operations are far fewer than the number of blocks; Calpont InfiniDB will read as many as 512 blocks together in one I/O operation.

CacheI/O - Approximate number of 8k blocks processed in memory, adjusted down by the number of discrete PhyI/O calls required.

BlocksTouched - Approximate number of 8k blocks processed in memory.

CalPartBlks - The number of block touches eliminated via the Calpont InfiniDB Extent Map elimination behavior.

MsgBytesIn, MsgByteOut - Message bytes in MB sent between nodes in support of the query.

Gathering Performance Metrics

Using /var/log/Calpont/debug.logAt the default trace level, Calpont InfiniDB records each statement being executed within the debug.log along with the start time and end time of the query. This information is stored in the /var/log/Calpont/debug.log file on the User Module executing the query.

Additional information can be emitted with other trace levels set by calsettrace();

Using calsettrace(1);The calsettrace(1) function enables more detailed information to be stored within the /var/log/Calpont/debug.log on the User module executing the statement.

The argument (1) enables output of this file, while (0) will turn off creation of the file. Enabling or disabling of creation of this file is at the session level. Other trace levels should only be set under instruction from Calpont support.

select calsettrace(1);

Note that setting calsettrace(1); will cause each statement to have warning information available. A sample warning might be:

1 row in set, 1 warning (0.06 sec)


Issue of the

Show warnings;

command will present the same information as 'select calgetstats();'

Using /var/log/mysqld.logAdditional logging information related to the interface between MySQL and Calpont InfiniDB is emitted in the /var/log/mysqld.log file on the User Module executing the statement. The information presented here does not present a full picture of operations executed in support of a query but may be used under instruction from Calpont support.

Understanding Extent Map Range Partitioning

The ExtentRows parameter defines the size for ranges of rows (default 8 million) called extents, which have the following characteristics:

1. All column files for an extent are created on the same mount point.

2. The extent is dynamically mapped to a given Performance Module at run-time.

3. The minimum value and maximum value for each column in an extent is recorded as a result of scanning the column. Scanning an extent is typically a sub-second operation, with the overhead of persisting the min and max values is less than 2%.

4. The minimum and maximum values are cleared from an extent if the data within that extent, but would be re-populated by the next scan of an extent.

The Extent Map stores minimum and maximum values within each Extent of data that covers 8 million rows. For data that is loaded as it is created (a real-time or near real-time), unique identifier fields and date fields generally follow ascending patterns that allow different min and max values for each extent that allows for elimination of significant blocks.


Scaling

For performanceWhen query response times perform slowly due to the amount and complexity of queries, you add Performance Modules to the system.

For concurrencyAs the number of Calpont InfiniDB system users increases, you can add User Modules to maintainperformance.

The number of User Modules is based on the number of users and the size of intermediate and final result sets.

For data capacityAs data is added to the database, the system can require additional disk arrays. As diskarrays are added, the cache ratio changes and additional user modules are required to efficiently access data. In some cases, additional disk controllers must be added. The total of arrays needed is highly dependent on a customer's clustering of data and the types of queries. The arrays added is probably consistent with the initial ratio.


Chapter 9

Configuring Modules

This chapter describes how to add, remove, and reconfigure Calpont InfiniDB modules. The primary reason for adding and reconfiguring modules is to improve performance or add redundancy.

Additionally, you may want to remove modules not in use or take modules offline that need hardware or software updates.

You can configure modules using the Calpont InfiniDB Console commands. This chapter provides the instructions for both.

Before configuring modules

Before configuring modules, it is highly recommended that you shut down the database and backup up the Calpont System and the database. See “Performing Backup and Recovery” on page 51 for instructions.

Ensure you have the user ID and password for all modules that will be added, removed, or reconfigured. The Calpont default user ID and password are:

NOTE: After adding modules, update the NMS system so that new modules are monitored as part of the system.

While removing modules, the NMS may issue an alert that servers are no longer functioning on the network.

Module IDsModules in the Calpont System are identified by the following 2-character module types followed by a unique number.

UMnn— User Module

PMnn— Performance Module

Configuring Modules 63

For example, the Calpont System defaults the first User Module to UM1. When another User Module is added, it defaults to UM2, and so forth.

Calpont always picks the first available module ID in the series. For example, if you have PM1, PM2, PM4 and you select to add a new Performance Module, Calpont names the new module PM3.

Automatic configurationAutomatic module configuration is less labor intensive when you accept the Calpont defaults. It allows you to configure more modules with fewer commands. Automatic configuration does the following:

Creates module IDs

Adds multiple modules of the same type with one command

Removes multiple modules of the same type with one command

Reconfigures the last module added

Manual configurationYou have the option of manually configuring modules; however, it requires entering more data with additional commands. Manual configuration allows you to do the following:

Create your own numbering scheme adhering to the module identification format previously described in this chapter

Add specific modules one at a time

Remove specific modules one at a time

Reconfigure specific modules not with the reconfigure command, but in a 2-step process. Remove the specific module and then add that module back as a different module type.

Adding modules

Adding Performance or User Modules effectively adds more processing power to the Calpont System. See “Performance Tuning” on page 57 to learn more about which types of modules you can add to increase the performance or concurrency of your system. You can add Director Modules for redundancy.

64 Configuring Modules

WARNING: Before adding a Director Module to the Calpont System, you must install and configure the front-end DBMS on the server.

When Calpont adds multiple modules of the same type with one command, it defaults the module IDs. To add modules using your own modules IDs, you must add modules one at a time.

NOTE: When adding Performance Modules with multiple NICs, you must add the host name for all NICs. If you do not, the add module process will fail with invalid parameters.

To add a module to the Calpont System through the CalpontConsole:

1. Log on to the Calpont InfiniDB Console.

2. Do one of the following:

To accept default module IDs and add multiple modules, type AddModule module_type number_of_modules IP_address_or_host_name (separated by commas) RPM_ package_server root_password

For example, to add two Performance Modules with host names SRVCPT1 and SRVCPT2, with RPM package “calpont-1.0-1.i386.rpm” located in the root directory, type the following:

AddModule PM 2 SRVCPT1, SRVCPT2 /root/calpont-1.0-1.i386.rpm Calpont1

To create IDs manually one at a time, type

AddModule module_ID IP_address_or_host_name RPM_package server root_ password

For example to add one User Module number 1285 with host name SRVCPT1 using RPM package “calpont-1.0-1.i386.rpm” in the root directory, type the following:

AddModule UM1285 SRVCPT1 /root/calpont-1.0-1.i386.rpm Calpont 1


The Calpont.xml is updated to add new modules, the appropriate files are installed to the new modules, and the modules are rebooted.

3. At the prompt, select Yes to start new modules and press Enter.

If you do not want to immediately start new modules, select No and press Enter. See “Starting modules” on page 68 to start modules at a later time.

If the module addition fails, the Calpont InfiniDB Console displays an error message. Additional details are located in the Calpont Log Files on the Director Module.

Removing modules

You can remove modules from the system when they are no longer needed or in the event that they need to be taken offline for hardware updates. You must stop the system before removing modules. See “Stopping the system or modules” on page 11.

You can quickly remove multiple modules from the system by accepting default module IDs. This removes the last modules added to the system.

You can also choose to remove specific modules individually by module ID.

NOTE: You cannot remove the last DM, UM, or PM module.

To remove a module using the CalpontConsole:

1. Log on to the Calpont InfiniDB Console. See “Logging on and off the Calpont InfiniDB Console” on page 1.


To remove the last modules added to InfiniDB, type

RemoveModule module_type number_of_modules

For example, to remove two Performance Modules, type the following: RemoveModule PM 2

To remove a specific module, type RemoveModule module_ID


For example to remove one User Module with module ID UM1285, type the following:

RemoveModule UM1285

Reconfiguring Modules

An important feature of the Calpont System is the ability to tweak performance by reconfiguring existing servers in the Calpont System. To increase concurrency, reconfigure Performance Modules to User Modules. To increase performance, reconfigure User Modules to Performance Modules. For additional information see “Performance Tuning” on page 57.

You must stop the modules before reconfiguring them. See “Stopping the system or modules” on page 11.

If you want to reconfigure a specific module, you can select that module, but Calpont will name the new module with the first available ID in the series.

If you want to reconfigure a specific module and give it a specific numbering scheme, do not use the following steps to reconfigure modules. Instead remove the specific module from the system and then add it back to the system as another module type.

To reconfigure a module using Calpont InfiniDB Console:



Type ReconfigureModule module_type

To reconfigure a User Module to a Performance Module, use the parameter UM. To reconfigure a Performance Module to a User Module, use the parameter PM.

For example to change the last User Module added to the system to a Performance Module, type the following:

ReconfigureModule UM.

Type ReconfigureModule module_ID

2. At the prompt, select yes to configure the module.


If you do not want to immediately start the new module, select No and press Enter. See “Starting modules” on page 68 to start the server at a later time.

If the module reconfiguration fails, the Calpont InfiniDB Console displays an error message. You can find additional details in the Calpont Log Files located on the Director Module.

Starting modules

You can start inactive modules at any time. If you do not know the module IDs of the modules to be started, type getsystemstatus at the Calpont InfiniDB Console to locate inactive modules with their module IDs.

To start a module:


2. Type StartModule module_ID

For example, to start module UM1, type the following:

StartModule UM1


Chapter 10

Operating Mode

Calpont InfiniDB has the ability to support full MySQL query syntax through an operating mode. This operating mode may be set as a default for the instance or set at the ses-sion level.

Please refer to the Calpont InfiniDB SQL Syntax Guide for setting the session level oper-ating mode.

To set the operating mode at the instance level, specify infinidb_vtable_mode on the command line or in my.cnf. If not specified, the default is 1 (distributed mode).

On command line: ~/mysql/libexec/mysqld --xxxxxx --infinidb_vtable_mode=value

In my.cnf file (/usr/local/Calpont/mysql):

[mysqld] xxxxx infinidb_vtable_mode=value

where value is:

0) a generic, highly compatible row-by-row processing mode. Some WHERE clause components can be processed by InfiniDB, but joins are processed entirely by mysqld using a nested-loop join mechanism

1) (the default) query syntax is evaluated by InfiniDB for compatibility with dis-tributed execution and incompatible queries are rejected. Queries executed in this mode take advantage of distributed execution and typically result in higher performance.

2) auto-switch mode: InfiniDB will attempt to process the query internally, if it cannot, it will automatically switch the query to run in row-by-row mode.

NOTE: For more information on supported query syntax for modes 0 and 2, please refer to the MySQL 5.1 SQL Syntax Guide.

Operating Mode 69

70 Operating Mode

Chapter 11

Decimal Scale

Calpont InfiniDB has the ability to support varied internal precision on decimal calculations.

infinidb_decimal_scale is used internally by the InfiniDB engine to control how many significant digits to the right of the decimal point are carried through in suboperations on calculated columns. If, while running a query, you receive the message ‘aggregate overflow’, try reducing infinidb_decimal_scale and running the query again. Note that, as you decrease infinidb_decimal_scale, you may see reduced accuracy in the least significant digit(s) of a returned calculated column.

This decimal scale may be set as a default for the instance or set at the session level.

Please refer to the Calpont InfiniDB SQL Syntax Guide for setting the session level decimal scale.

11.1 Enable/Disable Decimal Scale

To enable/disable the use of the decimal scale at the instance level, specify infinidb_use_decimal_scale on the command line or in my.cnf. The default is 0 (dis-abled).

On command line: ~/mysql/libexec/mysqld --xxxxxx --infinidb_use_decimal_scale=value


[mysqld] xxxxx infinidb_use decimal_scale=value

where value is 0 (disabled) or 1 (enabled).

Decimal Scale 71

11.2 Set Decimal Scale Level

To set the decimal scale at the instance level, specify infinidb_decimal_scale on the command line or in my.cnf. If not specified, the default is 8.

On command line: ~/mysql/libexec/mysqld --xxxxxx --infinidb_decimal_scale=value


[mysqld] xxxxx infinidb_decimal_scale=value

where value is the amount of precision desired for calculations.

72 Decimal Scale

Chapter 12

System Utilities

There are several system utilities that are useful in the operation of Calpont InfiniDB.

configxml.shThe script ‘configxml.sh’ reads and sets values in the Calpont.xml file. It must be run on the active performance module.

To read a value:

#./configxml.sh getconfig SystemConfig SystemName

Current value of SystemConfig / SystemName is caldb01

To set a value: WARNING: Do not make changes to the Calpont.xml file unless directed to do so by Calpont service engineers. Incorrect settings can render your system unusable and will require a service call to correct. Setting a value should be performed when the system is stopped.

Please see the stopSystem and startSystem commands in <??? link ????> for infor-mation on stopping and starting the system.

# ./configxml.sh setconfig SystemConfig SystemName caldb02

Old value of SystemConfig / SystemName is caldb01

/usr/local/Calpont/etc/Calpont.xml backed up to /usr/local/Calpont/etc/Calpont.xml.1254929966

Old value of SystemConfig / SystemName is caldb01

SystemConfig / SystemName now set to caldb02

System Utilities 73

colxmlcolxml creates an XML job file for your database schema before you can import data. You should create a new job file before each bulk load. For more information on colxml, please see the “Generate job file” on page 43.

cpimportcpimport is used to import data into Calpont InfiniDB. You must first create a new XML job file using colxml. For more information on cpimport.sh, please see the “Import data” on page 44.

viewtablelockIf an import is aborted or fails, this utility will allow you to view any current active table locks still held by the import.

cleartablelockAny active table locks still held by an aborted or failed import can be cleared using this utility. The format of this is cleartablelock schemaname tablename.

74 System Utilities

Chapter 13

Troubleshooting

UM memory exceeded error / distributed join performance InfiniDB dynamically determines the table with the largest cardinality for use in its dis-tributed join operation. Using the correct large side table can make a significant dif-ference in how the query performs and, in some cases, whether the query can be run within available resources. There are cases where InfiniDB will not predict the correct large side table. For these cases, the behavior can be overridden using the INFINIDB_ORDERED hint which will use the first table in the from clause as the large side table. For example, the query below will use lineitem as the large side table. select /*! INFINIDB_ORDERED */ count(*) from lineitem, supplier where l_suppkey = s_suppkey and l_shipdate <= '1998-12-31' and s_suppkey <= 10000;

Error in TupleAggregateStep If a query is executed that receives the InfiniDB error “The error came from TupleAggregateStep”, an aggregation overflow has occurred. Modifying the infinidb_decimal_scale value from its default (usually 8 unless an override has been made) may assist in getting the query to execute successfully. Please see the “Decimal Scale” on page 71 for further guidance.

Circular join detected error: If a query is executed that receives the InfiniDB not supported error with a "Circular join detected" warning message, there is likely a cross-table equal comparison involved in the query. InfiniDB cannot handle this statement by default. However, adding a "+ 0" to one side of the cross-table comparision will hint InfiniDB that the comparison is a cross-table equal comparison, rather than a join, and allow the query to be processed.

Troubleshooting 75

For example, the query below receives will not run as written. Example:mysql> select count(*) -> from customer, -> supplier, -> orders, -> lineitem -> where c_custkey = o_custkey and -> l_orderkey = o_orderkey and -> l_suppkey = s_suppkey and -> c_nationkey = s_nationkey and -> o_orderdate >= date '1997-01-01' and -> o_orderdate < date '1997-01-01' + interval '1' year;ERROR 138 (HY000):The query includes syntax that is not supported by InfiniDB.Use 'show warnings;' to get more infomation.Review the Calpont InfiniDB Syntax guide for additionalinformation on supported distributed syntax or consider changing the InfiniDB Operating Mode (calpont_vtable_mode).mysql>mysql> show warnings;+-------+------+-----------------------------------------------------------+| Level | Code | Message |+-------+------+-----------------------------------------------------------+| Error | 9999 | Fatal parse error in vtable mode: Circular join | detected. |+-------+------+-----------------------------------------------------------+1 row in set (0.00 sec)

Adding the + 0 (bolded in the statement below) to the c_nationkey = s_nationkey com-parison will allow the query to run.

mysql> select count(*) -> from customer, -> supplier, -> orders, -> lineitem -> where c_custkey = o_custkey and -> l_orderkey = o_orderkey and -> l_suppkey = s_suppkey and -> c_nationkey = s_nationkey + 0 and

76 Troubleshooting

-> o_orderdate >= date '1997-01-01' and -> o_orderdate < date '1997-01-01' + interval '1' year;+----------+| count(*) |+----------+| 36449 |+----------+

Troubleshooting 77

78 Troubleshooting

Appendix A

GNU Free Documentation License

GNU Free Documentation LicenseVersion 1.3, 3 November 2008

Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. <http://fsf.org/>

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

0. PREAMBLEThe purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

1. APPLICABILITY AND DEFINITIONSThis License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is

GNU Free Documentation License 79

addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by

80 GNU Free Documentation License

proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

The "publisher" means any person or entity that distributes copies of the Document to the public.

A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition.

The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.

2. VERBATIM COPYINGYou may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3. COPYING IN QUANTITYIf you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all


these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

4. MODIFICATIONSYou may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permis-sion.

B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at


least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.

C. State on the Title page the name of the publisher of the Modified Version, as the publisher.

D. Preserve all the copyright notices of the Document.

E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

H. Include an unaltered copy of this License.

I. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Doc-ument as given on its Title Page, then add an item describing the Modified Ver-sion as stated in the previous sentence.

J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network loca-tions given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

M. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version.

N. Do not retitle any existing section to be Entitled "Endorsements" or to con-flict in title with any Invariant Section.

O. Preserve any Warranty Disclaimers.


If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties—for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

5. COMBINING DOCUMENTSYou may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements".


6. COLLECTIONS OF DOCUMENTSYou may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

7. AGGREGATION WITH INDEPENDENT WORKSA compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.

8. TRANSLATIONTranslation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.

If a section in the Document is Entitled "Acknowledgements", "Dedications", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.


9. TERMINATIONYou may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.

However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.

Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.

Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.

10. FUTURE REVISIONS OF THIS LICENSEThe Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Document.

11. RELICENSING"Massive Multiauthor Collaboration Site" (or "MMC Site") means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such


a server. A "Massive Multiauthor Collaboration" (or "MMC") contained in the site means any set of copyrightable works thus published on the MMC site.

"CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.

"Incorporate" means to publish or republish a Document, in whole or in part, as part of another Document.

An MMC is "eligible for relicensing" if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.

The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.


Glossary

ad-hoc queryA customized query that cannot be determined prior to the moment the query is issued. Typically a SQL statement.

business intelligence (BI)The process of receiving business data that is reliable, consistent, understandable, easily manipulated and timely. With this data, business users are able to conduct analyses that yield overall understanding of where the business has been, where it is now and where it will be in the near future. Business intelligence serves two main purposes. It monitors the financial and operational health of the organization (reports, alerts, alarms, analysis tools, key performance indicators and dash-boards). It also regulates the operation of the organization providing two- way integration with operational systems and information feedback analysis.

Calpont InfiniDB ConsoleThe user interface to communicate with the Calpont Stack through commands entered as lines of text from the keyboard.

column-based relational databaseA database that is structured by column instead of by row.

COTS (commercial off the shelf components)Computer products that are available for sale to the general public and designed to be easily installed and compatible with other system components.

cron (Command Run ON)A daemon that is used to execute scheduled tasks. You can use cron to automate tasks to perform periodic backups or data loads.

Glossary 89

data definition language (DDL)A language used by a database management system which allows users to define the database, specifying data types, structures and constraints on the data. Exam-ples are the CREATE TABLE, CREATE INDEX, ALTER, and DROP statements.

data manipulation language (DML)Commands that manipulate data within existing database objects. Examples are the SELECT, UPDATE, INSERT, DELETE statements.

data warehouseA data warehouse is a database geared towards business intelligence require-ments. The data warehouses are updated on a periodic basis and contain historical information that enables analysis of business performance over time.

DDL (See data definition language)

DML (See data manipulation language)

DNS (See domain name server)

domainA set of network addresses that can directly transmit to each other on the same network without using a router.

domain nameThe name given to a specific IP address so that people can recognize and remem-ber it easier.

domain name server (DNS)A program running on a server which automatically translates domain names into their correct TCP/IP addresses.

90 Glossary

gatewayA network point that acts as an entrance to another network. The computers that control traffic within your company's network or at your local Internet service pro-vider (ISP) are gateway nodes.

host nameThe name of a computer that has two-way access to other computers on the Inter-net.

IP addressEvery computer on a network has a unique IP number which is four sets of num-bers divided by period with up to three numbers in each set. (Example: 10.0.0.127) Many computers also have a domain names that is easier to remem-ber.

NIC (network interface card)Computer hardware that provides physical access to a network. Many servers provide more than one NIC so that a server has multiple ports accessing a network.

NMS (network monitoring system)Software that monitors the components in a network.

redundant array of independent disks (RAID)A means of storing data on multiple hard disks that appears to the operating sys-tem to be a single logical hard disk. By placing data on multiple disks, the opera-tions of accessing data is balanced for improved performance.

SSH (secure shell) A set of standards that allows a secure channel between a local and a remote com-puter using authentication methods.

tokenizationThe means of creating smaller strings of input characters to represent larger strings. The smaller strings or tokens use less memory and space.

Glossary 91

92 Glossary

Index

Aactive 15, 16AlarmConfig.xml 31alarms 38auto offline 15, 16

CCalpont Console 15Calpont Platform Software 9Calpont.xml 31, 32case sensitivity 3command repeat option 4ConsoleCmds.xml 31CPU thresholds 23, 36

DDirector Module 65

Ffailed 15, 17

GgetLogConfig 38getsystemstatus 68

Hheartbeat monitoring 34

Mman offline 16, 17memory usage 21

Index 93

NNMS 15, 63

PProcessConfig.xml 16, 31

RRestart ServerRestart System 11

SsetAlarmConfig 38Shutdown ServerShutdown System 12Start ServerStart System 11Stop ServerStop System 11swap space usage 37

94 Index

calpont infinidb administrator guide (for version 1.0.3)

Documents

calpont software

system modules

trademarks of calpont

calpont corporation

viewing system status

calpont infinidb console

system status monitoring

system maintenance