cisco cloud object storage release 3.0.1 user guide€¦ · contents vi cisco cloud object storage...

Cisco Cloud Object Storage Release 3.0.1 User Guide

December 15, 2015

Cisco Systems, Inc.www.cisco.com

Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco website at www.cisco.com/go/offices.

http://www.cisco.com

http://www.cisco.com/go/offices

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

Cisco Cloud Object Storage Release 3.0.1 User Guide

© 2014-2015 Cisco Systems, Inc. All rights reserved.

http://www.cisco.com/go/trademarks

C O N T E N T S

iii

Preface vii

Audience vii

Document Organization vii

Document Conventions viii

Related Publications ix

Obtaining Documentation and Submitting a Service Request ix

C H A P T E R 1 Overview 1-1

Product Description 1-1

COS and MOS 1-1

Components 1-1

Networks 1-2

Platform and Applications Manager (PAM) 1-2

COS Nodes 1-2

COS Cluster 1-3

Object Store Metadata 1-3

Hardware Platforms 1-3

Features 1-3

Overview 1-4

COS Service Manager GUI 1-4

High Availability (HA) 1-5

Swauth API 1-5

Swift Object Store API 1-6

Object Store Metadata Resiliency 1-7

Object Data Resiliency 1-7

Service Load Balancing 1-7

CLI Utilities 1-8

Prerequisites 1-8

Restrictions and Limitations 1-8

iiiCisco Cloud Object Storage Release 3.0.1 User Guide

Contents

C H A P T E R 2 Deploying COS 2-1

Pre-Deployment Tasks 2-1

COS Node Hardware Installation 2-1

PAM Installation and Configuration 2-2

Setting Up the External DNS Server 2-3

Editing NTP Server Information 2-6

Viewing DNS Server Information 2-7

Configuring COS 2-7

Configuring the Service Instance Template 2-8

Configuring IP Pools 2-9

Editing or Deleting IP Pools 2-9

Configuring a COS Cluster 2-10

COS Node Initial Configuration 2-10

Adding a COS Node to a COS Cluster 2-12

Configuring a COS Node 2-12

Configuring the Service Interface of a COS Node 2-13

Configuring Cassandra Database Maintenance 2-14

C H A P T E R 3 System Monitoring 3-1

COS Application Status Monitoring 3-1

COS Node Status Monitoring 3-2

Viewing COS Node Status 3-2

AIC Client Monitoring 3-3

Storage Monitoring 3-3

Interface Monitoring 3-3

Services Monitoring 3-3

COS Management Events 3-3

Viewing COS Node Management Events 3-3

Service Events 3-4

COS AIC Client Events 3-5

COS AIC Server Events 3-6

COS Node Platform Monitoring with SNMP 3-6

Overview 3-6

Installation 3-6

Configuration 3-7

MIB Extensions 3-7

Extensions to HOST-RESOURCES-MIB::hrDeviceTable 3-7

Extensions to HOST-RESOURCES-MIB::hrDiskStorageTable 3-8

ivCisco Cloud Object Storage Release 3.0.1 User Guide

Contents

Extensions to HOST-RESOURCES-MIB::hrStorageTable 3-8

Extensions to UCD-SNMP-MIB::dskTable 3-9

Monitored Items 3-9

A P P E N D I X A Reference Information A-1

COS Service Model A-1

Using the COS Service Manager GUI A-2

Before You Begin A-2

Logging In A-3

Using the Home Page A-3

Nodes Allocated A-4

Alarm Summary A-5

Service Summary A-5

Using the Navigation Panel A-7

Selecting the View A-7

Home Button A-7

Infrastructure A-8

Service Domain Objects A-8

Services A-8

Monitoring A-8

Working with Pages A-9

Adding an Entry A-9

Editing an Entry A-9

Deleting an Entry A-10

Refreshing the Table A-10

Sorting Entries A-10

Changing the Order of Columns A-10

Filtering Contents A-10

Managing GUI Users A-11

COS Network Ports and Services A-11

COS Maintenance A-12

COS Node Reboot A-12

Behavior of COS Services on COS Node Boot A-13

COS Service Reliability A-13

COS Node Disks A-14

COS Services A-14

COS Node Interfaces A-14

Server Reachability A-14

COS Node Hard Drive Replacement A-15

vCisco Cloud Object Storage Release 3.0.1 User Guide

Contents

Replicating Objects During Swift Write Operations A-15

A P P E N D I X B Troubleshooting B-1

Viewing the Statuses of Primary System Services B-1

Service High-Availability B-1

Troubleshooting Swift and Swauth API Errors B-1

Unresponsive COS Service Manager GUI B-2

Viewing COS AIC Logs B-3

A P P E N D I X C COS Tunables C-1

Debug Tunables C-1

Info Tunables C-2

Configuration Tunables C-5

A P P E N D I X D COS Command Line Utilities D-1

Hardware Prerequisites D-1

Installing the CLI Utilities D-1

Using the CLI Utilities D-3

cos swauth D-3

cos-swift D-5

viCisco Cloud Object Storage Release 3.0.1 User Guide

Preface

This preface describes who should read the Cisco Cloud Object Storage Release 3.0.1 User Guide, how it is organized, and its document conventions. It contains the following sections:

• Audience

• Document Organization

• Document Conventions

• Related Publications

• Obtaining Documentation and Submitting a Service Request

AudienceThis guide is for the networking professional managing the Cisco Cloud Object Storage (COS) product. Before using this guide, you should have experience working with Linux platforms, and be familiar with the concepts and terminology of Ethernet, local area networking, clustering and high-availability, and network services like DNS and NTP.

Document OrganizationThis document contains the following chapters and appendices:

Chapters or Appendices Descriptions

Chapter 1, “Overview” Describes the COS, its components, features, and prerequisites for deployment.

Chapter 2, “Deploying COS” Gives procedures for deploying the COS components.

Chapter 3, “System Monitoring” Provides tools and methods for monitoring the COS system and components.

Appendix A, “Reference Information” Describes the COS GUI, additional features, maintenance details, and tools for managing the COS system.

Appendix B, “Troubleshooting” Provides information and procedures for troubleshooting COS problems.

viiCisco Cloud Object Storage Release 3.0.1 User Guide

PrefaceDocument Conventions

Document ConventionsThis document uses the following conventions:

Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual.

Tip Means the following information will help you solve a problem. The tips information might not be troubleshooting or even an action, but could be useful information, similar to a Timesaver.

Caution Means reader be careful. In this situation, you might perform an action that could result in equipment damage or loss of data.

Timesaver Means the described action saves time. You can save time by performing the action described in the paragraph.

Appendix C, “COS Tunables” Describes the Debug, Info, and Configuration tunables available in COS.

Appendix D, “COS Command Line Utilities”

Provides instructions for installing and using the COS CLI utilities.

Chapters or Appendices Descriptions

Convention Indication

bold font Commands and keywords and user-entered text appear in bold font.

italic font Document titles, new or emphasized terms, and arguments for which you supply values are in italic font.

[ ] Elements in square brackets are optional.

{x | y | z } Required alternative keywords are grouped in braces and separated by vertical bars.

[ x | y | z ] Optional alternative keywords are grouped in brackets and separated by vertical bars.

string A nonquoted set of characters. Do not use quotation marks around the string or the string will include the quotation marks.

courier font Terminal sessions and information the system displays appear in courier font.

< > Nonprinting characters such as passwords are in angle brackets.

[ ] Default responses to system prompts are in square brackets.

!, # An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line.

viiiCisco Cloud Object Storage Release 3.0.1 User Guide

PrefaceRelated Publications

Warning IMPORTANT SAFETY INSTRUCTIONS

This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device.

SAVE THESE INSTRUCTIONS

Warning Statements using this symbol are provided for additional information and to comply with regulatory and customer requirements.

Related PublicationsRefer to the following documents for additional information about COS 3.0.1:

• Release Notes for Cisco Cloud Object Storage Release 3.0.1

• Cisco UCS C3160 Rack Server Installation and Service Guide.

• Cisco Cloud Object Storage Release 3.0.1 API Guide

• Cisco Media Origination System Release 2.4 User Guide

• Open Source Used in COS 3.0.1

Obtaining Documentation and Submitting a Service RequestFor information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation at: http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.

Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.

ixCisco Cloud Object Storage Release 3.0.1 User Guide

http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html

PrefaceObtaining Documentation and Submitting a Service Request

xCisco Cloud Object Storage Release 3.0.1 User Guide

C

C H A P T E R 1
Overview
Product DescriptionCisco Cloud Object Storage (COS) provides distributed, resilient, high-performance storage and retrieval of binary large object (blob) data. The primary interface for managing COS content is the OpenStack Swift API, with enhancements that improve the quality of service when accessing large media objects.

Object storage is distributed across a cluster of hardware systems, or nodes. The storage cluster is resilient against hard drive failure within a node and against node failure within the cluster. Nodes may be added to or removed from the cluster as needed to provide for changes in cluster capacity. To administer the cluster, COS includes an HTTP-based cluster-management API.

COS also includes an authentication and authorization service that implements the OpenStack Swauth API.

COS and MOSCOS is designed to integrate transparently with the Cisco Media Origination System (MOS), which is designed for highly optimized ingest and storage. MOS uses a hierarchical storage design that supports huge content libraries while simplifying content storage management. Its distributed architecture can separate ingest and storage from streaming, allowing each function to be scaled independently as needed to dynamically increase network ingest and storage resources. For more information, see the Cisco Media Origination System User Guide, Software Version 2.4.

ComponentsCOS has a number of subsystems.

• Networks — Interfaces are grouped into distinct networks to isolate management functions from high-volume data traffic.

• Clusters and Nodes — COS services are provided by a cluster of nodes, with both the cluster and the individual nodes as distinctly manageable components.

• Object Metadata Store — The metadata for the cluster is stored in a high-performance distributed NoSQL database hosted on the COS nodes in a cluster.

• Platform and Applications Manager (PAM) — COS components are managed using services running on the Platform and Applications Management system (PAM).

1-1isco Cloud Object Storage Release 3.0.1 User Guide

Chapter 1 Overview Components

• Hardware Platforms — COS software is deployed on Cisco Content Delivery Engine (CDE) platforms.

Each of these components and subsystems are described further in the following sections.

NetworksCOS divides network interfaces into two groups: the management network and the data network. The management network is used for monitoring and managing the COS cluster and individual COS nodes. The data network is used by client applications to interact with the COS authentication and authorization services, and the COS object storage services. The client applications use Swauth API to interact with the COS authentication and authorization services, and the Swift API to interact with the COS object storage services.

In customer installations, COS management network traffic can be routed with management traffic meant for non-COS systems. When routing COS data network traffic, however, it must be ensured that high-volume data traffic does not congest networks used to manage COS and other systems.

For this reason, each network adapter in a COS node is assigned to either the management network or the data network. Typically, high-bandwidth network adapters are assigned to the data network. One or two lower-bandwidth adapters are usually configured for use on the management network.

Platform and Applications Manager (PAM)The PAM is a virtual instance dedicated to the installation, configuration, monitoring and recovery of other COS components. It acts as the management machine for the COS system and runs the following services and applications:

• The Platform Manager – It facilitates setting up of networks and configuration of external DNS servers, and hosts the NTP Server

• The Document Server, including MongoDB and Redis

• The COS Service Manager

• The COS Application Instance Controller (AIC), which facilitates management of COS appliances

For more information on the PAM and its components, see the Cisco Media Origination System User Guide, Software Version 2.4.

COS NodesThe COS software runs on a collection of computing systems, called nodes, that are connected via the management and data networks. Currently, there are two types of COS nodes: the cluster controller that runs the PAM software and the COS AIC software, and the storage nodes.

The storage nodes host software that manages object-store and authentication and authorization service metadata, stores and retrieves object contents, and communicates with the cluster controller. COS storage nodes may be added or removed without disrupting COS service availability. Adding nodes is a way of elastically increasing the storage and bandwidth capacity of the COS cluster.

1-2Cisco Cloud Object Storage Release 3.0.1 User Guide

Chapter 1 Overview Features

The COS node software includes a customized Linux distribution, currently based on CentOS 6. This provides the basic framework for the other software applications and modules that run on the node. Each node runs a number of daemons in the Linux user-space. These include daemons to coordinate service log files, to communicate with the cluster controller, to provide a distributed database for object-store metadata, and to communicate with the modules running in the kernel.

Kernel modules are used to provide real-time management of node hardware resources, to provide the distributed, resilient content-store used for object-store data, and to provide the Swift and Swauth API support via the data network. While the management-network interfaces communicate directly with the user-space daemons, the data-network interfaces communicate directly with the kernel modules.

COS ClusterCOS services are provided by software running on a set of nodes called a COS cluster that is connected by data and management networks. Each COS cluster has a single fully-qualified domain name (FQDN) that is used by client applications to access COS services. A COS cluster also has a number of configuration parameters that define the cluster's behavior. Some of these parameters include the Swift and Swauth API constraints, the IP address pools used to assign IP addresses to individual node network adapters, and the IP address of the PAM configuration document server. For a detailed description of the configuration parameters, see Deploying COS, page 2-1.

Object Store MetadataCOS object store metadata and Swauth service data are stored in the high-performance, resilient NoSQL Cassandra database. The cosd daemon running on each COS node acts as the Cassandra client and implements the schema for Swift and Swauth metadata documents stored in Cassandra. Each COS storage node runs an instance of the Cassandra server, so metadata storage capacity increases linearly along with content storage capacity as COS storage nodes are added to the cluster.

Hardware PlatformCOS software can be deployed on the Cisco UCS C-3160 Rack Server equipped with 54 x 4 TB hard disks (216 TB total storage). For information about this platform, see the Cisco UCS C-3160 Rack Server Installation and Service Guide.

Features• Overview, page 1-4

• Cisco UCS C3160 Support, page 1-5

• COS Service Manager GUI, page 1-5

• High Availability (HA), page 1-5

• Swift Object Store API, page 1-6

• Object Store Metadata Resiliency, page 1-7

• Object Data Resiliency, page 1-7

• Service Load Balancing, page 1-8


http://www.cisco.com/c/en/us/td/docs/video/cds/cde/cde205_220_420/installation/guide/cde205_220_420_hig.html


• CLI Utilities, page 1-8

• Multiple Endpoint and Cluster Support, page 1-9

• COS AIC Client Management, page 1-9

OverviewThe table below provides an overview of the COS features.

Table 1-1 Overview of COS Features

Feature Set Features

Cisco UCS C3160 Support • COS 3.0.1 supports installation on Cisco UCS C3160 Rack Servers configured with 54 x 4 TB hard disks (216 TB total storage).

COS Service Manager GUI • Lets you quickly and easily access many COS deployment and monitoring functions.

• Displays storage, network bandwidth, and session count for individual COS nodes and services.

• Supports configuration of the COS node service interface from the Service Manager GUI.

High Availability (HA) • COS supports HA as implemented in MOS, providing redundancy for the PAM VMs. The PAM uses both Cisco and third-party components to support HA.

Swauth API • Provides a simple Auth Service API for authentication of Swift operations.

• Based on Swauth Open-Source Middleware API.

• Can be used to manage accounts, users, and account service endpoints.

Swift Object Store API • An implementation of a subset of the continually evolving OpenStack Swift API.

• Command executions are authenticated using auth tokens provided by Swauth service.

• Used to create and manage containers and objects for persistent storage in a COS cluster.

Object Store Metadata Resiliency • Metadata resiliency is provided by a distributed and replicated Cassandra document database.

• Each COS node participates in the persistence of a subset of the Cassandra database.

• Manual administrative intervention required upon node failure.

Object Store Data Resiliency • Data is resilient to both hard drive and COS node failures.

• Local COS node data resiliency provided by local software RAID.

• COS cluster data resiliency provided by object replication.



Cisco UCS C3160 SupportThe C3160 is a modular, high-density server for service providers, enterprises, and industry-specific environments. The C3160 combines highly scalable computing with high-capacity local storage. Designed for cloud-scale applications, the C3160 is simple to deploy and is well suited for use in unstructured data repositories, media streaming, and content distribution applications.

The C3160 is a 4RU server. When configured for COS, the C3160 includes a pair of solid-state drives (SSDs) in RAID1 for operating system and COS installation, 54 hard disks in JBOD providing 216 TB total storage, two SSDs serving as RAM (supplementing additional internal RAM), and a total of four 10 GbE ports. For more information, see Deploying COS, page 2-1.

COS Service Manager GUIThe COS Service Manager GUI enables you to quickly and easily configure the COS infrastructure, service domain objects, and services. The GUI also provides valuable monitoring functions, including alarm and alarm history display, system and service diagnostics, event display, log analysis and display, and configuration display.

For information on launching an navigating the GUI, the COS Service Manager GUI section of Reference Information, page A-1.

High Availability (HA)The PAM has two classes of components for HA:

• Third party components, such as ZooKeeper, MongoDB, and Redis, use their own proprietary clustering and redundancy schemes.

• Cisco components, such as the COS Service Manager GUI and DocServer, use ZooKeeper for leader election.

In an HA environment, multiple PAM VMs provide redundancy for the applications that run on the PAM. HA requires at least three PAM VMs, because applications such as ZooKeeper, MongoDB, and Redis require at least three components to form a working cluster.

Service Load Balancing • COS cluster load balancing is provided by DNS round-robin of a FQDN to multiple physical IPv4 addresses hosted by COS nodes.

• Optimal load balancing is provided by extensions to the Swift API through the implementation of HTTP redirect.

Command Line Interface (CLI) Utilities

• COS provides CLI utilities for access to the Swift and Swauth APIs from the command line.

Multiple Endpoint and Cluster Support

• Each COS endpoint can have its own COS service instance, cluster, and asset redundancy policy.

COS Application Instance Controller (AIC) Client Management

• The COS AIC Client process is monitored and, if found not running, is restarted automatically.

Table 1-1 Overview of COS Features

Feature Set Features



Many of these applications also require a majority in order to form a quorum. That is, a cluster of three components can recover from the failure of a single component, because there are still two components to form a majority. But if two components fail, the single remaining component is not a majority, and the cluster cannot recover until one of the failed components recovers.

Therefore, we recommend that you configure more than three PAM VMs to ensure recovery in the event of multiple failures and to support high performance, especially the sharing of databases and other applications.

Swauth APICOS includes a basic authentication service that can be used when COS is not installed along with other OpenStack services such as the Keystone Identity service. The API for the COS authentication service is derived from the OpenStack Swauth middleware component API. The authentication service API provides the following functions for managing accounts, users, and service endpoints:

• Listing Accounts

• Retrieving Account Details

• Creating an Account

• Deleting an Account

• Creating or Updating a User

• Retrieving User Details

• Deleting a User

• Creating or Updating Account Service Endpoints

• Getting an Authentication Token

For a detailed description of these functions, see the Cisco Cloud Object Storage Release 3.0.1 API Guide.

Swift Object Store APIThe COS object storage API is based on the OpenStack Swift API. It is implemented as a set of Representational State Transfer (ReSTful) web services. All account, container, and object operations can be performed with standard HTTP calls. The requests are directed to the host and URL described in the X-Storage-Url HTTP header that is part of the response to a successful request for an authentication token.

The COS object storage API defines restrictions on HTTP requests. These restrictions, borrowed from the Swift API, are listed in the table below.

Table 1-2 COS API Restrictions

Constraint Value

Maximum # of HTTP Headers per request 90

Maximum length of all HTTP Headers 4096 bytes

Maximum length per HTTP request line 8192 bytes

Maximum length of container name 256 bytes

Maximum length of object name 1024 bytes



Also, the container and object names must be UTF-8 encoded and then URL-encoded before inclusion in the HTTP request line.

Note All the length restrictions are enforced against the URL-encoded request line.

The COS object store API provides the following functions, some of which provide extended functionality beyond the standard SWIFT API defined by OpenStack:

• Listing Containers

• Listing Objects

• Creating a Container

• Deleting a Container

• Retrieving an Object

• Creating or Updating An Object

• Deleting an Object

• Creating or Updating Container Metadata

• Retrieving Container Metadata

• Deleting Container Metadata

• Retrieving Object Metadata

For a detailed description of these functions, see the Cisco Cloud Object Storage Release 3.0.1 API Guide.

Object Store Metadata ResiliencyCOS stores metadata for Swift and Swauth accounts, users, containers, and objects as documents in a Cassandra database instance. Cassandra is a distributed document store. In a typical multi-node Cassandra cluster, no single node persists (saves) to local disk a copy of the entire database. Instead, each Cassandra cluster node locally persists a subset of the database. To ensure resiliency of the data in case of node failure, Cassandra has configuration options to specify the number of document replicas to maintain on separate cluster nodes.

For metadata resiliency in COS, each COS cluster node participates in the Cassandra cluster, and each COS node locally persists a part of the Cassandra database. The database cluster is automatically configured to create document replicas to be resilient to a single node failure.

If a second node fails before full metadata resiliency is restored or before full content resiliency is restored, there is a risk of data loss.

Object Data ResiliencyThe COS stores Swift object data to local disk or to external facing hard drives. To maintain data resiliency in the event of a loss of a local hard drive, the COS system, by default, uses erasure coding techniques (software RAID) to stripe the object data across multiple local drives. The default scheme for erasure coding is to maintain two parity blocks for twelve data blocks. This ensures that data is resilient to the loss of two local hard drives. In the event of a hard drive loss, the COS system immediately begins



to recover the data from the lost drive and replace the missing data blocks on surviving hard drives. The execution of this recovery process, including the duration, is scheduled with low priority, and depends on the availability of system resources and available disk space.

For data resiliency in the event of a COS node failure, the COS cluster can be configured to maintain copies of object data on one or more additional COS nodes. Recommended practice is to configure the COS cluster to maintain at least two copies of object data for resiliency. When configured for multiple object copies, the COS cluster automatically attempts to create the configured object copy count within the cluster in the event of a COS node failure, without manual administrative action. As soon as the COS cluster detects a node failure, the cluster begins to create additional copies of objects that were maintained on the failed node. Upon restoring the failed COS node, the COS cluster purges unnecessary copies to recover disk space.

Service Load BalancingThe COS cluster is composed of COS nodes. Each node has limited CPU, network, and disk resources. To ensure the best performance and quality of service, the workloads of the Swift and Swauth operations must be distributed among the COS nodes. The recommended solution for service load balancing is to use a DNS system to round-robin clients to different physical IP addresses hosted by the various COS nodes. While not a perfect solution, such a DNS round-robin should provide sufficient distribution of workloads across the COS nodes.

In addition to using DNS to distribute workload, the COS Swift implementation supports intelligently redirecting a Swift client to an optimal location for Swift object create and read operations using standard HTTP redirect semantics. Given a Swift client that supports HTTP redirect semantics, the client can provide an X-Follow-Redirect: true HTTP header in the HTTP PUT and GET requests for Swift object create and read operations. In the event that a more optimal location should be used for the operation, the COS node will respond with an HTTP 307 (temporary redirect) status, indicating to the client where the operation should be requested.

For Swift object read operations, COS provides two levels of service and transfer profile: best-effort and committed rate. These levels of service contribute to service load balancing. COS provides extensions to Swift object read that allows the client to request a guaranteed and committed transfer rate as the data is sent from the COS node. A COS node can reject a read request if the client has requested a committed rate transfer, and the COS node does not have sufficient resources available to satisfy the client's request. If a client does not request a committed rate transfer, the COS node will attempt to satisfy the request with the system resources available, and at a priority lower than that of any in-progress committed rate requests. For more information, see the Cisco Cloud Object Storage Release 3.0.1 API Guide.

CLI UtilitiesCOS provides the following command line utilities for use on Linux:

• cos-swift — provides command-line access to the Swift API.

• cos-swauth — provides command-line access to the Swauth API.

Note These utilities do not work between two COS nodes or between a COS node and a local node, as the HTTP request will be refused.

For more information on the COS command line utilities, see COS Command Line Utilities, page D-1.


Chapter 1 Overview Prerequisites

Multiple Endpoint and Cluster SupportEach COS endpoint can have its own COS service instance, cluster, and asset redundancy policy. Each COS endpoint can be enabled and disabled individually and dynamically, and each has its own AppStatus message for reporting service level agreement (SLA) status.

If an endpoint is enabled or disabled, only the network interfaces of the COS nodes attached to the endpoint or cluster are added to or removed from the DNS.

COS AIC Client ManagementThe COS AIC Client process is monitored by the monit process that runs on each COS node. The AIC Client process creates a PID file that is added to the monit script so that it can be monitored and restarted automatically if the monit process discovers the AIC Client process not running.

Command line scripts are also available to stop and restart the AIC Client process manually, bypassing the automatic restart process.

PrerequisitesThe COS management and configuration operations require specific hardware components for deployment. For more information on the hardware requirements, see the Cisco Media Origination System User Guide, Software Version 2.4.

The COS system is most effective in engineered networks, with separate routes for management and data flow. In designing and provisioning networks, capacity for the high data-network throughput for the expected application of COS must be ensured. Also, the high data traffic generated by the COS systems must not interfere with the management network segment or other important network segments.


Chapter 1 Overview Restrictions and Limitations

Restrictions and Limitations• The COS does not support IPv6.

• The OpenStack Swift and Swauth APIs continue to evolve. COS does not currently implement all the Swift or Swauth API functions. For a list of supported functions, see Swift Object Store API and Swauth API in this chapter.

• Secure Sockets Layer (SSL) or other means for providing session security and encryption are not supported with the Swift and Swauth APIs.

• See the Release Notes for Cisco Cloud Object Storage 3.0.1 for open caveats and known issues related to this release of the COS software.


C

C H A P T E R 2
Deploying COS
To deploy the COS 3.0.1 software, perform the following tasks in order:

1. Pre-Deployment Tasks, page 2-1

2. COS Node Hardware Installation, page 2-1

3. Configuring a COS Node for Data Mover, page 2-18

4. Configuring COS, page 2-26

Pre-Deployment TasksBefore deploying the COS, you must complete the following preliminary tasks:

Step 1 Prepare your COS VMware datacenter topologies and networks. The COS requires the following networks:

• Management — This is the primary COS network that connects all of the components.

• Cache Fill — This is the data network. It connects the COS Appliances.

Step 2 Download the COS Service Manager Open Virtual Appliance (OVA) file and load the image into a repository (HTTP server) that is accessible by vCenter.

Step 3 Determine the blade and VM layout that you will use, and your management IP address allocation scheme for the network interfaces.

Step 4 Install the external DNS and NTP servers.

• Determine the origin service FQDN and prepare the downstream client (client-facing) DNS servers to point to the COS Appliance dataplane (primary interface) IP addresses.

COS Node Hardware InstallationCOS 3.0.1 software can currently be deployed on the Cisco UCS C3160 Rack Server. For information about installing the hardware, see the Cisco UCS C3160 Rack Server Installation and Service Guide.

The instructions in this section apply only to COS installation on the Cisco UCS C3160 Rack Server. Before you begin, be sure that you have the following:

• C3160 server installed per manufacturer instructions

• Cisco Integrated Management Controller (CIMC) Connection to the server


Chapter 2 Deploying COS COS Node Hardware Installation

• ISO image of the COS Software

C3160 Hardware OverviewThe C3160 is a 4RU server. When configured for COS, the C3160 includes:

• 2 solid state drives in top slots 55 and 56, configured as RAID 1, to receive the operating system and COS installation

• 54 hard disk drives in top slots 1-54, configured as JBOD with 216 TB total capacity (4 TB per drive)

• 2 SSD drives in rear-loading slots (RAM1)

• Four 10 GbE ports, two per SIOC

• RAM2

C3160 Network Requirements The C3160 provides four 10 GbE ports for video and management traffic. To separate the management traffic from video, port 1 on each storage I/O controller (SIOC) must be divided into two vLANs, with the management vLAN limited at 1 Gbps and the video vLAN limited at 9 Gbps. You must provide the appropriate vLAN IDs defined by the switch to which these ports connect.

Although not essential, the second 10 GbE port on each SIOC should be configured as a vLAN, limited at 10 Gbps, and assigned to the same video vLAN ID as the 9 Gbps vLAN. The vLAN are defined in hardware by virtue of the modular LAN on motherboard (mLOM) of the virtual interface card (VIC) attached to the SIOC. The vLANs appear to Linux as physical NIC ports.

Figure 2-1 provides a view of the network architecture of a COS cluster based on the UCS C3160.



Figure 2-1 C3160 COS Cluster Architecture

Configuring VLANs

You must configure the first downstream switch to receive the traffic from these two vLAN IDs and route the traffic appropriately. A Bash script named setup_’UCSC-C3160_vLAN.sh’ is provided to help configure the vLANs. You can download this script from the same Cisco download page that contains the COS product ISO package.

This script is executed from a remote Linux node with network access to the C3160 CIMC. Execution of the script occurs after the C3160 CIMC IP address is defined but before COS is installed.

This script requires the following input parameters:

• CIMC IP address

• CIMC username and password



• Management vLAN ID

• Video/data vLAN ID

When it runs, the script prompts for any parameters missing from the command line. Password values entered in response to such a prompt are not echoed to the screen.

Example./setup_UCSC-C3160_vLAN.sh -c 10.10.10.10 -u admin -m 42 -d 2001Password:

Following COS installation, this script can be used to reset the C3160 VICs to the COS configuration, provided that the operating system network is set up properly and the CIMC has an IP address.

Installation ProceduresInstallation of COS on the C3160 involves the following procedures, which must be performed in the prescribed sequence:

1. Prepare the C3160 for COS installation

2. Install the COS ISO image on the C3160

3. Configure the COS node network on the C3160

4. Configure the DNS resolver

5. Restart the network service

6. Configure the NTP service

7. Configure Cassandra for COS

8. Configure the COS daemon

9. Configure Cserver

10. Verify that all services are running

11. Verify log files creation

12. Create user accounts and verify node/cluster health

This section describes these steps in detail.

Prepare the C3160 for COS Installation

Step 1 Remove any hard drives (up to four possible) inserted from the rear of the C3160.

Step 2 Connect the CIMC RJ45 port on SIOC1 (on the left from the rear) to your management network.

Step 3 Connect the 10 Gbps fiber connections to your video switch.

Note Your video network switch should already be configured to support the video and management vLAN IDs that will be created on port 1 of SIOC1.

Step 4 Connect the power cords to the C3160.

Step 5 Temporarily attach a monitor and keyboard to the C3160.



Step 6 Apply power to the C3160 and press F8 early in the power-up sequence to enter Cisco CIMC Configuration.

Step 7 Under NIC Properties, set the CIMC IP, mask, and gateway as appropriate for your installation.

Step 8 Press F10 to save your changes, then press F5 to refresh the screen and confirm that the IP address is set properly.

Step 9 Press ESC to continue with the CIMC configuration (the system will not yet reboot).

Step 10 Configure the vLANs using the setup_UCSC-C3160_vLAN.sh script as described in Configuring VLANs, page 2-3.

Install the COS ISO Image on the C3160

Step 1 Attach the COS installation ISO file using one of the following methods:

• Connect an external DVD drive at any USB port on the C3160.

Note When using an external DVD drive, installation may fail with missing files reported. If this occurs, the likely cause is that the DVD drive is drawing excessive current from the USB port of the C3160. To work around this issue, use a DVD drive that has its own AC power adapter.

• From the CIMC web page, navigate to Server > Remote Presence > Virtual Media > Add New Mapping and create a virtual media device.

Note Experience has shown that using the HTTP protocol produces the best results.

After the virtual media mapping is defined, select the CIMC BIOS > Config Boot Order option, define a CIMC Mapped DVD subtype mapping, and place it in boot position 1.

• Use the CIMC Java KVM Concole Virtual Media menu option to create a virtual media device associated with a COS ISO found on your desktop computer. After the virtual media mapping is defined, select the CIMC BIOS > Config Boot Order option, define a KVM Mapped DVD mapping, and place it in boot position 1.

Step 2 Reboot the C3160 and confirm that it boots from the COS ISO image.

Note If the boot order was not properly set before rebooting, press F6 early in the C3160 power-up sequence to select the boot device.

Installation is automatic, and typically requires 10-15 minutes to complete from DVD (installation from virtual media is typically slower).

Note The default system baud rate is 9600. To define a different baud rate, enter auto ks_baud_rate=<rate> at the installation boot: prompt before the 10 second auto-installation timeout occurs.

Step 3 When the Reboot message appears, disconnect the installation DVD drive or disable the virtual installation media, and then press the Enter key to reboot the C3160.



Configure the COS Network on the C3160

Step 1 When the C3160 finishes rebooting, log in to the console using the following credentials:

• Username — root

• Password — rootroot

The COS initialization (cosinit) script launches automatically to set up the management network that will be located on eth0.

Executing cosinit for platform configurations only

ATTENTION!!!

cosinit script should be run only to configure the device after an image installation.This script modifies the network and other critical configurations based on the deployment type.

Improper use of this script may result in mis-configuring the device or making it inaccessible.

If a new image is installed on this server, a reboot is required before running cosinit.If a reboot is already performed, please continue. Otherwise, please exit and executei cosinit after rebooting the server

Do you want to continue ? (yes/no) [y]: y

Note If you exit the cosinit script, you can relaunch it manually from the /opt/cisco/cos/config folder.

Step 2 When asked if you want to continue, enter y and configure the management interface as follows (IP addresses shown below are examples only):

Enter management interface [eth0]:Please ensure an IP address and netmask are configured formanagement interface eth0:

Select option 1 to configure Management InterfaceSelect the option for 'Done' to exit this menu:1. Configure management interface (eth0)2. DoneChoice [2]: 1Do you want to disable interface eth0? (yes/no) [n]: nEnter an IP address for eth0: 10.10.10.20Enter a netmask for eth0: 255.255.255.0Enter a broadcast address for eth0 [10.10.10.255]:

Select option 1 to configure Management InterfaceSelect the option for 'Done' to exit this menu:1. eth0 ip:10.10.10.20 mask:255.255.255.0 bcast:10.10.10.255 2. DoneChoice [2]: 2Backing up old scripts in /etc/sysconfig/network-scriptsWriting new ifcfg-ethX scripts

Note If you exit the cosinit script, you can relaunch it manually from the /opt/cisco/cos/config folder.

Step 3 Continue the script by entering the hostname of the node and the details of the default gateway as follows (IP addresses shown below are examples only):



Enter a hostname [localhost.localdomain]: COS-Node-1Enter the number of the eth interface that connects to the gateway: 0Enter the default gateway IP address [10.10.10.1]: 10.10.10.1Backing up /etc/sysconfig/networkWriting new /etc/sysconfig/networkBacking up /etc/hostsWriting new /etc/hostsRestarting network services, this may take a minute:Shutting down loopback interface: [ OK ]Bringing up loopback interface: lo: Disabled Privacy Extensions[ OK ]Bringing up interface eth0: ADDRCONF(NETDEV_UP): eth0: link is not readyigb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RXADDRCONF(NETDEV_CHANGE): eth0: link becomes ready[ OK ]Network services restarted; may take a few seconds to establish connectivityReboot for hostname changes to take effectNetwork configuration complete

Step 4 Ping the gateway to validate the configuration:

[root@localhost ~]# ping 10.10.10.164 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms64 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms64 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms64 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms

Step 5 Edit /etc/cods.conf, uncomment the db host field (about two pages down), and specify the IP address of the local management interface as the DB Host.

db host <mgmt ip>

Step 6 Reboot the COS node.

[root@localhost -]# reboot

Step 7 Configure IP Pools as described in Configuring IP Pools, page 2-27.

Initialize the COS Node - PAM Configuration

Step 1 Provided that cosinit ran in an earlier procedure, you can now execute cosinit to skip the network configuration and begin PAM configuration, as shown in the following example:

[root@Colusa-69 config]# ./cosinit -skipnw

ATTENTION!!!cosinit script should be run only to configure the device after an image installation.This script modifies the network and other critical configurations based on the deployment type. Improper use of this script may result in mis-configuring the device or making it inaccessible.

If a new image is installed on this server, a reboot is required before running cosinit.If a reboot is already performed, please continue. Otherwise, please exit and execute cosinit after rebooting the server

Do you want to continue ? (yes/no) [y]:Use detected platform: C3160-R1 ? (yes/no) [y]:Configuring the BAUD rate to 9600 ...=====================================================IP Pool Configuration



=====================================================Enter IP Pool name (Hit 'Enter' to skip IP Pool configuration): colusa-69

Please select the interfaces you want to add to colusa-69:[ ] 1. eth1[ ] 2. eth2[ ] 3. eth4[ ] 4. eth55. Done

Choice [5]: 1

Note The IP pool name should be the same as the one created in the COS Service Manager GUI.

Step 2 Enter the appropriate IP address of the COS Service Manager PAM, as follows:

• For a single PAM, enter the IP address as the docServer address.

• For multiple PAMs in an HA environment, enter the fully qualified domain name (FQDN) of the docServer; for example, mgmt-docserver.cos88.cisco.com.

Note The docserver port is 5087.

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[ ] 2. eth2[ ] 3. eth4[ ] 4. eth55. Done

Choice [5]: 2

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[*] 2. eth2[ ] 3. eth4[ ] 4. eth55. Done

Choice [5]: 3

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[*] 2. eth2[*] 3. eth4[ ] 4. eth55. Done

Choice [5]: 4

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[*] 2. eth2[*] 3. eth4[*] 4. eth55. Done

Choice [5]: 5Enter IP Pool name (Hit 'Enter' to complete IP Pool configuration):Enter Cluster Name:=====================================================



DocServer Configurations=====================================================Enter DocServer IP or FQDN: 172.22.116.35Enter DocServer Port [5087]:Backing up exisitng /tmp/.cosnodeinit.../tmp/.cosnodeinit generated successfullyShutting down monit: [ OK ]Starting monit: Monit start delay set -- pause for 120s [ OK ]Shutting down cos-aic-client: [ OK ]monit: Cannot connect to the monit daemon. Did you start it with http support?Starting cos-aic-client: [ OK ]monit: Cannot connect to the monit daemon. Did you start it with http support?Stopping syslog-ng: [ OK ]Starting syslog-ng: [ OK ]

cosinit finished successfully. Please reboot the box!!!

Execution of cosinit results in the following:

• The file /etc/sysconfig/network-scripts/ifcfg-eth0 is generated, containing the management interface configuration.

• The file /etc/sysconfig/network is generated, containing the hostname and default gateway.

• The file /etc/hosts is generated, containing the host information.

• The network is restarted.

• The file /tmp/.cosnodeinit is created, reflecting the configuration specified in cosinit.

• The file /boot/grub/grub.conf is updated, reflecting the current baud rate.

• The COS AIC client service (cos-aic-client) is started.

Step 3 Confirm that the COS AIC client service is started as follows:

[root@Colusa-69 config]# service cos_aicc statuscos-aic-client is running

The COS node is now registered with COS Service Manager.

Configure the DNS Resolver

The DNS Resolver file, resolv.conf, is added automatically by cos-aid-client after cosinit is run. Do not edit the file resolve.conf manually.

Restart the Network Service

Step 1 Restart the network service as follows:

[root@Colusa-69 config]# service network restartShutting down interface eth0: [ OK ]Shutting down loopback interface: [ OK ]Bringing up loopback interface: [ OK ]Bringing up interface eth0: [ OK ]

Step 2 Confirm that you can now ping the router and other servers in your network.



Configure the NTP Service

Step 1 Do not edit the NTP Server file manually. This file is updated automatically by cos-aic-client after cosinit is run.

Step 2 Enable and start the NTP service as follows:

[cos-node@ root]# chkconfig ntpd onStarting ntpd: [ OK ]

Configure Cassandra for COS

Step 1 Enable the syslog service to be started upon reboot:

[cos-node@ root]# chkconfig syslog-ng on

Step 2 Ensure that the host name is set properly (IP addresses shown are examples only):

[cos-node@ root]# cat /etc/hosts

# Lines below this one modified by cosnetconfig127.0.0.1 localhost.localdomain localhost10.168.5.18 cos02.cisco.com::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

Step 3 Create the directory for the Cassandra database:

[cos-node@ root]# mkdir /arroyo/db/cassandra[cos-node@ root]# chown cassandra:cassandra /arroyo/db/cassandra

Step 4 Validate the proper ownership:

[cos-node@ root]# ls –latotal 36drwxr-xr-x. 4 root root 4096 Nov 20 08:13 .drwxr-xr-x. 6 root root 4096 Nov 20 07:38 ..drwxr-xr-x 5 cassandra cassandra 4096 Nov 20 08:13 cassandradrwx------. 2 root root 16384 Nov 20 07:33 lost+found

Step 5 Do not edit the cassandra configuration file (/etc/cassandra/conf/cassandra.yaml) manually. This file is updated automatically by cos-aic-client after the COS node is added to a cluster.

Step 6 Stop the cosd service, then temporarily stop cserver and cosd from starting automatically.

Note Cserver must be started after Cassandra.

[cos-node@ root]# service cosd stop[cos-node@ root]# chkconfig cserver off[cos-node@ root]# chkconfig cosd off

Step 7 Start the Cassandra service:

[cos-node@ root]# service cassandra start

Step 8 Verify that the Cassandra service was successfully started:

[cos-node@ root]# service cassandra status



Step 9 Confirm that the Cassandra is running as follows:

Note The number of the lines in the output is directly related to the number of nodes in the cluster (single node – one line, two nodes – two lines, etc…)

[cos-node@ root]# nodetool status

Datacenter: datacenter1=======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Tokens Owns (effective) Host ID RackUN COS_NODE_1_IP 117.35 KB 256 100.0% bf862009-40eb-474a-a86b-1700c724755a rack1UN COS_NODE_2_IP 768.4 KB 256 100.0% 39e62fbf-d0fb-4a92-9eb9-869b488ed5af rack1

Step 10 Apply the cos scheme in the Cassandra database as follows (IP addresses shown are examples only):

[cos-node@ root]# cqlsh 10.168.5.18 -f /opt/cisco/cos/config/cos.cql

Note • Before running this command, ensure that the scheme is correctly configured for the number of nodes in the cluster. Edit the cos.cql file and update the value of DC according to number of nodes in the cluster (one node – DC: 1, two nodes – DC: 2, three nodes and above – DC: 3), save the file, and then execute the command.

• When creating a cluster, it is necessary to run this command only on one node.

Step 11 After verifying that the command completed successfully, enable Cassandra on startup:

[cos-node@ root]# chkconfig cassandra on

Configure the COS Daemon

Step 1 Edit the file /opt/cisco/cos/etc/cosd.conf by uncommenting and setting desired values for the following:

• auth prefix AUTH_

• auth admin user .super_admin

• auth admin key rootroot

• cluster name

• cluster url

• auth token duration

Note The cluster url is the storage URL, and must be configured per local settings.

Sample Configuration File (All IP addresses are examples only)

[cos-node@ root]# cat /etc/cosd.conf

# cosd.conf.default#



# Copyright (c) 2013-2014 by Cisco Systems, Inc.# All Right Reserved.# The “auth prefix” distinguishes the built-in COS authentication service# from other authentication service providers.auth prefix AUTH_# The “auth admin user” is the user accoutn that has all administrative# privileges for the built-in COS authentication service.auth admin user .super_admin# The “auth admin key” is the authentication key (password) for the# auth admin user.auth admin key rootroot# The “cluster name” is the name of this cluster.cluster name cluster.ca.01# The “cluster url” is the URL for the built-in COS authentication service# for this cluster. (The default value is suitable only for single-server# clusters.)cluster url http://auth01.domain.net/v1# The “auth token duration” specifies the length of time in seconds that# an authentication token will be valid. The validity period begins# at the time the token was initially created. The default duration is# 24 hours.auth token duration 86400# The “log facility” specifies the syslog “faility” that will be# associated with COS daemon entries. The default value is that used for# system daemons.#log facility 3# The “log priority” specifies the maximum log-entry priority level that# will be logged. (Lower priority values are associated with more urgent# log entries.) The default value specifies that “notice”-priority and# lower will be logged. (Debug- and info-priority entries are excluded# by default.)#log priority 5# The “db host” value specifies a list of hosts:ports separated by “;”# to the database administration port of the cluster. (ex:# “host1;host2:9000;host3” would try to connect to host1 on port 8091,# if that fails it’ll connect to host2 on port 9000 etc). The hostname# may also be specified as a URI looking like: http://localhost:8091/poolsdb host 10.168.5.18# The “db user” specifies the database user name for COS daemon# transactions.#db user default# The “db password” specifies the database password for COS daemon# transactions.#db password <none># The “db bucket” specifies the database “bucket” name used to store# COS metadata#db bucket default# The “db object view” specifies the database view used to list containers# or objects, and to count the objects and content bytes subordinate to# accounts and containers.#db name view_design/cos/_view/object

Step 2 Start the cosd service:

[cos-node@ root]# service cosd start



Configure Cserver

Step 1 Create and configure the CServer setup file:

[cos-node@ root]# vi /arroyo/test/setupfile

When configuring Cserver, keep the following in mind:

• When building a cluster, the serverid value changes from server to server.

• The enic numbering is the numbering by which the Linux kernel recognizes the vNICs.

• Remember to update the number of disks.

Sample Setup File

In the sample file below:

• RAID is enabled in the node. To move from RAID to replication, set allow vault raid to 0. For local replication, add the line vault local copy count 1, where the number 1 represents the number of copies COS holds for the objects. For mirroring between nodes in a cluster, add the line vault mirror copies 1, where the number 1 represents the number of copies COS holds for the objects.

• For the enics that the management VLAN was derived from, it is also necessary to configure rate limiting on the CServer side. In the line that configures enic 2 below, erl 1 is added to indicate that this enic is rate limited.

# CServer core configuration. Changes to this file require a server reboot.serverid 1

groupid 3333 arrayid 6666 streamer 0 vault 0 cache 0 bypass_isacheck 1 cos 1 #e1000e adapters: maxrate 975 #igb adapters: maxrate 975 #ixgbe adapters: maxrate 9850 enic adapters: egress rate limited maxrate 8850 enic 2: streaming 1 fill 1 ip 18.60.0.101 tport 0 cport 0 tgid 0 erl 1 enic adapters: maxrate 9850 enic 3: streaming 1 fill 1 ip 18.60.0.102 tport 0 cport 0 tgid 0 enic 4: streaming 1 fill 1 ip 18.60.0.103 tport 0 cport 0 tgid 0 enic 5: streaming 1 fill 1 ip 18.60.0.104 tport 0 cport 0 tgid 0 allow vault raid 1 allow tcp traffic 1 disks 58 model test 4 er_enable 0 rtp_enable 0

Step 2 Configure the CServer subnet table, /arroyo/test/SubnetTable.

Note Skip this step if the server is not part of a cluster.

On every COS node, create /arroyo/test/RemoteServers and add the cache-fill IP addresses of the other COS nodes.

Syntax

network COS_NODE_NETWORK netmask COS_NODE_NETMASK gateway COS_NODE_GATEWAY



Sample SubnetTable For Adding 2 Remote Servers

remote server id <remote server id x> ip <cache-fill ip1> ip <cache-fill ip2> ip <cache-fill ip3> ip <cache-fill ip4> end remote server id <remote server id y> ip <cache-fill ip1> ip <cache-fill ip2> ip <cache-fill ip3> ip <cache-fill ip4> end remote server

Step 3 Start CServer for the first time, using the -C option to initialize cache drives for the first time:

[cos-node@ root]# /arroyo/test/run -C

Note The log rotate scripts are not installed, so keep an eye on the size of the /arroyo/log partition and clean up as needed.

If CServer was successfully started, re-enable cosd and CServer on startup:

[cos-node@ root] chkconfig cosd on[cos-node@ root] chkconfig cserver on

Step 4 Reboot the server.

Once the machine is up, continue to the next step.

Verify That All Services are Running

Step 1 Verify that cassandra is running:

[cos-node@ root] nodetool statusDatacenter: datacenter1=======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Tokens Owns (effective) Host ID RackUN 10.168.5.2 117.35 KB 256 100.0% bf862009-40eb-474a-a86b-1700c724755a rack1UN 10.168.5.18 768.4 KB 256 100.0% 39e62fbf-d0fb-4a92-9eb9-869b488ed5af rack1

Step 2 Verify that cosd is functional:

[cos-node@ root] curl –v http://fqdn/info

{"cluster":{"config_ver":2,"fqdn":"cos-utah50.lindon.lab.cisco.com","name":"local","enable_wos":true},"swauth":{"reseller_prefix":"AUTH_","path_prefix":"auth/","max_key_len":256,"max_user_len":256,"token_life":86400,"max_token_life":86400,"max_account_len":256},"swift":{"version":"2.2.0","max_account_len":256,"max_container_len":256,"max_object_len":1024,"max_container_list":10000,"max_object_list":10000},"log":{"default":"notice"},"rio":{"path_prefix":"rio/“}}



Verify Log Files Creation

Verify that log files are being written to /arroyo/log (for example, cosd.log, http.log, protocoltiming.log) as follows:

[cos-node@ root] ls /arroyo/log/cosd.*/arroyo/log/cosd.log.20141120

[cos-node@ root] ls /arroyo/log/http.*/arroyo/log/http.log.20141120

[cos-node@ root] ls /arroyo/log/protocoltiming.*/arroyo/log/protocoltiming.log.20141120

Create User Accounts and Verify Node/Cluster Health

To create users and accounts for a COS node, install the swauth package on any Linux server and use the commands described below.

Note For additional details on the CLIs, see COS Command Line Utilities, page D-1 and the Cisco Cloud Object Storage Release 3.0.1 API Guide.

COS Username Scripts

COS stores the usernames and passwords for the GUI in the file etc/opt/cisco/cos/public/users.properties stored on the PAM VM. COS provides the following scripts to help you manage usernames and passwords:

• To list current users, enter the following command:

listusers-gui

• To add a new user, enter the following command:

adduser-gui username password

• To change an existing user, enter the following command:

chpass-gui username password

• To delete a user, enter the following command:

deluser-gui username

You can enter multiple commands when adding, changing, or deleting multiple users or passwords. For example, the following sequence of commands adds two users, changes two users, and deletes one user:

adduser-gui username1 password1

adduser-gui username 2 password2

chpass-gui username3 password3

chpass-gui username4 password4

deluser-gui username5

Note Usernames and passwords for the COS GUI can be strings of any length, are not case-sensitive, and can include uppercase or lowercase letters, numbers, and any special characters. However, the characters <, >, |, !, and & must be preceded by a backslash.



COS Swift CLI

The cos-swift command line utility is based on the Swift API, and can be used to manage storage accounts, storage containers, and storage objects in a COS cluster.

Syntax

cos-swift [–t <auth-token>] [–a <storage-url>] [–v/– –verbose][–h/– –help <subcommand>] [subcommand <options>]

COS Swauth CLI

The cos-swauth command line utility is based on the Swauth API, and can be used to manage authentication accounts and users in a COS cluster.

COS Cluster CLI

The cos-cluster command line utility can be used to manage the COS cluster membership of individual nodes.

• join – add a COS node to a COS cluster

• <cluster> – specify the domain name or IP address of a node in the cluster

• leave – temporarily remove a COS node from a COS cluster

Note When a node is temporarily removed, it is failed over, and the cluster may become non-resilient if only one copy of the objects on the node remains. After temporarily removing a COS node from a COS cluster, the cluster must be rebalanced. Only one node can be removed from the cluster at a time using this subcommand.

• –w/ – –wait <minutes> – specify the time in minutes to allow for the conclusion of TCP sessions on the node

• rejoin – add a COS node back to the COS cluster from which it was temporarily removed using the leave command

• remove – permanently remove a COS node from a COS cluster

Note When a node is permanently removed, the cluster is rebalanced.

• view – view a list of all the COS nodes in a COS cluster and the status of each node

PAM Installation and ConfigurationTo deploy the PAM from the vCenter, do the following:

Step 1 Log into the vCenter.

Step 2 Click File > Deploy OVF Template. The Source dialog appears.

Step 3 Select the location of the OVF template (for example, mendocino-shenandoah-301.16144) and click Next. The OVA Template Details dialog appears.



Step 4 Verify the OVF template details and click Next. The End User License Agreement dialog appears.

Step 5 Accept the End User License Agreement and click Next. The Name and Location dialog appears.

Step 6 Specify a name and location for the deployed template and click Next. The Deployment Configuration dialog appears.

Step 7 Select the MOS PAM deployment configuration from the drop-down list and click Next. The Storage dialog appears.

Step 8 Select a destination storage file for the VM files and click Next. The Disk Format dialog appears.

Step 9 Confirm that Thin Provision disk format for virtual disk storage is selected and click Next. The Network Mapping dialog appears.

Step 10 For each Source Network listed, choose the appropriate Destination Network from the drop-down list and click Next. The Properties dialog appears.

Step 11 Enter or choose the following information to customize the software solution for the deployment:

• IP address(es) of the NTP server(s)

• IP address of the DNS server

• External DNS transaction signature key (enter twice to confirm)

• Transaction signature algorithm

• IP address or hostname of two HA peers, if enabled

• (Optional) machine hostname

• Machine domain name

• Network adapter IP address

• Network adapter subnet mask

• Network adapter gateway IP address

Step 12 Click Next, then click Finish to deploy the OVA and create the PAM VM. The progress bar shows the status of the deployment.

Step 13 Verify connectivity and open an SSH session into the PAM VM.

Upgrading PAM

Perform these steps as needed to upgrade the current PAM installation.

Step 1 Disable the COS service by placing it in Maintenance status.

Step 2 Run a backup procedure on one of the PAMs in HA mode as follows:

ssh admin@nodesudo /opt/cisco/mos/bin/backup.py backup

Note Before running the backup.py script, be sure that the LANG or LC_* environment variables are set correctly.

Step 3 Confirm that the backup script generates a db folder and three files:

• node.txt

• state.json

• version.txt


Chapter 2 Deploying COS Configuring a COS Node for Data Mover

Step 4 Upgrade the COS nodes to the 3.0.1 image, and then shut them down.

Step 5 Shut down the three older PAMs.

Step 6 Rename the PAM VMs so they can be recovered in the event of a problem with the upgrade.

Step 7 Launch the new (Version 2.4) PAMs with the same names as the older ones.

Step 8 Restore the database backup on one of the new PAMs as follows:

ssh admin@nodeIPsudo /opt/cisco/mos/bin/restore.py backupsudo /opt/cisco/mos/bin/nodeRestore.py backup"

Step 9 Reboot the three new PAMs.

During reboot, cos-aic runs migration scripts, migrating all objects and references to new schemas and creating new aic_* objects as required. The COS Service remains in Maintenance status during this time.

Step 10 Restart each of the COS nodes.

Step 11 Return the COS Service to active (inService) status.

Configuring a COS Node for Data MoverCOS Release 3.0.1 includes components for a new data-mover architecture. This architecture includes the nginx software package, which is installed automatically either when installing COS 3.0.1 or when upgrading from a previous release. Other procedures are needed to configure any non-data-mover COS nodes to use the data-mover components.

After installing COS 3.0.1 or updating from a previous release, first verify that the nginx RPM was installed automatically as follows:

[root@utah10 ~]# cos_pkgs | grep nginx| nginx | 1.6.0 | 2.1.0.cos0.45 | b91 |

After confirming that nginx is installed, proceed with the configuration steps described in this section.

Configuration OverviewThe main areas for configuration updates are the network adapters. In earlier COS releases, the COS network configuration is generally divided into two functions: management and data. The Swift and Swauth functions are serviced on the data network, with the cserver network stack owning the data network interfaces.

With the data-mover architecture, the data network is further divided into service and internal interfaces. The service interfaces are externally facing, and handle the servicing of Swift and Swauth functions. Further, the Linux network stack owns the service network interfaces. The internal network interfaces are used for the cserver cluster communication internal to the cluster, and remain owned by cserver.

The necessary network configuration changes focus on dividing the data network interfaces into the service and internal interfaces. This simply means releasing interfaces from cserver and configuring them solely for use by the Linux network stack. Our general recommendation is to divide half of the data interfaces to service interfaces, and the other half to internal interfaces.

The configuration files that need to be changed on a COS node to convert to a data-mover configuration are as follows:



• /arroyo/test/setupfile

• /arroyo/test/RemoteServers

• /etc/sysconfig/network-scripts/route-eth*

• /etc/modprobe.d/blacklist.conf

After updating these files as described below, you can configure the nginx service and verify its functionality.

Updating /arroyo/test/setupfile and /arroyo/test/RemoteServersThe /arroyo/test/setupfile and /arroyo/test/RemoteServer configuration files must be updated to remove any configurations that apply to interfaces being converted to service interfaces. These changes release the interfaces from use by cserver.

Below are some examples of changes to configuration of a CDE250 configured with four 1 Gb data interfaces. The changes release eth2 and eth3 for use as service interfaces, and leave eth4 and eth5 for cserver internal use.

Prior to the configuration change:

[root@utah10 ~]# cat /arroyo/test/setupfile ...e1000e 0: streaming 1 fill 1 ip 10.0.10.0 tport 0 cport 0 tgid 0 e1000e 1: streaming 1 fill 1 ip 10.0.10.1 tport 0 cport 0 tgid 0 e1000e 2: streaming 1 fill 1 ip 10.0.10.2 tport 0 cport 0 tgid 0 e1000e 3: streaming 1 fill 1 ip 10.0.10.3 tport 0 cport 0 tgid 0 ... [root@utah10 ~]# cat /arroyo/test/RemoteServers remote serverid 1ip 10.0.10.0ip 10.0.10.1ip 10.0.10.2ip 10.0.10.3end remote server

After the configuration change:

[root@utah10 ~]# cat /arroyo/test/setupfile ...e1000e 2: streaming 1 fill 1 ip 10.0.10.2 tport 0 cport 0 tgid 0 e1000e 3: streaming 1 fill 1 ip 10.0.10.3 tport 0 cport 0 tgid 0 ... [root@utah10 ~]# cat /arroyo/test/RemoteServers remote serverid 1ip 10.0.10.2ip 10.0.10.3end remote server

Updating /etc/sysconfig/network-scripts/route-eth*The route-eth* files contain configuration for source-based routing in the Linux network stack. A small but important change must be made to this configuration. The following is an example of the necessary change, with areas of change highlighted.

Before the change:



[root@utah10 bin]# cat /etc/sysconfig/network-scripts/route-eth210.0.0.0/16 dev eth2 10.0.10.0 table 102default via 10.0.0.1 dev eth2 table 102

After the change:

[root@utah10 bin]# cat /etc/sysconfig/network-scripts/route-eth210.0.0.0/16 dev eth2 src 10.0.10.0 table 102default via 10.0.0.1 dev eth2 src 10.0.10.0 table 102

Updating /etc/modprobe.d/blacklist.confThe e1000e and ixgbe directives included in /etc/modprobe.d/blacklist.conf must be removed. This change allows the service interfaces to start automatically on boot.

Before the change:

[root@utah10 bin]# cat /etc/modprobe.d/blacklist.conf | tailblacklist snd-pcsp

# I/O dynamic configuration support for s390x (bz #563228)blacklist chsc_schblacklist e1000blacklist e1000eblacklist ixgbeblacklist csdblacklist cdd_sysblacklist avs_cserver

After the change:

[root@utah10 bin]# cat /etc/modprobe.d/blacklist.conf | tailblacklist snd-pcsp

# I/O dynamic configuration support for s390x (bz #563228)blacklist chsc_schblacklist e1000blacklist csdblacklist cdd_sysblacklist avs_cserver

Configuring the nginx ServiceThe nginx software component service is managed with a SYS V init script. That is included in the nginx RPM. The service needs to be turned on for the service to automatically start upon boot. No special configuration file changes are needed for the nginx component at this time.

[root@utah10 ~]# chkconfig nginx on[root@utah10 ~]# chkconfig ...cassandra 0:off1:off2:on3:on4:on5:on6:offcos_aicc 0:off1:off2:on3:on4:on5:on6:offcosd 0:off1:off2:on3:on4:on5:on6:off...cserver 0:off1:off2:on3:on4:on5:on6:off...nginx 0:off1:off2:on3:on4:on5:on6:off...


Chapter 2 Deploying COS Setting Up the External DNS Server

Note For previous COS systems, all data interfaces were published to DNS as part of the COS cluster hostname. For the data-mover architecture, only the service interfaces are published to DNS. Therefore, the internal interface IP addresses must be removed.

Verify nginx FunctionalityAfter applying the configuration changes, reboot the system and allow the services to start. You can then verify proper functionality of the nginx component using a simple curl command executed directly on the COS node. The following example exercises the Swauth list-accounts API:

[root@utah10 bin]# curl http://10.0.10.0/auth/v2 -H "X-Auth-Admin-User: .super_admin" -H "X-Auth-Admin-Key: rootroot" -v* About to connect() to 10.0.10.0 port 80 (#0)* Trying 10.0.10.0... connected* Connected to 10.0.10.0 (10.0.10.0) port 80 (#0)> GET /auth/v2 HTTP/1.1> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.6.0

zlib/1.2.3 libidn/1.18 libssh2/1.4.2> Host: 10.0.10.0> Accept: */*> X-Auth-Admin-User: .super_admin> X-Auth-Admin-Key: rootroot> < HTTP/1.1 200 OK< Server: nginx/1.6.0; cos/2.1.0-b91< Date: Fri, 30 Jan 2015 19:31:39 GMT< Content-Type: application/json; charset=utf-8< Content-Length: 35< Connection: keep-alive< {"accounts": [ {"name": "ryan"}]}* Connection #0 to host 10.0.10.0 left intact* Closing connection #0

Setting Up the External DNS ServerThe administrator is responsible for creating the transaction signature (TSIG) key for the TSIG algorithm on the external DNS server. Valid TSIG algorithms are hmac-md5, hmac-sha1, hmac-sha224, hmac-sha256, hmac-sha384, and hmac-sha512. The following procedure shows how to create the TSIG key for the hmac-md5 algorithm.

Step 1 Generate the TSIG key by entering the following command:

dnssec-keygen -a HMAC-MD5 -b 128 -n HOST testdns.com.

where:

• HMAC-MD5 is the TSIG algorithm.

• 128 is the number of bits in the key.



• testdns.com. is the name of the key. The name of the key, the domain name of the PAM, and the DNS zone in the external DNS server should all be the same (in this example, testdns.com).

• The command must end with a period (.), which is required when generating the key.

Sample output:

dnssec-keygen -a HMAC-MD5 -b 128 -n HOST testdns.com.

Ktestdns.com.+157+05519

This command creates a key file and a private key file.

• Sample .key file:

Ktestdns.com.+157+05519.key

testdns.com. IN KEY 512 3 157 ujLdXfCZenQZQKZlFy42fw==

• Sample .private key file:

Ktestdns.com.+157+05519.private

\Private-key-format: v1.3

Algorithm: 157 (HMAC_MD5)

Key: ujLdXfCZenQZQKZlFy42fw==

Bits: AAA=

Created: 20140325141250

Publish: 20140325141250

Activate: 20140325141250

Step 2 Create the key file (in this example, testdns.com.key) in the /etc/ directory.

Sample key file:

testdns.com.key

key testdns.com. {

algorithm hmac-md5;

secret “ujLdXfCZenQZQKZlFy42fw==“;

};

where:

• hmac-md5 is the TSIG algorithm.

• ujLdXfCZenQZQKZlFy42fw== is the TSIG key.

Step 3 Add the key file path to the /etc/named.conf file by inserting the following line:

include “/etc/testdns.com.key“;

Step 4 Configure the DNS zones (used when deploying the PAM) on the external DNS server as shown in the following examples.

a. Update the /etc/named.conf file with the details of the DNS zone (in this example, testdns.com) and reverse zone, for all interfaces.

In the following sample information:

– The DNS zone is related to the 172.20.216.xx subnet, which can be the Management interface.

– For the Data In and Data Out interfaces, similar information related to the reverse zone must be added.



– Data In is related to the 15.1.1.x subnet.

– Data Out is related to the 25.1.1.x subnet.

– testdns.com. is the key name.

DNS Zone Detailszone testdns.com IN {type master;file “slaves/db.testdns.com“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Management Interface)zone 216.20.172.IN-ADDR.ARPA IN {type master;file “slaves/db.216.20.172“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Data In Interface)zone 1.1.15.IN-ADDR.ARPA IN {type master;file “slaves/db.1.1.15“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Data Out Interface)zone 1.1.25.IN-ADDR.ARPA IN {type master;file “slaves/db.1.1.25“;allow-update { key “testdns.com.“; };notify yes;};

b. Create db.* files in the /var/named/slaves directory in the external DNS server for the DNS zone and reverse zone, for all interfaces.

Sample db File for DNS Zone (db.testdns.com)@ 86400 IN SOA dns pam 2014031001 3600 1800 604800 86400

@ 86400 IN NS dns

dhcp 86400 IN CNAME dns

dns 86400 IN A pam_ip_address

ntp 86400 IN CNAME dns

Sample db File for Reverse Zone of Management Interface (db.216.20.172)$ORIGIN .

$TTL 86400 ; 1 day

216.20.172.IN-ADDR.ARPA IN SOA dns.testdns.com. pam.testdns.com. (



2012071021 ; serial

3600 ; refresh (1 hour)

1800 ; retry (30 minutes)

604800 ; expire (1 week)

86400 ; minimum (1 day)

)

NS dns.testdns.com.

$ORIGIN 216.20.172.IN-ADDR.ARPA.

$TTL 7200 ; 2 hours

xx PTR dns.testdns.com. //xx = last two digits of the PAM IP

Sample db File for Reverse Zone for Data In Interface (db.1.1.15)

$ORIGIN .

$TTL 86400 ; 1 day


2012071021 ; serial





)

NS dns.testdns.com.


$TTL 7200 ; 2 hours

1 PTR dns.testdns.com.

Sample db File for Reverse Zone for Data Out Interface (db.1.1.15)

$ORIGIN .

$TTL 86400 ; 1 day


2012071021 ; serial





)

NS dns.testdns.com.



Chapter 2 Deploying COS Editing NTP Server Information

$TTL 7200 ; 2 hours

1 PTR dns.testdns.com.

Step 5 Make sure that the PAM and the external DNS server are in sync for the time and date.

Editing NTP Server Information

Note NTP servers are configured when the PAM is deployed. You cannot add or delete NTP servers using this procedure.

To edit NTP server information using the COS Service Manager GUI, follow these steps:

Step 1 Log in to the COS Service Manager GUI as described in Using the COS Service Manager GUI, page A-2.

Step 2 Go to Infrastructure > Platform Services.

In the NTP Servers table, the following information is displayed for each NTP server:

• Name (required)

– The name of the NTP server is a string of up to 30 characters. Acceptable characters include uppercase and lowercase letters, numbers, periods (.), dashes (-), and underscores (_).

– The name must not begin with a period (.), and it is not case-sensitive.

• Description — The description is a string of up to 100 characters, and can include uppercase or lowercase letters, numbers, and any special characters.

• Region — The region in which the NTP server resides.

• Servers — The hostname or IPv4 address of the NTP server.

Step 3 Edit the Description, Region, and Servers fields of an NTP server as required.

Viewing DNS Server Information

Note DNS servers are configured when the PAM is deployed.

To view DNS server information using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > Platform Services.

In the DNS Servers table, the following information is displayed for each DNS server:

• Name (required)


Chapter 2 Deploying COS Configuring COS

– The name of the DNS server is a string of up to 35 characters. Acceptable characters include uppercase and lowercase letters, numbers, periods (.), dashes (-), and underscores (_).

– The name cannot begin with a period (.), and it is not case-sensitive.

• Description — The description is a string of up to 100 characters, and can include uppercase or lowercase letters, numbers, and any special characters.

• Region — Region in which the DNS server resides.

• IPv4 Address — IPv4 address of the DNS server.

• TSIG — Transaction Signature (TSIG) of the DNS server.

• TSIG Algorithm — Name of the TSIG algorithm used by the DNS server.

Configuring COSTo configure COS using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > Regions & Zones.

A zone is a set of Cloud platform components (compute, network, storage, and security) that are fate-shared. The zone can be mapped to the underlying Cloud platform provider, such as a datacenter in vCenter, an Availability Zone, or any other combination of fate-shared Cloud resource topologies. Each zone is associated with one Cloud Controller.

A region is made up of one or more zones. It is an abstract representation of the underlying Cloud platform. A region can be associated with a geographical region, one or more datacenters, or a service area. Release 3.0.1 of the COS software supports only one region.

Step 3 Expand a region to display the zones associated with that region.

Step 4 Click Add Row to add zones to the region, as needed.

Step 5 Go to Infrastructure > IP Address Pools and Networks to define IP pools for the COS node.

Step 6 Go to Infrastructure > COS Cluster and create a COS cluster.

Step 7 Go to Services > Cloud Object Stores, click the COS service instance, and create one COS endpoint.

Step 8 On the COS node, execute /opt/cisco/cos/config/cosinit and configure the IP pool name, DocServer IP and Port (which is the COS Service Manager management IP). This will register the COS node with the COS Service Manager.

Step 9 Go to Infrastructure > COS Nodes and configure Cluster Name by choosing the created cluster name from the drop-down list.

Step 10 Ensure that the Admin State for the node is set to Inservice. Click Save.

Step 11 Go to Services > Overview and confirm that you can see the registered COS Node.



Configuring the Service Instance TemplateTo configure the COS service instance template using the COS Service Manager GUI, do the following:


Step 2 Go to Services > Cloud Object Storage. The Service Summary page opens, displaying a list of COS service instances.

Step 3 Click the COS service instance to be configured. The Service Definition tab for that instance opens, containing the General and Service Endpoints areas.

Step 4 The fields of the General area are listed below. You can edit them appropriately.

• Title — The title of the COS service. The default title is An unused COS Service.

• Description — A brief description of the COS service. The default description is COS Service.

• Admin State — The administrative state of the COS service. From the drop-down list, you can choose to Enable or Disable the service.

• Service Template — Currently, this read-only field identifies the default service template, Cisco Object Store (COS) Service.

Step 5 There is no default COS endpoint when the PAM is brought up, so you must create one manually. The following parameters are displayed for the service endpoint and can be edited appropriately:

• Name — The name of the service endpoint is set to the default value ce1 and cannot be changed.

• Description — A brief description of the service endpoint.

• Region — The region to which the service endpoint belongs.

• Min Nodes — The minimum number of nodes that must be associated the service endpoint.

• Desired Nodes — The desired number of nodes for the service endpoint.

• Max Nodes — The maximum number of nodes that can be associated with the service endpoint.

• Max Storage — This field is to be ignored for Release 2.0.1.

• Cluster Name — No default cluster is loaded when the PAM is brought up, so you must create one manually.

• Auth Profile — The default value is auth-1.

• State — The state of the service endpoint. From the drop-down list, choose Disabled or Enabled.

Step 6 For your changes to take effect, click Save. To discard your changes, click Cancel.

Configuring IP PoolsBefore installing COS nodes, you must create a network and configure IP pools using the COS Service Manager GUI.


Step 2 Go to Infrastructure > IP Address Pools and Networks and create a suitable network for the IP pools.



Step 3 To add a new IP pool, click IP Address Pools > Add Row.

Step 4 In the Name field, enter a unique name for the IP pool. In the Description field, enter a brief description for the IP pool.

Step 5 To associate an IP address range with the IP pool, click IP Address Ranges > Add Row.

Step 6 Enter appropriate values in the fields described below:

Note If you attempt to initialize a COS node using an IP pool with an insufficient number of available IP addresses, the initialization fails and an Event is generated. For more information on this class of events, see COS AIC Server Events, page 3-6.


Note After you have associated an IP pool with a COS node, do not edit or delete the pool before dissociating the pool from the node.

Editing or Deleting IP PoolsTo edit or delete configured IP pools using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > IP Pools and Networks.

Step 3 Before editing or deleting an IP pool, ensure that it is not serving any COS nodes.

a. Dissociate any COS nodes associated with the IP pool to be edited or deleted, and link these nodes to another IP pool.

Step 4 Check the box against the name of the IP pool to be edited or deleted.

Step 5 Click Edit or Delete.

Table 2-1 IP Address Ranges Fields

Field Description

Range Start The first IP address of the range.

Range End The last IP address of the range.

Netmask The netmask for the range.

Gateway The Gateway for the range.



Configuring a COS ClusterAs part of the COS installation, a single default node cluster is created. The cluster can be configured using the COS Service Manager GUI.

To view or edit the settings of a COS node cluster, follow these steps:


Step 2 Go to Infrastructure > COS Clusters.The default COS node cluster is displayed.

Step 3 Edit the fields described below based on your deployment.


COS Node Initial ConfigurationCOS nodes ship with a pre-installed image. When provisioning a COS node on a network, you can upgrade the node to a different image. After installing the image and rebooting the node, you are prompted to start configuration by running the cosinit script.

• Enter yes to run cosinit immediately.

• If you wish to configure the node later, enter no. You can run the cosinit later using the following command: run /opt/cisco/cos/config/cosinit

When it runs, the cosinit script performs the initial setup and adds the node to the COS management system.

The cosinit script can be run with the options shown below:

cosinit [–skipnw] [–input <configFile>] [–help]

• [–skipnw] — Skip network configurations

• [–input <configFile>] — Specify an input file for configuration

• [–help] — Displays the usage

Example 1 /opt/cisco/cos/config/cosinit -skipnw

This is used to configure IP pool and DocServer, and register the cos node into COS Service Manager.

Example 2/opt/cisco/cos/config/cosinit -input configFile

Table 2-2 COS Node Cluster Fields

Field Description

Authentication FQDN The FQDN for COS authentication requests.

Storage FQDN Currently, this must be the same as the Authentication FQDN.



This enables cosinit to run with a config file. All the parameters will be read from the config file.

Sample configFileInterfaceConfig : skip

(Optional; if the line above is added, management network configuration is skipped.)

IpAddress : 172.22.99.218NetMask : 255.255.254.0Broadcast : 172.22.99.255DefaultGateway : 172.22.98.1HostName : CDE460-218PartNum : CDE460-4R1DocServerHost : 172.22.116.35DocServerPort : 5087BaudRate : 9600eth4 : 4eth5 : 4eth6 : 4eth7 : 4

Executing the cosinit script completes the following tasks:

• Configuring the management interface — ethX, IP, netmask, broadcast.

• Configuring the default gateway.

• Configuring the hostname.

• Mapping the network interfaces to IP addresses in the network.

• Configuring Doc Server IP address and Port.

• Recording appliance model name.

• Configuring the baud rate by reading the value in install_baud_rate. If the install_baud_rate file is not found under the /root directory, the operator is prompted to enter a baud rate during cosinit execution or specify the value in the configFile.

When the cosinit script is successfully executed:

• /etc/sysconfig/network-scripts/ifcfg-eth0 is generated and the management interface configuration is saved in it.

• /etc/sysconfig/network is generated, and the hostname and default gateway info are saved in it.

• /etc/hosts is generated and the host info is saved in it.

• Network is restarted.

• /tmp/.cosnodeinit file is created with the configuration specified in cosinit.

• The baud rate is updated in the /boot/grub/grub.conf file.

• The COS AIC client and other applications are started.

• The COS AIC client reads the cosnodeinit generated by cosinit.

• Using the Doc Server IP address and port mentioned in cosnodeinit, the AIC client connects to the Doc Server.

• The AIC client sends the cosannounce to the Doc Server. cosannounce is a partially populated smcosnode document.

The COS node is now registered with the COS management system. You can configure the node using the COS Manager GUI, add it to a COS cluster, and make it operational.



Automated Node Configuration (Optional)

Beginning with COS 3.0.1, you can use automated configuration by providing a file to cosinit that contains a ClusterName and an IP Pool reference for at least one service interface. This enables the system to configure the COS node without further intervention through the Service Manager GUI and API.

Example

Run cosinit with the input file:

/opt/cisco/cos/config/cosinit -input cosinit_input_file

[root@cos470-1 ~]# cat cosinit_autoInterfaceConfig : skipPartNum : C3160-R1ClusterName :cluster-1DocServerHost : 172.22.98.59DocServerPort : 5087eth1 : colusa-57eth2 : colusa-57eth4 : colusa-57eth5 : colusa-57

This results in one of the following:

• If ClusterName has a value, automated configuration is triggered. Then, if at least one service interface has an IP Pool configured, the AIC Client sets the adminstate to inService in the smcosnode document before sending cosannounce to DocServer. Otherwise, adminstate is set to Maintenance.

The AIC Server handles the rest of the configuration automatically by assigning an IP address from the specified IP Pool and adding the node to the cluster. The AIC Client then writes the necessary configuration files to the COS node as usual. When configuration is complete, the AIC Client automatically starts the cassandra, cosd and cserver cos node services.

• If ClusterName has no value, manual configuration is needed. You must manually set the COS node adminstate from Maintenance to inService, assign the node to a cluster, and enable the service interfaces from the GUI.

Note When using automated configuration of multiple COS nodes, configure the first node, and then wait before configuring the second node until the Cassandra database service to appear as Running in the GUI of the first node configured. Otherwise, there may be unexpected behavior in the seed list configuration for the Cassandra database of the nodes added after the first node.

Adding a COS Node to a COS ClusterAfter a COS node is installed and initialized, it must be added to a COS cluster to begin servicing COS requests.

To add a node to the COS cluster using the COS Service Manager GUI, do the following:


Step 2 Go to Infrastructure > COS Nodes.



Step 3 Check the box against the name of the node you want to edit and click Edit.

Step 4 From the Cluster Name drop-down list, choose the created COS cluster.


When a COS node is added to a COS cluster, the following files are written on to the node:

• setupfile — This file holds the primary configuration data for the COS node and is required by CServer.

• RemoteServers — This file holds the service interface IP addresses for all of the COS nodes in the cluster.

• SubnetTable — This file holds the IP, Netmask, Gateway and Network data for each service interface of a COS node.

• cosd.conf — This file holds the configuration for the cosd service.

• cassandra.yaml — This file, located in /etc/cassandra/conf/, is responsible for the configuration of the Cassandra service.

When the node Admin State is set to Inservice, all enabled service interfaces of that node are written to the DNS (internal or external).

Note To avoid possible bootstrapping issues with the Cassandra service, be sure to provide (or use a script to provide) a delay of at least two minutes between adding two nodes in sequence to the cluster.

If the replication factor is too high, use the following script on any one node in the cluster to adjust it:

sh /opt/cisco/cos-aic-client/cassandra/cassandra-adjust-replication.sh { 1 | 2 | 3 }

For the final argument, use 1 for one node, 2 for two nodes, and 3 for three or more nodes.

Configuring a COS NodeAfter a COS Node is installed and initialized, you can modify its configuration parameters using the COS Service Manager GUI as follows:




Step 4 Edit the fields described below based on your requirements.

Table 2-3 COS Node Fields

Field Description

Description A brief description of the node.

Model The part number of the node.

Zone The zone to which the node belongs.



Note The Admin State of a COS node must be set to Maintenance before removing the node from a COS cluster.

Step 5 To modify the service interfaces of the node, follow the procedure Configuring the Service Interface of a COS Node, page 2-33.


Configuring the Service Interface of a COS NodeTo modify the settings of a service interface of a functioning COS node using the COS Service Manager GUI, follow these steps:




Step 4 Check the box against the name of service interface you wish to edit and click Edit.

Step 5 Edit the fields described below based on your requirements.


Configuring Cassandra Database MaintenanceThe Cassandra database requires periodic maintenance with an anti-entropy repair that must be manually configured to execute every two days on each COS node. For more information on the Cassandra repair process, see the Cassandra 2.1 documentation available at:

Cluster Name The COS cluster to which the node belongs.

Admin State Indicates whether the node is Inservice or under Maintenance.

Table 2-3 COS Node Fields

Field Description

Table 2-4 Service Interface Fields

Field Description

IP Pool The IP pool from which this interface will be assigned an IP address. You may associate a different IP pool with this interface only when the node Admin State is set to Maintenance.

Enabled From the drop-down list, choose True or False to enable or disable the service instance, respectively.



www.datastax.com/documentation/cassandra/2.1/

To execute the periodic repair, we recommend configuring a CRON job on each node, and scheduling the CRON jobs on the nodes to begin at different times so as to avoid the repair running concurrently on more than one node at a time. As a reference, we recommend providing 15 minutes between scheduling of the repair operation on each node.

The command syntax for the repair is:

/usr/bin/nodetool repair -par -inc


C

C H A P T E R 3
System Monitoring
The COS Service can be monitored through the following:

• COS Application Status Monitoring

• COS Node Status Monitoring

• COS Management Events

COS Application Status MonitoringThe COS service is implemented through one or more instances of the Application Instance Controller (AIC). Each application instance serves a service instance, or an endpoint within a service instance.

Earlier releases of COS supported a single service instance with one endpoint, one cluster, and one redundancy policy. Beginning with Release 3.0.1, COS deployments can support multiple endpoints and clusters. Each endpoint can have zero or one COS cluster attached to it, and each endpoint has its own Asset Redundancy Policy.

The COS AIC client now supports the monit process to allow for continuous monitoring of each service instance. Each endpoint has its own AppStatus message for reporting its service level agreement (SLA) status. Application status is reported by the AIC to the COS Service Manager. This status is accessible in the Service Summary pane of the Home page of the COS Service Manager GUI. For each endpoint, the endpoint details, status as per the resource or capacity SLA, and Management events are reported.

The resource SLA includes a constraint based on the minimum, desired, and maximum number of COS nodes that can be in service.

• If none of the COS nodes is in service, the node status is set to critical.

• If the number of COS nodes in service is less than the minimum number of nodes, or greater than the maximum number of nodes, in the SLA, the node status is set to warning.

• If the number of COS nodes in service is within the SLA range, the node status is set to normal.

• If the COS Service is not properly configured, or if it is set to an inactive (maintenance) state, the node status and other SLA parameters, such as bandwidth and session statuses, is set to unavailable as part of the Application Status message.

In addition to the node status, the storage status is reported. Currently, the storage status is not calculated as per the SLA, but instead, is defined according to the following criteria:

• If the cumulative available storage on all nodes is 70% or less of the cumulative total storage on all nodes, the storage status is set to normal.


Chapter 3 System Monitoring COS Node Status Monitoring

• If the cumulative available storage on all nodes is greater than 70% but no more than 90% of the cumulative total storage on all nodes, the storage status is set to warning.

• If the cumulative available storage on all nodes is more than 90% of the cumulative total storage on all nodes, the storage status is set to critical.

The session and bandwidth status are also reported as part of the application status.

The session status is determined by analyzing the aggregate HTTP traffic for incoming and outgoing sessions separately. Based on the analysis, a status (unavailable, normal, warning, critical) is assigned to both the incoming and the outgoing sessions. The more severe of the two statuses is reported as the session status.

Similarly, the bandwidth status is determined by analyzing the ingress and egress traffic separately. Both the ingress and egress traffic are further resolved into committed external bandwidth (associated with CBR streaming), best effort bandwidth, and internal CServer traffic.

The bandwidth usage of each component is calculated as a 15 minute moving average of 90 samples (or the number available, if less than 90) taken at 10-second intervals. The sum of the committed external bandwidth usage and the best effort bandwidth usage is measured against the total capacity, and a status is assigned to both the ingress and the egress traffic. The more severe of the two statuses is reported as the bandwidth status.

Beginning with COS Release 3.0.1, the Service Manager GUI allows for configuration of the COS node service interface, and for the display of storage, network bandwidth, and session counts for individual COS nodes and services. The GUI does not support alarms reporting in this release.

COS Node Status MonitoringA COS node is in service if the associated service instance and asset workflow template are enabled. The status of each node that is in service and part of a COS cluster is displayed in the COS Service Manager GUI. This status is updated once every 30 seconds.

The COS AIC client on each COS node posts a document containing the status information for the node to the DocServer once every 10 seconds.

Once every 30 seconds, the COS AIC parses the data in this document and renders the data in a format suitable for display in the COS Service Manager GUI. If the AIC cannot find status information for a node, it waits for 30 seconds. If the information is not received within this waiting period, the AIC reports that the node is not functional.

The status of each node reflects the status of the node’s network interfaces, disks, and services.

• If either all the interfaces of the node are not functional, the node status is reported as critical.

• If all the disks of the node are not functional, the node status is reported as critical.

• If a service is down, the node status is reported as critical.

The issue is described in detail in the Fault Detail field.

Viewing COS Node StatusTo view the status of a COS node using the COS Service Manager GUI, do the following:

Step 1 Log in to the COS Service Manager GUI as described in Reference Information, page A-1.

Step 2 Go to Services > Cloud Object Storage.


Chapter 3 System Monitoring COS Management Events

Step 3 Click on the Overview tab.

AIC Client MonitoringThe COS AIC client running on a COS node periodically monitors the disks, interfaces, and services (processes) of that node and posts the data to the DocServer as a COS-specific document.

The AIC client begins the monitoring activity when a node is configured and added to a COS cluster. As long as the node is running and is part of a COS cluster, monitoring occurs once every 10 seconds.

Storage Monitoring

The AIC client can monitor and report storage (disk) state and statistics only if the CServer is running on the node. The following information is reported for each disk:

• Disk name

• Bytes read

• Bytes written

• Requests

• State

• S.M.A.R.T. Status

The client also reports the total storage space on all disks and the total storage space currently in use.

Interface Monitoring

For each interface, the AIC client reports the interface state and the transmit and receive statistics. The client can monitor and report the state and statistics of CServer interfaces only when the CServer is running on the node.

Services Monitoring

The AIC client monitors the following services:

• Cisco Cache Server (CServer)

• Cisco Cloud Object Storage Daemon (cosd)

• Cassandra Server

• NTP Daemon

• SNMP Daemon

COS Management EventsThe COS management events indicate the current state of COS services and the health of COS nodes.



Viewing COS Node Management EventsThe management events for a COS node are displayed in the COS Service Manager GUI.

To see the management events for a COS node, follow these steps:

Step 1 Log in to the COS Service Manager GUI as described in Reference Information, page A-1.

Step 2 Go to Monitoring > Events.

All the events for the node are listed with the oldest event first.

Events belong to one of the following levels of severity:

• Info – The event represents information only and does not require operator intervention.

• Warning – The event represents an issue that is possibly transitory and the operator should investigate the cause.

• Critical - The event represents an issue from which the node may not recover without operator intervention, and the operator must act immediately because the issue may cause service outage.

Service EventsThe COS service events are listed in the table below.

Table 3-1 COS Service Events

Event Name Description Severity Event TypeEvent Subtype

Service.Activated COS service activated by enabling Service Instance

Info Config Config

Service.Deactivated COS service deactivated by disabling Service Instance

Critical Config Config

Service.Deactivated COS Service Instance deleted from DocServer


Service.Activated COS service activated by enabling Asset Workflow Template

Info Config Config

Service.Modified COS service modified — Asset Redundancy Policy Reference changed in AWT

Info Config Config

Service.Deactivated Asset Workflow Template disabled or deleted from DocServer

Info Config Config

ServiceInstance.NotFound

COS AIC could not fetch Service Instance object from DocServer


CaptureEndpoint.NotFound

COS AIC could not fetch Capture Endpoint object from DocServer


COSNodeCluster.NotFound

COS AIC could not fetch Node Cluster object from DocServer




COS AIC Client EventsCOS AIC client generates events pertaining to storage (disk), network (interface), and service (process) on each node. These events are generated only if the AIC client monitoring is enabled. For more information on this monitoring activity, see AIC Client Monitoring, page 3-3.

The events generated by a COS AIC client are listed in the table below.

• An event is generated for every change in the state of operation of a disk, interface, or process.

• Although events are generated only on a state change, monitoring is done every 10 seconds, so events can be generated as often as every 10 seconds.

AssetWorkflowTemplate.NotFound

COS AIC could not fetch Asset Workflow Template object from DocServer


COSNode.Down COS node has not reported status for last 30 seconds

Critical Node Accessibility

Service.StorageStatus SLA status for storage is warning Warning Application SLA

Service.StorageStatus SLA Status for Storage is critical Critical Application SLA

Service.SessionStatus SLA status for sessions is warning Warning Application SLA

Service.SessionStatus SLA status for sessions is critical Critical Application SLA

Service.BandwidthStatus SLA status for sessions is warning Warning Application SLA

Service.BandwidthStatus SLA status for sessions is critical Critical Application SLA

COS-AIC Started COS AIC has started Info Application Health

COS-AIC Terminated COS AIC is exiting Critical Application Health

COS-AIC Terminated COS AIC is exiting after receiving SIGINT

Critical Application Health

COS-AIC Terminated COS AIC is exiting after receiving SIGTERM


COS-AIC Terminated An uncaught exception was found; the program will end


Table 3-1 COS Service Events

Event Name Description Severity Event TypeEvent Subtype

Table 3-2 COS AIC Client Events

Event Name Description Severity Event Type Event Subtype

CosNodeDiskDown Disk disk_name down Warning COS-Node Health

CosNodeDiskUp Disk disk_name up Info COS-Node Health

CosNodeInterfaceDown Interface if_name down Warning COS-Node Health

CosNodeInterfaceUp Interface if_name up Info COS-Node Health

CosNodeServiceDown Service svc_name down Warning COS-Node Health

CosNodeServiceUp Service svc_name up Info COS-Node Health


Chapter 3 System Monitoring COS Node Platform Monitoring with SNMP

COS AIC Server EventsThe COS AIC server generates the events listed in the table below.

COS Node Platform Monitoring with SNMP

OverviewEach COS node provide hardware statistics, events, and alerts using Simple Network Management Protocol (SNMP). In addition to hardware platform information, some basic state for the COS node disk and file system usage is also available with SNMP.

Each COS node executes an instance of the Net-SNMP snmpd service as an SNMP agent. An instance of the Net-SNMP snmptrapd service also executes on each COS node, and can be customized for environment-specific configuration for the handling of traps and notifications generated by the snmpd service.

For the CDE 460 and CDE 470 hardware platforms, Super Micro Computer’s SuperDoctor 5 service is leveraged to provide hardware status and monitoring. The SuperDoctor 5 utility integrates with SNMP and provides platform information through SNMP Management Information Bases (MIBs) defined by Super Micro Computer.

InstallationThe SNMP services for COS nodes are provided by the following RPMs:

• net-snmp

• net-snmp-libs

• net-snmp-utils

• SuperDoctor

• cos_snmp

These RPMs come pre-installed on each COS node, and are preconfigured with a basic configuration for read-only access to the SNMP service.

Table 3-3 COS AIC Server Events

Event Name Description Severity Event Type Event Subtype

AddCosNode New COS node added Info COS-Node Accessibility

DeleteCosNode COS node deleted Warning COS-Node Accessibility

CosNodeInterfaceError No available IP addresses in IP pool

Critical COS-Node Accessibility

CosUpdatedActiveIpPool

An active IP pool was edited

Critical COS-Node Accessibility



ConfigurationBasic configuration for the Net-SNMP services (snmpd and snmptrapd) and the SuperDoctor 5 utility is provided by the cos_snmp RPM. The snmpd and snmptrapd service configuration can be customized to accommodate customer-specific environments by manually editing the /etc/snmp/snmpd.conf and /etc/snmp/snmptrapd.conf configuration files, respectively.

Note Customization made to these files will be replaced with an update to the cos_snmp RPM. The cos_snmp RPM will back up configuration files in the /etc/snmp directory upon upgrade.

MIB ExtensionsNet-SNMP is distributed with support for various MIBs that provide OID instances for some hardware and Linux platform information. The definitions for MIBs distributed with Net-SNMP service are stored in /usr/share/snmp/mibs on each COS node. For more information about the standard MIBs provided by Net-SNMP, see:

http://www.net-snmp.org/docs/mibs/

Definitions for MIBs distributed with the SuperDoctor 5 service are stored in /opt/Supermicro/SuperDoctor5/mibs on each COS node. For more information about the SuperDoctor 5 service, see:

http://www.supermicro.com/solutions/SMS_SD5.cfm

Extensions have been added to Net-SNMP to add CServer disk and storage information to the following tables:

• HOST-RESOURCES-MIB::hrDeviceTable

• HOST-RESOURCES-MIB::hrDiskStorageTable

• HOST-RESOURCES-MIB::hrStorageTable

• UCD-SNMP-MIB::dskTable

Extensions to HOST-RESOURCES-MIB::hrDeviceTable

This table has been extended to include status for the external facing disk drives that are owned by CServer. The columns of primary interest are hrDeviceDescr and hrDeviceStatus. The combination of these two columns identify a disk device and its current operating state.

The following is a snip of the table showing the extension:

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost HOST-RESOURCES-MIB::hrDeviceTableSNMP table: HOST-RESOURCES-MIB::hrDeviceTable

hrDeviceIndex hrDeviceType hrDeviceDescr hrDeviceID hrDeviceStatus hrDeviceErrors

... 7680 HOST-RESOURCES-TYPES::hrDeviceDiskStorage Cisco csd1 SAS HDD SNMPv2-SMI::zeroDotZero running 0 7681 HOST-RESOURCES-TYPES::hrDeviceDiskStorage Cisco csd2 SAS HDD SNMPv2-SMI::zeroDotZero running 0



... 7751 HOST-RESOURCES-TYPES::hrDeviceDiskStorage Cisco csd72 SAS HDD SNMPv2-SMI::zeroDotZero running 0

According to the HOST-RESOURCES-MIB definition, valid values of hrDeviceStatus are unknown, running, warning, testing, and down.

[root@utah97 ~]# snmptranslate HOST-RESOURCES-MIB::hrDeviceStatus -Tp +-- -R-- EnumVal hrDeviceStatus(5) Values: unknown(1), running(2), warning(3), testing(4), down(5)

The SNMP device status is mapped from output of the cddm utility. For more information on cddm, see the cddm man page on any COS node with man cddm.

The mapping of cddm output to SNMP status is as follows:

• cddm state contains DEV_READY and cddm smart_status is OK => SNMP running

• cddm state contains DEV_READY and cddm smart_status is ADVISORY => SNMP warning

• cddm state contains (DEV_SUSPENDING or DEV_SUSPENDED or DEV_TRACK_MAP or DEV_USER_PREP or DEV_ABANDONED) => SNMP warning

• cddm state contains (DEV_LOG_REMOVED|DEV_SICK|DEV_REMOVED) => SNMP down

All other cddm status are mapped to an SNMP unknown state.

Extensions to HOST-RESOURCES-MIB::hrDiskStorageTable

This table has been extended to show disk capacity for disk drives used by CServer for storing of object data.

The following is a snip of the table showing the extension:

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost HOST-RESOURCES-MIB::hrDiskStorageTableSNMP table: HOST-RESOURCES-MIB::hrDiskStorageTable

hrDiskStorageAccess hrDiskStorageMedia hrDiskStorageRemoveble hrDiskStorageCapacity readWrite unknown false 194335744 KBytes readWrite hardDisk true 3907018583 KBytes

... readWrite hardDisk true 3907018583 KBytes readWrite hardDisk true 3907018583 KBytes

Viewing the table in this manner does not easily correlate to a particular device, because the table does not define a device description. You must manually correlate the entries from the HOST-RESOURCES-MIB::hrDiskStorageTable and HOST-RESOURCES-MIB::hrDeviceTable tables.

Extensions to HOST-RESOURCES-MIB::hrStorageTable

This table has been extended to provide information about the CServer file system that includes the file system storage totals and used.

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost HOST-RESOURCES-MIB::hrStorageTableSNMP table: HOST-RESOURCES-MIB::hrStorageTable

hrStorageIndex hrStorageType hrStorageDescr hrStorageAllocationUnits hrStorageSize hrStorageUsed hrStorageAllocationFailures



1 HOST-RESOURCES-TYPES::hrStorageRam Physical memory 1024 Bytes 264524232 246219724 ? 3 HOST-RESOURCES-TYPES::hrStorageVirtualMemory Virtual memory 1024 Bytes 268620224 246219724 ?

... 7680 HOST-RESOURCES-TYPES::hrStorageRemovableDisk Cisco CServer Storage 2097152 Bytes 135448398 179575 ?

The file system byte size and bytes used can be obtained by multiplying the hrStorageAllocationUnits by hrStorageSize and hrStorageUsed respectively.

Extensions to UCD-SNMP-MIB::dskTable

As with HOST-RESOURCES-MIB::hrStorageTable, this table has been extended to include CServer file system information. Along with size (total) and used, this table includes an available count as well as a percentage used.

[root@utah97 ~]# snmptable -v2c -cpublic -M/opt/Supermicro/SuperDoctor5/mibs:/usr/share/snmp/mibs -mall localhost UCD-SNMP-MIB::dskTableSNMP table: UCD-SNMP-MIB::dskTable

dskIndex dskPath dskDevice dskMinimum dskMinPercent dskTotal dskAvail dskUsed dskPercent dskPercentNode dskTotalLow dskTotalHigh dskAvailLow dskAvailHigh dskUsedLow dskUsedHigh dskErrorFlag dskErrorMsg 1 / /dev/sda8 100000 -1 25728092 21299064 3122100 13 2 25728092 0 21299064 0 3122100 0 noError 2 /arroyo/db /dev/sda2 100000 -1 70555056 66755080 215976 0 0 70555056 0 66755080 0 215976 0 noError

... 7680 Cisco CServer Storage -1 -1 ? ? ? 0 ? 2520412160 64 2152642560 64 367769600 0 noError

This table shows sizes in kilobyte units. The SNMP values used to store the size are 32-bit values, and can only represent file systems less than 2 TB in size in a single value. To accommodate file systems of 2 TB or larger, this table has Low and High values that combine to hold a 64-bit value. In the example output above, the total byte size would be computed as:

((64 * 2^32) + 2520412160) * 1024 bytes = 284,055,878,762,496 bytes

Monitored ItemsThe default Net-SNMP configuration in /etc/snmp/snmpd.conf includes directives to monitor several items on the host COS node. This includes monitoring for a running cosd service, memory errors, Linux disk errors, and the Linux load average.

The default snmpd configuration has also been extended to include monitor directives to alert in the event of a CServer disk device status change, as well as to alert on overall CServer percentage of file system usage.

In the event that a disk status changes or file system usage percentage changes above or below certain thresholds, Net-SNMP generates a local trap that is received by the local snmptrapd process. The default behavior of snmptrapd is to log the event to /var/log/messages.



Disk device status change monitors are defined as follows:

monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in unknown state" hrDeviceStatus == 1monitor -S -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in running state" hrDeviceStatus == 2monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in warning state" hrDeviceStatus == 3monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in testing state" hrDeviceStatus == 4monitor -r 60 -o hrDeviceIndex -o hrDeviceDescr -o hrDeviceStatus "Cisco device in down state" hrDeviceStatus == 5

Net-SNMP monitors for disk status changes every 60 seconds with this configuration. The following is an example trap generated by Net-SNMP for a disk device with warning status:

Sep 10 12:15:59 utah97 snmptrapd[23614]: 2014-09-10 12:15:59 localhost.localdomain [UDP: [127.0.0.1]:45956->[127.0.0.1]]:Sep 10 12:15:59 utah97 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (131) 0:00:01.31 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: Cisco device in warning state DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: HOST-RESOURCES-MIB::hrDeviceStatus.7698 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 3 HOST-RESOURCES-MIB::hrDeviceIndex.7698 = INTEGER: 7698 HOST-RESOURCES-MIB::hrDeviceDescr.7698 = STRING: Cisco csd19 SAS HDD HOST-RESOURCES-MIB::hrDeviceStatus.7698 = INTEGER: warning(3)

Further details for device status can be obtained from the cddm utility. In this particular case, the device is in warning status due to an AVISORY S.M.A.R.T. status:

[root@utah97 ~]# cddm -s all 19--- csd19 ---alleged_media_errors 0bytes_read 3363291648bytes_written 31124533760companion 20connection 0.19device_type SAS HDD dev_link_rate 6000dev_max_operating_temp 69dev_max_operational_starts 1048576dev_mfg_wk 01dev_mfg_yr 2012dev_temp 40direct_submits 66679dubious_LBAs 0errors 0errors_reported 0errors_reported_rate 0hard_resets 0location 1.2.3.1media_error_rate 0media_errors 0model WD4001FYYG-01SL3name csd19proc_flags 0x20reqs_in_cb_queue 0reqs_in_progress 0reqs_lost 0reqs_queued 0requests 126882reset_rate 0resets 0



retries 0rev VR02sector_size 512serial WMC1F0762907slot 10smart_age 00.26.11smart_exceptions 0x5d.0x32.0x28 - Impending - data error rate too high; smart_glist_count 11smart_nonmedium_errors 0smart_rd_corrected_errors_long 0smart_rd_corrected_errors_short 0smart_rd_correction_algorithm_use 0smart_rd_retries 0smart_rd_total_bytes_processed 0smart_rd_total_corrected_errors 0smart_rd_uncorrected_errors 0smart_startups 10smart_status ADVISORYsmart_wr_corrected_errors_long 0smart_wr_corrected_errors_short 0smart_wr_correction_algorithm_use 0smart_wr_retries 0smart_wr_total_bytes_processed 0smart_wr_total_corrected_errors 0smart_wr_uncorrected_errors 0state 0x800007; DEV_ALLOCATED DEV_ATTACHED DEV_READY timeout_rate 0timeouts 0total_sectors 7814037167vendor WD

CServer file system usage monitors are defined as follows:

monitor -I -r 60 -o UCD-SNMP-MIB::dskPath.7680 -o UCD-SNMP-MIB::dskPercent.7680 "Cisco CServer Storage High" UCD-SNMP-MIB::dskPercent.7680 >= 98monitor -S -I -r 60 -o UCD-SNMP-MIB::dskPath.7680 -o UCD-SNMP-MIB::dskPercent.7680 "Cisco CServer Storage Normal" UCD-SNMP-MIB::dskPercent.7680 < 98

Net-SNMP monitors for high-usage events if the percentage used is 98 percent or above. Net-SNMP monitors for normal usage events if the percentage used falls below 98 percent.

The following is an example trap logged to /var/log/messages for high and normal usage status:

Sep 10 13:28:27 utah97 snmptrapd[23614]: 2014-09-10 13:28:27 localhost.localdomain [UDP: [127.0.0.1]:46969->[127.0.0.1]]:Sep 10 13:28:27 utah97 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (6137) 0:01:01.37 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: Cisco CServer Storage High DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::dskPercent.7680 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 98 UCD-SNMP-MIB::dskPath.7680 = STRING: UCD-SNMP-MIB::dskPercent.7680 = INTEGER: 98

Sep 10 13:30:27 utah97 snmptrapd[23614]: 2014-09-10 13:30:27 localhost.localdomain [UDP: [127.0.0.1]:46969->[127.0.0.1]]:Sep 10 13:30:27 utah97 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (18138) 0:03:01.38 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: Cisco CServer Storage Normal DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::dskPercent.7680 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 97 UCD-SNMP-MIB::dskPath.7680 = STRING: UCD-SNMP-MIB::dskPercent.7680 = INTEGER: 97



Manual customizations can be made to the snmpd and snmptrapd services to monitor additional items, and to forward traps and notifications to customer-specific operations.


Cisco Cl

A

P P E N D I X A
Reference Information
This section contains additional reference material for further understanding the COS system, and information on performing commonly executed tasks and system maintenance.

COS Service ModelThe COS service model is shown in the Unified Modeling Language (UML) diagram below:

Figure A-1 COS Service Model

A COS operator can assume one of the following roles:

• System Operator — Provisions the system resources and creates the Tenant. In Figure A-1, the system resources appear in yellow and the entities managed by the Tenant appear in green.

System Operator

Resources

Network Service Instance

EndPointAssetWorkflow

Template

Tenant

Policy

COS Cluster

IP PoolLogs

AuthProfile

SLA

ResourceSLA

CapacitySLA

Capture

ContentRedundancy

Use API

Storage

Status

Statistics

Events

Alarms

Diagnostics

DNS

NTP

COS Node

COS Cluster

COS NodeRegion

AuthorizationProfile

Zone

HighAvailability

*

*

*

*-Owns

-Owns

-creates

* -creates

*

* -creates

1

1

1

1

1

1

1

1

1*

1*

3830

81

1..*

A-1oud Object Storage Release 3.0.1 User Guide

Appendix A Reference Information Using the COS Service Manager GUI

• Tenant — Creates service instances using pre-existing service templates. The Tenant provisions a service instance by assigning system resources to the service instance and configuring it. When the service instance is activated, all related configurations are performed and the service becomes available, producing the outputs appearing in blue in Figure A-1.

Using the COS Service Manager GUIThe COS Service Manager GUI enables you to quickly and easily access many COS deployment and monitoring functions. This section provides basic login and navigation instructions for the GUI.

Steps for using the GUI to perform specific deployment and monitoring tasks are provided in chapters devoted to these topics.

Note The COS Service Manager GUI is described here as it appears in a COS-only deployment. When COS is deployed as a service of MOS, its GUI is presented as a subset of the MOS Service Manager GUI, and some navigational details differ accordingly.

Before You BeginPlease note the following before logging in and using the COS Service Manager GUI:

• The COS Service Manager GUI runs on the following operating systems and browsers:

– Windows Internet Explorer 9 (IE9) or later for Windows 7

– Mozilla Firefox 20 or later for Windows 7

– Google Chrome 30.x for Windows 7

– Apple Safari 7.x for Widows 7 on MAC OS Version 10.9 or later

• The GUI requires a display resolution of 1600 x 900 or better. If the PAM IP address on which you access the GUI is NATed, response time can be noticeably slow.

• The GUI times out after one our (60 minutes) of activity. To avoid timeouts, make and save a configuration change, or click a tab or refresh icon, before the one-hour idle timeout expires.

• Configuration changes that you make in any section of the GUI are saved when you click the Save button. If you log out of the GUI or if the GUI times out before you save your changes, all changes made since your last save are lost.

• If you try to delete an object in the GUI that has been associated with another object (for example, a capture endpoint that has been associated with an asset workflow), the GUI prevents the deletion and displays an explanatory error message.

• If you redeploy or upgrade the COS OVA, you must clear your browser cache for the GUI changes to take effect.

Logging InTo access the COS Service Manager GUI, perform the following steps:

A-2Cisco Cloud Object Storage Release 3.0.1 User Guide


Step 1 Access the GUI using the PAM IP address. The URL format is:

https://pam_ip_address:8443

Step 2 Log in to the GUI with your Username and Password.

The default username is admin. The default password is default. We recommend that you change these after deployment is complete.

Figure A-2 COS Service Manager GUI Login Page

The home page of the GUI appears, and you can now begin using the GUI.

Step 3 To exit the GUI from any page, click the Username displayed at upper right in the GUI and choose Log out from the drop-down menu.

Using the Home PageThe Home page appears by default, and provides a high-level overview information for COS. The Home page displays the following information, which is refreshed every 5 minutes.



Figure A-3 COS Service Manager GUI Home Page

Nodes Allocated

The Nodes Allocated section of the Home page displays COS node usage broken down by service type (such as Live or VOD), available nodes, and nodes that are in maintenance mode. You can click on sections of the chart to navigate to pages containing more details.

Figure A-4 Home Page - Nodes Allocated Section

Icons at the bottom of this section let you choose between chart view and table view, choose different chart types, and export the underlying data.



Alarm Summary

The Alarms Summary section of the Home page displays the number of alarms logged for each service type, further broken down by severity of the alarms (critical, major, or minor).

Figure A-5 Home Page - Alarm Summary Section

Icons at the bottom of this section let you choose between chart view and table view, choose different chart types, and export the underlying data.

Service Summary

Located below these two panes is a Service Summary table that lists all of the service instances currently defined for the deployment.

Figure A-6 Home Page - Service Summary Section

The Service Summary section displays the following information for each COS node (you cannot edit these fields):

• Service Type — Type of service instance, such as Live or VOD.

• Service Instance Name — Name of the service instance.

• Service State — Indicates whether the service instance is currently Enabled or Disabled.

• Nodes Allocated — Number of nodes allocated for use by the service instance.

• Nodes Used — Number of nodes currently being used by the service instance.

• App Status — Current status of the application associated with the service instance.

• Storage Status — Current status of the storage associated with the service instance.



• Alarms — Total number of alarms logged for the service instance.

• Critical — Number of Critical alarms logged for the service instance.

• Major — Number of Major alarms logged for the service instance.

• Warning — Number of Warning alarms logged for the service instance.

To display the endpoint summary for a service instance, click its Expand arrow.

Figure A-7 Home Page - Endpoint Summary Section

The Endpoint Summary table displays the following information for each endpoint (you cannot edit these fields):

• Endpoint Name — Name of the endpoint.

• Endpoint Type — Type of endpoint. Valid values are Capture and Playback.

• Nodes Allocated — Number of nodes allocated for use by the service instance.

• Nodes Used — Number of nodes currently being used by the service instance.

• Node Status — Current status of the node associated with the service instance.

• Storage Status — Current status of the storage associated with the service instance.

• Total Alarms — Total number of alarms logged for the service instance.

• Critical — Number of Critical alarms logged for the service instance.

• Major — Number of Major alarms logged for the service instance.

• Warning — Number of Warning alarms logged for the service instance.

To collapse the interface information, click the Collapse arrow.

Working with PagesThe COS Service Manager GUI presents detailed information on pages using tabs, fields, and tables. For example, clicking a Service Instance Name in the Endpoint Summary section takes you to the page shown in Figure A-8.



Figure A-8 Typical COS Service Manager GUI Page

The elements on this page a typical of those used throughout the GUI.

In this example:

• Tabs at the top of the page let you choose Service Definition or Overview information.

• A General section includes fields for information and a button for changing the service state.

• A table displays information about each service endpoint.

• A toolbar above the table lets you quickly perform needed functions, such as:

– Add, select and edit, or select and delete table entries.

– Sort entries by table column

– Change the order of the columns.

– Use filters to control what is displayed.

Toolbar Functions

This section describes the functions typically available for tables in the COS Service Manager GUI.

Add an Entry

To add an entry to a table, click Add Row at the top of the table. Enter or select values for the new entry, and then click Save to add it to the table.

Edit an Entry

To edit an entry in a table, use one of the following methods:

• Select it and click Edit at the top of the table.

• Click in any of the fields for the entry.



• Enter or select new values for the entry, then click Save to save the new values in the table.

Delete an Entry

To delete an entry form a table, select the entry and click delete at the top of the table. When prompted, click OK. The entry is removed from the table.

Refresh the Table

To refresh the entries in a table, click Refresh at the top of the table.

Sort Entries

To sort entries in a table based on the contents of a column, click the column header. Click once to order the table in alphanumeric ascending order. Click again for descending order.

Change Column Order

To change the order of columns in a table, drag-and-drop the column headings.

Filter Contents

To filter the contents of a table, choose a filter from the drop-down list at top right of the table, or click the Filter icon to toggle between different filters.

• All — Display all of the table contents. This is the default display.

• Quick Filter — Display only those elements of one or more columns that contain specified text strings.

• Advanced Filter — Display only those elements of the tale that match a set of one or more rules that you specify.

For each rule, specify a column; a type of filter, such as Contains or Starts with; and a text string.

To add another rule to the filter, click the plus sign (+). You can specify more than one rule for a given column.

If you have specified more than one rule, the Match drop-down list appears.

To specify that an element of the table must match all of the specified rules in order to be included in the list, select Match > All.

– To specify that an element of the table need only match one or more of the specified rules in order to be included in the list, select Match > Any.

– To delete a rule from the filter, click the minus sign (-).

– To apply the filter to the elements of the table, click Go.

– To clear the filter and display all elements of the table, click Clear Filter.

• Manage Preset Filters — Edit or remove Advanced Filters that you have saved.

Note When you refresh the pages in your browser, all saved filters are removed.



Using the Navigation PanelClicking the Navigation button at upper left in the GUI opens the Navigation panel.

Figure A-9 Home Page - Navigation Panel

Selecting the View

Use the icons at the top of the panel let you toggle between three views:

• The Navigation menu, open by default

• The Index, which includes a Search function

• The Favorites list

Bookmarking a Page

You can bookmark a page of interest and add it to the Favorites list. To do this, navigate to the page and click the Pin icon located to the right of the Favorites button.

Home Button

Click the Home button to return to the COS Service Manager GUI Home page from any other page. This also closes the Navigation panel.



Infrastructure

Click the Infrastructure button to open (or if open, to close) a list of pages for viewing and configuring various infrastructure components.

• Choose Compute > Nodes to add, delete, or change the status of nodes.

• Choose Storage > COS Nodes to configure COS nodes.

• Choose Storage > COS Clusters to configure clusters.

• Choose Storage > COS Container Stores to configure containers.

• Choose Storage > NAS Stores to configure NAS.

• Choose IP Address Pools and Networks to configure IP pools.

• Choose Platform Services to set up, edit, or view server information.

• Choose Software Image Manifests to review and configure image manifests.

• Choose Regions and Zones to display and configure regions and zones.

• Choose COS Overview to display bandwidth, session, storage, and node usage.

Note Instructions related to all of the above tasks are provided in Deploying COS, page 2-1.

Service Domain Objects

To view, add, edit, or delete COS service domain objects, select Service Domain Objects, and then select one of the following:

• Choose Media Sources > Channels to work with channels.

• Choose Media Sources > NAS Media Sources to work with NAS media sources.

• Choose Media Sources > Dynamic Sources to work with dynamic sources.

• Choose Asset Lineups > Channel Lineups to work with channel lineups.

• Choose Asset Lineups > Dynamic Lineups to work with dynamic lineups.

• Choose Profiles > Auth Profiles to work with authorization profiles.

• Choose Profiles > ESAM Profiles to work with ESAM profiles.

• Choose Profiles > Key Profiles to work with key profiles.

• Choose Policies > HTTP Header Policies to work with HTTP header policies.

• Choose Policies > Asset Lifecycle Policies to work with asset lifecycle policies.

• Choose Policies > Asset Redundancy Policies to work with asset redundancy policies.

• Choose Templates > Publish Templates to work with publish templates.

Services

To view, add, edit, or delete COS service domain objects, select Services, and then choose Services > Cloud Object Store to work with the COS service.



Note After choosing Services > Cloud Object Store, an Overview tab appears that lists all Cache Interfaces. If an interface is in Active status, the associated port is ready for a connection. However, if there are any issues with the port, they are listed on the Fault Detail tab only.

Note Additional menu options such as Services > Unified Media Service indicate a COS + MOS deployment. See the Cisco Media Origination System User Guide, Software Version 2.4 for instructions on using these options.

Monitoring

To access COS monitoring features, select Monitoring, and then select one of the following

• Choose Alarms > Active to access a list of currently active alarms.

• Choose Alarms > History to access a list of all alarms, both active and inactive.

• Choose Diagnostic Settings > System Diagnostics to access system diagnostics.

• Choose Diagnostic Settings > Service Diagnostics to access service diagnostics.

• Choose Events to access a list of significant system events.

• Choose Logs > Log Viewer to display the event log.

• Choose Logs > Log Analysis to perform event log analysis.

• Choose Configuration Viewer to access an interactive and searchable view of network objects, relationships, and configurations. Choose between tree view and graph view, then click any object to view its configuration.

Note Step-by-step instructions related to all of the above tasks are provided in System Monitoring, page 3-1.

Managing GUI UsersCOS stores the usernames and passwords for the GUI in the file etc/opt/cisco/cos/public/users.properties stored on the PAM VM.

Note We recommend that you change the default username and password following installation.

COS provides the following scripts to help you manage usernames and passwords:

• To list current users, enter the following command:

listusers-gui

• To add a new user, enter the following command:

adduser-gui username password

• To change an existing user, enter the following command:

chpass-gui username password


Appendix A Reference Information COS Network Ports and Services

• To delete a user, enter the following command:

deluser-gui username

You can enter multiple commands when adding, changing, or deleting multiple users or passwords. For example, if you are adding two users, changing two users, and deleting one user, you can enter the following sequence of commands:

adduser-gui username1 password1adduser-gui username 2 password2chpass-gui username3 password3chpass-gui username4 password4deluser-gui username5

Usernames and passwords for the COS GUI:

• Can be strings of any length.

• Can include uppercase or lowercase letters, numbers, and any special characters. However, the characters <, >, |, !, and & must be preceded by a backslash.

• Are not case-sensitive.

COS Network Ports and ServicesThe following table identifies open network ports for COS nodes and the services that own these ports.

Table A-1 COS Network Ports and Services

Scope/Interface Port Purpose Owning Service

Management TCP 7000 Cassandra internode communication cassandra

Management TCP 7199 Cassandra JMX communication cassandra

Management TCP 9042 Cassandra CQL native transport port cassandra

Management TCP 9160 Cassandra Thrift client API cassandra

Management TCP 9090 Cosd request listener cosd

Data TCP 80 HTTP traffic for Swift and Swauth interfaces

cserver

Data UDP 3478 STUN traffic cserver

Data UDP 48879 Internal COS node communication cserver

Data UDP 57005 Internal COS node communication cserver

Management UDP 123 Network Time Protocol ntpd

Management TCP 25 Postfix mail system traffic postfix

Management TCP 5666 SuperDoctor5 Nagios Remote Plugin Extender (NRPE)

sd5

Management TCP 5999 SuperDoctor5 Nagios Remote Plugin Extender (NRPE)

sd5

Local TCP 31000-32999 * SuperDoctor5 internal communication sd5

Management UDP 7777 SuperDoctor5 internal communication sd5

Local TCP 199 Simple Network Management Protocol snmpd

Management UDP 161 Simple Network Management Protocol snmpd


Appendix A Reference Information COS Maintenance

* The sd5 service chooses a single available port in the range of 31000-32999 on startup.

COS MaintenanceIt may be necessary to reboot or shut down a COS node for conditions such as routine maintenance. On such occasions, be sure to avoid any impact to services provided by the affected COS cluster.

You can reboot or shut down a COS node from the command line or, with limitations, from the COS Service Manager GUI. This section describes both methods.

Command Line RebootTo reboot a COS node, execute the reboot command from a terminal console or remote shell. The system will begin a shutdown phase to reboot.

• Any active HTTP and TCP sessions with the COS node data network interfaces will be reset by the node. The client will have the responsibility of retrying operations with the remaining COS nodes in the cluster.

• The COS management system will automatically update the DNS registry to remove listings for the COS node.

• The COS services will automatically be restarted when the system is back online after reboot.

• If the COS cluster has been configured to replica data, copies of any object data residing on the COS node will be accessible from the remaining COS nodes.

Service Manager GUI RebootYou can change a COS node from In Service to Maintenance status from the GUI as shown in the example of Figure A-10.

Note Changing the node to Maintenance status from the GUI causes the script cserver-control stop to run, placing the Cserver system service in maintenance mode.

Management UDP 162 SNMP Traps snmpdtrapd

Management TCP 22 Secure Shell (SSH) ssh

Table A-1 COS Network Ports and Services

Scope/Interface Port Purpose Owning Service



Figure A-10 Placing a COS Node in Maintenance Mode from Service Manager GUI

Node Decommissioning and RemovalCOS 3.0.1 lets you decommission a node at the Cserver level. Decommissioning tells Cserver to copy the data objects of the node to other nodes in the cluster until the target number of mirror copies is reached.

Decommissioning is currently a CLI-only operation. To decommission a node, run the script cserver-control.pl decommission, installed on the node at /opt/cisco/cos-aic-client/cserver-control.pl.

Decommissioning can take several hours, and for this reason, the CLI does not monitor for completion of decommissioning. To check the progress of the decommission, use the command cserver-control.pl decommission --stats as needed until it confirms that the operation is complete.

After decommission is complete, you can safely remove the node from the cluster either via the GUI or the API.

Figure A-11 COS Node Removal from Service Manager GUI



Note A node cannot be decommissioned after it has been removed from a cluster using the GUI or API, so you must decommission a node before removing it.

Behavior of COS Services on COS Node BootThere are four primary system services that provide COS functionality on a COS node: cassandra, cosd, cos_aicc, and cserver. These services can be manipulated using standard Linux system service tools.

To prevent these services from starting automatically on a COS node boot, execute the following commands as the root user from a shell on that node:

[root@cos-node-1 ~]# chkconfig cassandra off[root@cos-node-1 ~]# chkconfig cosd off[root@cos-node-1 ~]# chkconfig cos_aicc off[root@cos-node-1 ~]# chkconfig cserver off

To enable automatic service loading on node boot, execute the following commands:

[root@cos-node-1 ~]# chkconfig cassandra on[root@cos-node-1 ~]# chkconfig cosd on[root@cos-node-1 ~]# chkconfig cos_aicc on[root@cos-node-1 ~]# chkconfig cserver on

To view the current state of service loading, execute the following command:

[root@cos-node-1 ~]# chkconfig --listauditd 0:off 1:off 2:on 3:on 4:on 5:on 6:offcos_aicc 0:off 1:off 2:on 3:on 4:on 5:on 6:offcosd 0:off 1:off 2:on 3:on 4:on 5:on 6:offcassandra 0:off 1:off 2:on 3:on 4:on 5:on 6:offcrond 0:off 1:off 2:on 3:on 4:on 5:on 6:offcserver 0:off 1:off 2:on 3:on 4:on 5:on 6:offip6tables 0:off 1:off 2:on 3:on 4:on 5:on 6:offiptables 0:off 1:off 2:on 3:on 4:on 5:on 6:offnetconsole 0:off 1:off 2:off 3:off 4:off 5:off 6:offnetfs 0:off 1:off 2:off 3:on 4:on 5:on 6:offnetwork 0:off 1:off 2:on 3:on 4:on 5:on 6:offntpd 0:off 1:off 2:off 3:off 4:off 5:off 6:offntpdate 0:off 1:off 2:off 3:off 4:off 5:off 6:offpostfix 0:off 1:off 2:on 3:on 4:on 5:on 6:offrdisc 0:off 1:off 2:off 3:off 4:off 5:off 6:offrestorecond 0:off 1:off 2:off 3:off 4:off 5:off 6:offrsyslog 0:off 1:off 2:off 3:off 4:off 5:off 6:offsaslauthd 0:off 1:off 2:off 3:off 4:off 5:off 6:offsshd 0:off 1:off 2:on 3:on 4:on 5:on 6:offsyslog-ng 0:off 1:off 2:on 3:on 4:on 5:on 6:offudev-post 0:off 1:on 2:on 3:on 4:on 5:on 6:off

To manually start the services, execute the following commands:

[root@cos-node-1 ~]# service cassandra start[root@cos-node-1 ~]# service cosd start[root@cos-node-1 ~]# service cos_aicc start[root@cos-node-1 ~]# service cserver start


Appendix A Reference Information COS Service Reliability

COS Service ReliabilityThis section describes the response of the COS AIC server to changes in the states of disks, interfaces, and services on COS nodes. The COS AIC client and the Service Monitor convey the changes in state by sending appropriate events to the COS AIC server.

If a change is critical and indicates that the node cannot service requests, the AIC server ensures that the node’s interfaces are not part of the DNS, so that service requests are not addressed to the node.

COS Node DisksWhen the COS AIC server receives an event indicating the failure of all the disks on a COS node, it checks the status of the node in its records.

• If the node status is down, the server removes the node’s interfaces from the DNS server.

• If the node status is up, no action is necessary.

When the COS AIC server receives an event indicating some or all of the disks on the COS node are functional, it checks the status of the node in its records.

• If the node status is down, the server adds the node’s interfaces back to the DNS server.


COS ServicesWhen the COS AIC server receives an event indicating that one or more of the COS services — Cisco Cache Server (CServer), Cisco Cloud Object Storage Daemon (cosd), and Cassandra Server — are not functional, it checks the status of the node in its records.



When the COS AIC server receives an event indicating all the COS services are operational, it checks the status of the node in its records.



COS Node Interfaces When the COS AIC server receives an event indicating that one or more interfaces are not functional, along with a list of the faulty interfaces, it checks the status of these interfaces in its records.

• If the fault interfaces are part of the DNS, they are removed from it.

• If an interface is not in the list of faulty interfaces, and is not part of the DNS, the AIC server adds the interface to the DNS server.


Appendix A Reference Information COS Node Hard Drive Replacement

Server ReachabilityThe Service Monitor periodically receives the status of a COS node from the COS AIC client on that node. If the Service Monitor does not receive status information for a COS node within the stipulated time, it determines that the node is not reachable. Consequently, it sends an event to the COS AIC server indicating that the node is down. In response, the server checks the status of the node in its records.



When the Service Monitor receives status information for a COS node, indicating that the node is reachable, it sends an event to the COS AIC server conveying that the node is up. In response, the server checks the status of the node in its records.



COS Node Hard Drive ReplacementCOS software can be deployed currently on Cisco UCS C-3160 Rack Server. For instructions on replacing hard drives on these platforms, see the Cisco UCS C-3160 Rack Server Installation and Service Guide.

Replicating Objects During Swift Write OperationsWhile an object is being created or modified using Swift write operations, copies of the object data can be stored in real time on the local COS node as well as the node’s peers in the COS cluster. This functionality works only if the RAID feature on the node is disabled.

To disable the RAID feature on the node, open /arroyo/test/setupfile and set allow vault raid to 0.

To replicate object data on the node, open /arroyo/test/setupfile and set vault local copy count to a value greater than 1. This value specifies the how many copies of the object data are to be stored on the node.

To replicate object data on the peer nodes, open /arroyo/test/setupfile and set vault mirror copies to a value greater than 1. This value specifies how many remote copies of the object data are to be maintained.


http://www.cisco.com/c/en/us/td/docs/video/cds/cde/cde205_220_420/installation/guide/cde205_220_420_hig.html

Appendix A Reference Information Replicating Objects During Swift Write Operations


Cisco Cl

A

P P E N D I X B
Troubleshooting
Viewing the Statuses of Primary System ServicesThe status of the primary system services that provide COS services on a COS node can be viewed by executing the following commands:

[root@utah97 ~]# service cassandra statuscassandra is running[root@utah97 ~]# service cosd statuscosd (pid 9235) is running...[root@utah97 ~]# service cserver statuscserver is running

Service High-AvailabilityEach COS node leverages the monit framework to provide high-availability of the cos service. In the event that the cosd process crashes, the monit framework will restart the service.

Troubleshooting Swift and Swauth API ErrorsIf a Swift or Swauth API operation returns a 500 level HTTP error status, this can be an indication of an issue with one of the three primary system services — cassandra, cosd, or cserver. These error statuses are sometimes returned due to temporary resource exhaustion on the COS node. However, if the error statuses persist for a long period of time, verify that the primary system services are running. See Viewing the Statuses of Primary System Services, page B-1.

Information helpful in tracing and analyzing Swift, Swauth, and Cassandra transactions can be found in the following log files collected on each individual COS node:

• /arroyo/log/http.log.<DATE>

– This log collects HTTP transaction information from Swift and Swauth operations.

• /arroyo/log/cosd.log.<DATE>.

– This log records Cassandra transactions that are executed when a client invokes a Swift or Swauth API operation.

Note When debugging, it may be helpful to raise the HTTP log level of the service in question using the command echo 9 > /proc/calypso/tunables/http_log_level.

B-1oud Object Storage Release 3.0.1 User Guide

Appendix B Troubleshooting Unresponsive COS Service Manager GUI

Each COS node also includes a cos_stats utility that can be executed from a shell. The utility reports the current operational state of the COS node with respect to Swift and Swauth operations. This utility also provides information on resource and network utilization. The following is a sample output obtained by executing the cos_stats utility.

Fri Jun 27 13:23:12 PDT 2014----------------------------------------------------------------------------| Operation | Active | Ops/Sec | Errors | Total |----------------------------------------------------------------------------| SWAUTH Account Create | 0 | 0.0 | 0 | 6 || SWAUTH Account Delete | 0 | 0.0 | 0 | 0 || SWAUTH Account Meta | 0 | 0.0 | 0 | 0 || SWAUTH User Create | 0 | 0.0 | 0 | 6 || SWAUTH User Delete | 0 | 0.0 | 0 | 0 || SWIFT Container Create | 0 | 0.0 | 0 | 0 || SWIFT Container Delete | 0 | 0.0 | 0 | 0 || SWIFT Container List | 0 | 0.0 | 0 | 0 || SWIFT Container Meta | 0 | 0.0 | 0 | 3298 || SWIFT Object Delete | 0 | 0.0 | 0 | 0 || SWIFT Object List | 0 | 0.0 | 0 | 0 || SWIFT Object Meta | 0 | 0.0 | 0 | 0 || SWIFT Object Read | 3999 | 1.9K | 0 | 13818 || SWIFT Object Write | 0 | 0.0 | 0 | 6694580 || SWIFT Token Get | 0 | 0.0 | 0 | 6 |----------------------------------------------------------------------------| Eth | Sessions | TCP RX bps | TCP RX pps | TCP TX bps | TCP TX pps |----------------------------------------------------------------------------| eth6 | 2004 | 196.320M | 407.518K | 8.511G | 716.957K || eth7 | 1024 | 170.025M | 352.604K | 8.239G | 688.469K || eth8 | 1095 | 159.576M | 328.401K | 8.038G | 673.944K || Total | 4123 | 525.920M | 1.089M | 24.788G | 2.079M |----------------------------------------------------------------------------| Poll: 0% : 823K/s | NetInt: 62% | RX: 37% | TCP TX: 74% | FQ: 0% |----------------------------------------------------------------------------

Unresponsive COS Service Manager GUIIf the COS Service Manager GUI is not responding, you should check COS AIC service status on the COS Service Manager CLI.

Step 1 Login to the COS Service Manager CLI.

Step 2 To view the status of all running services, execute this command: supervisorctl

Step 3 Verify the status of the cos-controller. This represents the status of the COS AIC.

[root@PM-35-C90 supervisor]# supervisorctl cli_insVmConfig_app RUNNING pid 3661, uptime 10 days, 14:51:03cli_mongoWatcher RUNNING pid 3656, uptime 10 days, 14:51:03cli_mosdns_app RUNNING pid 3655, uptime 10 days, 14:51:03cli_pamds_app RUNNING pid 3659, uptime 10 days, 14:51:03cli_platformservice_app RUNNING pid 3658, uptime 10 days, 14:51:03cli_restapi_app RUNNING pid 3654, uptime 10 days, 14:51:03cli_vmLoadBalancer_app RUNNING pid 3664, uptime 10 days, 14:51:03cli_zone_app RUNNING pid 3660, uptime 10 days, 14:51:03cos-controller RUNNING pid 30717, uptime 0:43:01docserver_app RUNNING pid 3644, uptime 10 days, 14:51:03drm-system_app RUNNING pid 3650, uptime 10 days, 14:51:03mos-controller_app RUNNING pid 3648, uptime 10 days, 14:51:03

B-2Cisco Cloud Object Storage Release 3.0.1 User Guide

Appendix B Troubleshooting Viewing COS AIC Logs

redis RUNNING pid 3640, uptime 10 days, 14:51:03roleserver_app RUNNING pid 3653, uptime 10 days, 14:51:03sm_app RUNNING pid 3646, uptime 10 days, 14:51:03statsd RUNNING pid 3643, uptime 10 days, 14:51:03unified_translogd RUNNING pid 3638, uptime 10 days, 14:51:03

Viewing COS AIC LogsIn addition to the COS Status and COS Events reported in COS Service Manager GUI, detailed COS AIC logs can be found on the COS Service Manager CLI. There are three logs:

• /var/log/opt/cisco/mos/errorlog/COSController-errorlog.current

– This log records COS AIC execution information.

• /var/log/supervisor/cos-aic.log

– This log records COS AIC related stdout information. Typically, any node.js related error messages written to stdout are captured in this log.

• /var/log/supervisor/cos-aic.err.log

– This log records COS AIC related stderr information. Typically, any node.js related error messages written to stderr are captured in this log.


Appendix B Troubleshooting Viewing COS AIC Logs


Cisco Cl

A

P P E N D I X C
COS Tunables
Tunables are configuration parameters for CServer, which is a core component in COS. This appendix describes three categories of COS tunables:

• Debug Tunables

• Info Tunables

• Configuration Tunables

Debug TunablesDebug tunables are used in production environment to debug issues. Changes to these tunables affect the behavior of the system.

Table C-1 lists the Debug tunables.

Table C-1 Debug Tunables

Tunable name/LocationRead/Write/RW Description Units/Type

Default Value

shutdown_adapter_index RW Brings down the designated adapter. Echo X is the input command line used. For example, Echo 2 brings down the second adapter. This affects driver code as well. This is equivalent to if down.

Index ‐1

/proc/calypso/internal/enable_receive_capture

RW Enables logging of the received packets on the server.

Related tunable: /proc/calypso/internal/enable_transmit_capture

functionality (enable/disable)

0

/proc/calypso/internal/enable_transmit_capture

RW Enable logging of the transmitted packets from the server.

Related tunable: /proc/calypso/internal/enable_receive_capture


0

C-1oud Object Storage Release 3.0.1 User Guide

Appendix C COS TunablesInfo Tunables

Info TunablesThe Info tunables are used to get a snapshot of various parameters in the system.

Table C-2 lists the Info tunables.

/proc/calypso/internal/reset_adapter_index

RW Forces execution of the Adapter Reset Logic on a specified adapter index.


0

/proc/calypso/tunables/http_log_level

RW Enables verbose level logging on the http.log.

LogLevel LOG_I

NFO

Table C-1 Debug Tunables

Tunable name/LocationRead/Write/RW Description Units/Type

Default Value

Table C-2 Info Tunables

Tunable name/Location

Read/Write/RW Description Units/Type

Default Values

/proc/calypso/cm/cache2app_operational

Read Checks whether the cache2app services are operational or not.

Note This tunable is no longer used.

Status Value

0

/proc/calypso/cm/cachehits

Read Validates the number of objects fully present on the server.

Count 0

/proc/calypso/cm/cachemisses

Read Checks the number of objects not fully present on the server.

Count 0

/proc/calypso/cm/diskreadsectors

Read Counts the number of sectors read from the disk after starting a stream.

Count 0

/proc/calypso/cm/diskreadsegments

Read Counts the number of object segments read from the disk for streaming.

Count 0

/proc/calypso/cm/freedsegments

Read Counts the total number of free object segments.

Count 0

/proc/calypso/cm/memoryhits

Read Traces the memory hits of the object. Count 0

/proc/calypso/cm/romreadsectors

Read Reports the length of the object received from ROM in sectors.

Count 0

/proc/calypso/cmm/mapped/availableblocks

Read Reports the amount of memory that is available for use.

Count 0

/proc/calypso/cmm/mapped/totalblocks

Read Counts the total number of memory blocks on the system.

Count 0

C-2Cisco Cloud Object Storage Release 3.0.1 User Guide


/proc/calypso/status/bwm_settings

Read Queries the status of BandwidthManager and the thin pipes configured.

String N/A

/proc/calypso/status/diskbandwidthinfo

Read Returns verbose details on the disk bandwidth information.

String N/A

/proc/calypso/status/diskinfo

Read Returns verbose details on the configured disks on the server.

String N/A

/proc/calypso/status/hw_validation

Read Validates whether the hardware configured is according to the model of the device.

String N/A

/proc/calypso/status/networkbandwidth

Read Returns network bandwidth related information.

String N/A

/proc/calypso/status/pipe_settings

Read Reports the connected number of thin pipes.

String N/A

/proc/calypso/status/resiliencyinfo

Read Queries the configured ServiceAddress information (Setup or Control Service, HTTP Redirector Service, and Bandwidth Manager Service).

String N/A

/proc/calypso/status/resiliencystatus

Read Returns verbose details on the configured ServiceAddress in ACTIVE state.

String N/A

/proc/calypso/status/server_settings

Read Queries the server configurations and other details related to the server.

string N/A

/proc/calypso/status/streamer/activestreamslots

Read Holds a count of the streams currently streamed from this server.

Count 0

/proc/calypso/status/streamer/allocatedstreamslots

Read Counts the “Allocated streams,” that is, the streams that have been created. The sum of the allocated streams from this streamer group should be close to the value in the alloc field in the LRR line present in proto log.

Count 0

/proc/calypso/status/streamer/cutthruinfo

Read Displays the limitation on the disk write allowed for the object based on the PopularityBased Caching design.

This tunable gathers information about CutThruMark that is used in CServer to determine the disk write required.

Count 0

/proc/calypso/status/streamer/fillinfo

Read Logs the CCP Fill status. String N/A

Table C-2 Info Tunables (continued)



Default Values



/proc/calypso/status/streamer/resiliencyinfo

Read Queries the configured ServiceAddress information (Setup or Control Service, HTTP Redirector Service, and Bandwidth Manager Service).

String N/A

/proc/calypso/status/streamer/resiliencystatus

Read Returns verbose details on the configured ServiceAddress.

String N/A

/proc/calypso/status/streamer/streaminfo

Read Queries the stream related information. String N/A

/proc/calypso/status/syslog/last_open_error

Read Stores the return value if a file cannot be opened through syslog layer.

Integer 0

/proc/calypso/status/syslog/last_write_error

Read Stores the return value if a write to syslog fails.

Integer 0

/proc/calypso/status/syslog/multiple_writes

Read Reports the number of times a log message required multiple writes to the syslog socket. Generally, it should be possible to send one full message in one single write to the syslog socket.

Integer 0

/proc/calypso/status/syslog/open_errors

Read Reports the number of times the system encountered errors at the time of opening a log file using the syslog API.

Integer 0

/proc/calypso/status/syslog/write_errors

Read Reports the number of times log messages have failed to write to the syslog layer.

Integer 0

/proc/calypso/status/vault/fillinfo

Read Queries the CCP Fill status on the vault. String N/A

/proc/calypso/status/vault/httpinfo

Read Displays the configured HTTP Adapter information.

String N/A

/proc/calypso/status/vault/laninfo

Read Displays the details of Capacity, Connections, Inbound Request, and Outbound Request.

String N/A

/proc/calypso/status/vault/settings

Read Displays vault settings such as smoothing, mirroring, local copy, and remote mirror copy

String N/A

/proc/calypso/status/vault/streaminfo

Read Displays stream-related information for the vault.

String N/A

/proc/calypso/status/cache/fillinfo

Read Queries the CCP Fill status of the cache. String N/A

/proc/calypso/status/cache/httpinfo

Read Displays the configured HTTP Adapter information.

String N/A

/proc/calypso/status/cache/streaminfo

Read Displays stream-related information for the cache.

String N/A




Default Values


Appendix C COS TunablesConfiguration Tunables

Configuration TunablesThe Configuration tunables are used to configure the various parameters in the system. These tunables have a GUI counterpart.

Table C-3 lists the Configuration tunables.

/proc/calypso/tunables/runtimedebuggerversion

Read Provides information related to the version of runtimedebugger used in the system.

Version 1

/proc/calypso/internal/ingest_vault_max_display_count

Read Counts the maximum number of ingesting vaults configured to display statistics.

Count 10




Default Values

Table C-3 Configuration Tunables


Read/Write/RW type Description Units/Type

default Values

/proc/calypso/cm/active_trickspeed_0

Read Reads the configured trick speed 0, which is configured from CDSM.

Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



Speed/Hex 0



/proc/calypso/test/arp_readroutingtable

Write Reads and updates the entries from the RoutingTable.

To make a change to a routing table entry take effect, enter echo 1 > /proc/calypso/test/arp_readroutingtable.

Boolean 0

/proc/calypso/test/readarptable

Write Reads and updates the entries in the configuration file /arroyo/test/ArpTable.

To make a change to an ArpTable entry take effect, enter echo 1 > /proc/calypso/test/readarptable.

Boolean 0

/proc/calypso/test/readfillsourceconfig

Write Reads and updates the entries in the configuration file /arroyo/test/FillSourceConfig.

To make a change to a FillSourceConfig entry take effect, enter echo 1 >/proc/calypso/test/readfillsourceconfig.

Boolean 0

/proc/calypso/test/readrecordingchannelmap

Write Reads and updates the entries in the configuration file /arroyo/test/RecordingChannelMap.

To make a change to a RecordingChannelMap entry take effect, enter echo 1 >/proc/calypso/test/readrecordingchannelmap.

Boolean 0

/proc/calypso/test/readremoteservers

Write Reads and updates the entries in the configuration file /arroyo/test/RemoteServers.

To make a change to a RemoteServers entry take effect, enter echo 1 >/proc/calypso/test/readremoteservers.

Boolean 0

/proc/calypso/test/readstreamdestinationmap

Write Reads and updates the entries in the configuration file /arroyo/test/StreamDestinationMap.

To make a change to a StreamDestinationMap entry take effect, enter echo 1 >/proc/calypso/test/readstreamdestinationmap.

Boolean 0




default Values



/proc/calypso/test/readsubnettable

Write Reads and updates the entries in the configuration file /arroyo/test/SubnetTable.

To make a change to a SubnetTable entry take effect, enter echo 1 >/proc/calypso/test/readsubnettable.

Boolean 0

/proc/calypso/test/readtgidtable

Write Reads and updates the entries in the configuration file /arroyo/test/TransportGroupIdTable.

To make a change to a TransportGroupIdTable entry take effect, enter echo 1 >/proc/calypso/test/readsubnettable.

Note This tunable is no longer used.

Boolean 0

/proc/calypso/test/shutdown

Write Shuts down all processes running in the system.

Boolean 0

/proc/calypso/tunables/trickspeed_0

Write Sets both positive and negative trick speed 0 as a pair.

Because this tunable does not have directional information, there cannot be different positive and negative speeds.

Note This is an older version tunable.

Speed/Hex -1





Speed/Hex -1





Speed/Hex -1




default Values



/proc/calypso/tunables/trickspeedv2_0

Write Sets positive or negative trick speed 0. Speed/Hex -1























/proc/calypso/tunables/stream_null_packets

Write Enables or disables the stream null packet logic at the remaining length.

This tunable fills the remaining length with null packets, while adding the stream data in the stream slot.

Boolean 1

/proc/calypso/tunables/stream_using_qam

Write Configures the IP or QAM based setup.

If the flag is not set, the IP setup and Motorola PreEncryption use the original program number for the Encyrption data (Inband).

If the flag is set, the QAM based setup and Motorola PreEncryption reference the program number and data from the port (Outband).

Boolean 1




default Values



/proc/calypso/tunables/bwm_readConfFile

Write Reads and updates the entries in the configuration file /arroyo/test/BandwidthManager.

To make a change to a BandwidthManager entry take effect, enter echo 1 >/proc/calypso/tunables/bwm_readConfFile.

Boolean 0

/proc/calypso/tunables/cm_playnowatlivepoint

Write Enables or disables play at the live point. When enabled, this tunable allows for play at the live point.

Boolean 1

/proc/calypso/tunables/convert_npt_using_pts

Write Improves accuracy of Play computation by using actual frame PTS information rather than an NPT offset.

When enabled, the NPT computation uses actual frame PTS information to improve play accuracy.

Boolean 1

/proc/calypso/tunables/disable_fade_frame

Write Enables or disables the frame fading option at the file transition.

When enabled, there is no extra fade frame inserted at the file transition.

When disabled, 4 fade frames are inserted at the splice point.

Boolean 0

/proc/calypso/tunables/ftpoutmaxrate

Write Configures the ftpmax rate for total active FTP sessions.

Hex/bps 19000000

/proc/calypso/tunables/ftpoutmaxsessions

Write Configures the number of FTP sessions. No of session/Hex

0

/proc/calypso/tunables/popularity_half_life

Read Affects how often CServer plays the content. The popularity_half_life value determines the rate of increase of the bump-on-play value.

The bump-on-play value is managed by the class BumpOnUse, which resides in cm/PopularityValue and is set via CDSM.

Sec/Hex A880

/proc/calypso/tunables/read_etc_hosts

Write Reads the ETC hosts from /etc/hosts. Boolean 1




default Values



/proc/calypso/tunables/pipe_add

Write Adds the new pipe with any reboot. Pipe format Empty

Format:

pipe <pipeName> transmit <bw in Mbps> receive <bw in Mbps> [httpnetwork <ip>/<netmask>]

. . . .

[arrayid|serverid <id>] > /proc/calypso/tunables/pipe_add

/proc/calypso/tunables/pipe_del

Write Deletes the existing pipe. Pipe id Empty

Format:

<pipeName> > /proc/calypso/tunables/pipe_del




default Values



/proc/calypso/tunables/pipe_modify

Write Modifies the existing pipe configuration.

Pipe format Empty

Format:

pipe <pipeName> transmit <bw in Mbps> receive <bw in Mbps> [httpnetwork <ip>/<netmask>]

. . . .

[arrayid|serverid <id>] > /proc/calypso/tunables/pipe_modify

/proc/calypso/tunables/stream_from_local_array

Write Enables streaming content from a local server. This tunable is intended specifically to control the server.

Boolean 0

/proc/calypso/tunables/vaultdecommission

Write Decommissions the vault. When the vault is removed, this tunable migrates all information to another vault and logs the status under decommission information.

Boolean 0

/proc/calypso/tunables/vaultmirrorcopies

Read Reads the number of mirror copies that need to be present on the site from the setup file.

Number of copies

2




default Values



/proc/calypso/tunables/bwm_updateInterval

Write Defines the time interval at which Bandwidth management updates are exchanged by the servers.

At each update interval, the servers report their bandwidth usage to the primary server, and the primary assigns bandwidth to other servers as needed.

This tunable also reports BMW when the committed bandwidth reaches first 90% of total bandwidth, and again when it reaches 95% of total bandwidth.

Seconds 10

/proc/calypso/internal/offline

Write Takes the server offline when the designated server goes offline.

Boolean 0

/proc/calypso/tunables/bPSIErrorFailsIngest

RW If enabled, ensures that a trick code aborts the ingest if the bitstream does not contain PAT/PMT information.

Boolean TRUE

/proc/calypso/tunables/bRateErrorFailsIngest

RW If enabled, causes the ingest to fail if the ingest bit rate cannot be computed.

Boolean TRUE

/proc/calypso/tunables/nCCErrorsFailIngest

RW Defines the number of CC errors allowed after which the ingest is marked as failed.

Number None

/proc/calypso/tunables/nDiscErrorsFailIngest

RW Defines the number of discontinuity errors allowed after which the ingest is marked as failed.

Number None

/proc/calypso/tunables/nPicGapsFailIngest

RW Defines the number of PicGaps allowed after which the ingest is marked as failed.

Number 3

/proc/calypso/tunables/nPicGapsinHSFailIngest

RW Defines the duration of a PicGap in hundredths of a second that is counted toward an ingest failure.

Time in hundredth of seconds

5 (seconds)

/proc/calypso/tunables/nSyncLossesFailIngest

RW Defines the number of sync losses allowed after which the ingest is marked as failed.

Number 3

/proc/calypso/tunables/nSyncLossesinHSFailIngest

RW Defines the duration of a sync loss in hundredths of a second that is counted toward an ingest failure.

Time in hundredth of seconds

5 (seconds)

/proc/calypso/tunables/maintenancemirrorcopies

RW Defines the number of copies for mirroring during server maintenance.

Number 2

/proc/calypso/tunables/enable_rtp_rtx

RW Enables VOD error repair feature with RTP RTX.

Boolean default 0

Range: 0,1




default Values



/proc/calypso/tunables/nEveryNMinutes

RW Allows ingestion depending on the ingestknob settings, such as a check of a sample based on the time specified.

Minutes 30

/proc/calypso/tunables/tm_restrict_enable_ffw_resume

Write Enables or disables the streamer resuming fast forward after playing fast forward trick restricted content in 1x.

Boolean FALSE

/proc/calypso/tunables/tm_restrict_enable_forward_jump

Write Disables or enables forward jump restriction.

Boolean FALSE

/proc/calypso/tunables/tm_restrict_enable_forward_jump_resume

Write Enables or disables the streamer resuming forward jump after playing forward jump restricted content in 1x.

Boolean FALSE

/proc/calypso/tunables/tm_restrict_enable_rewind_skip

Write Enables or disables rewind skip. If enabled, rewind jumps to the beginning of the restricted content and continues rewinding from there.

Boolean FALSE

/proc/calypso/tunables/vaultarraymirror

Write Configures mirror partner, array id to which the server is mirroring.

To stop the array mirroring, enter echo <negative value of the partner array id>.

Related tunable:

/proc/calypso/tunables/allowvaultarraymirroring

Array id 1

/proc/calypso/cm/active_maxpacketsize

Read Specifies the Active Max packet size to be sent out on the streaming network.

Packet Size 7 MPEG Cells for Standard Size packets, 21 MPEG Cells for Jumbo size packets

/proc/calypso/tunables/vaultlocalcopycount

RW Configures the number of local mirror copy count on Mirroring Policy.

Count 1

/proc/calypso/tunables/maintenancearraymirroringallowed

RW Enables array mirroring during the maintenance interval.


TRUE




default Values


Cisco Cl

A

P P E N D I X D
COS Command Line Utilities
COS provides the following command line utilities (CLIs) for use on a separate Linux installation that leverage the APIs exposed by the COS node:

• cos-swift — provides command-line access to the Swift API

• cos-swauth — provides command-line access to the Swauth API

This appendix provides the information needed to install and use these CLI utilities.

Note For details on the Swift and Swauth APIs, see the Cisco Cloud Object Storage Release 3.0.1 API Guide.

Hardware PrerequisitesBefore installing the COS CLI utilities, you must have the following hardware available:

• A COS node, release 2.1.1 or later, staged in either standalone or cluster mode

• A CENTOS 6 Linux workstation on which to install the CLI utilities

Installing the CLI UtilitiesPerform the following steps to install the COS CLI utilities.

Note In the examples shown below, commands that you enter are shown in bold, while system responses are shown in normal type. System responses are typical examples and may vary by installation.

Step 1 Copy the COS 3.0.1 ISO image to the Linux workstation that will host the CLI utilities.

ll cos_repo-3.0.1.iso

-rw-r--r-- 1 root root 250613760 Dec 12 20:39 cos_repo-3.0.1.iso

Step 2 Mount the ISO image.

mount -oloop cos_repo-3.0.1.iso /mnt/cdrom

ls /mnt/cdromcos-3.0.1-0b8-local.repo local_repo_setup Packages repodata

D-1oud Object Storage Release 3.0.1 User Guide

Appendix D COS Command Line Utilities Installing the CLI Utilities

Step 3 Install the utilities on the local workstation.

./local_repo_setup

Loaded plugins: fastestmirror, securityLoading mirror speeds from cached hostfile * base: mirror.cogentco.com * extras: mirrors.advancedhosters.com * updates: centos.mirror.nac.netbase | 3.7 kB 00:00base/group_gz | 216 kB 00:00base/filelists_db | 6.1 MB 00:00base/other_db | 2.8 MB 00:00cos-3.0.1-0b8 | 3.6 kB 00:00 ...cos-3.0.1-0b8/group_gz | 536 B 00:00 ...cos-3.0.1-0b8/filelists_db | 66 kB 00:00 ...cos-3.0.1-0b8/primary_db | 43 kB 00:00 ...cos-3.0.1-0b8/other_db | 20 kB 00:00 ...extras | 3.4 kB 00:00extras/filelists_db | 31 kB 00:00extras/prestodelta | 605 B 00:00extras/other_db | 37 kB 00:00updates | 3.4 kB 00:00updates/filelists_db | 1.5 MB 00:00updates/prestodelta | 194 kB 00:00updates/other_db | 19 MB 00:02Metadata Cache CreatedLoaded plugins: fastestmirror, securityLoading mirror speeds from cached hostfile * base: mirror.cogentco.com * extras: mirrors.advancedhosters.com * updates: centos.mirror.nac.netrepo id repo name statusbase CentOS-6 - Base 6,518cos-3.0.1-0b8 Cisco Cloud Object Store 3.0.1 57extras CentOS-6 - Extras 37updates CentOS-6 - Updates 748repolist: 7,360

Step 4 Run the YUM installation for the COS CLI utilities.

yum install cos-cli

Loaded plugins: fastestmirror, securityLoading mirror speeds from cached hostfile * base: mirror.cogentco.com * extras: mirrors.advancedhosters.com * updates: centos.mirror.nac.netSetting up Install ProcessResolving Dependencies--> Running transaction check---> Package cos-cli.x86_64 0:3.0.1-cos0.1 will be installed--> Finished Dependency Resolution

Dependencies Resolved

==================================================================================================================================================================================== Package Arch Version Repository Size====================================================================================================================================================================================Installing: cos-cli x86_64 3.0.1-cos0.1 cos-3.0.1-0b8 142 k

D-2Cisco Cloud Object Storage Release 3.0.1 User Guide

Appendix D COS Command Line Utilities Using the CLI Utilities

Transaction Summary====================================================================================================================================================================================Install 1 Package(s)

Total download size: 142 kInstalled size: 635 kIs this ok [y/N]: yDownloading Packages:Running rpm_check_debugRunning Transaction TestTransaction Test SucceededRunning Transaction Installing : cos-cli-3.0.1-cos0.1.x86_64 1/1 Verifying : cos-cli-3.0.1-cos0.1.x86_64 1/1

Installed: cos-cli.x86_64 0:3.0.1-cos0.1

Complete!

The CLI Utilities are now ready for use. The following section provides usage information.

Using the CLI Utilities

cos swauthThe cos-swauth command line utility is used to manage authentication accounts and users in a COS cluster. The command line utility has the following form:

cos-swauth [–a <auth-ip>] [–u <admin-user>] [–k <admin-key>] [–h/– –help <subcommand>]

[–v/– –verbose] [subcommand <options>]

You can use the following parameters with the command line utility:

• –a <auth-ip>

– The IP address of the COS authentication service.

– The IP address can also be specified by using the COS_AUTH_IP environment variable.

• –u <admin-user>

– Name of the user authorizing the command.

– When unspecified, the user name defaults to .super_admin.

– The user name, except in the case of the super admin, must be specified in the form <account>:<user>.

– The user name can also be specified by using the COS_ADMIN_USER environment variable.

• –k <admin-key>

– The authorization key (password) of the user.

– The key can also be specified by using the COS_ADMIN_KEY environment variable.

• –h/– –help <subcommand>

– Display help information for the specified subcommand.



• –v/– –verbose

– Display detailed information as part of the output.

• subcommand <options>

– The subcommands include –l/list, –sa/set-account, –su/set-user, –g/get, –i/info, –d/delete.

– Each subcommand is associated with a set of relevant options. Table D-1 provides a listing of the subcommands and their options.

Table D-1 cos-swauth Subcommands and Their Options

Subcommand/Option Description

–l/list

No account specified List the accounts for the specified admin user.

<account> List the users belonging to the specified account.

–sa/set-account

<account> Create or update the specified account.

–s/– –suffix Specify the suffix to be used in creating an account ID.

If a suffix is not provided, a random UUID will be used.

Note A suffix may only be specified for new accounts.

–e/– –endpoint < name>:[<IP>] [–t/– –type <service type>]

Specify a service endpoint for the account.

If the IP address is omitted, the specified endpoint will be deleted.

For the reserved endpoint name default, the name of the default endpoint for the service type must be specified instead of the IP address.

If unspecified, the service type will default to storage.

–su/set-user

<user> Create or update the details of the specified user.

–K/– –key <user key> Specify the user’s key (password).

This option can be used by admin or reseller-admin users to assign a password to a new user. It can also be used by existing users to change their password.

–A/– –admin Assign the user admin privileges for the specified account.

Account admins can create, delete, or modify users within an account, and have admin privilege in the services associated with the account.

The user privileges default to –N/– –normal if –A/– –admin and –R/– –reseller_admin options are omitted.



cos-swiftThe cos-swift command line utility can be used to manage storage accounts, storage containers, and storage objects in a COS cluster. The command line utility has the following form:

cos-swift [–t <auth-token>] [–a <storage-url>] [–v/– –verbose][–h/– –help <subcommand>] [subcommand <options>]

You can use the following parameters with the command line utility:

• –t <auth-token>

– The authentication token returned by the cos-swauth get operation.

–R/– –reseller_admin Assign the user reseller-admin privileges for the COS cluster.

Reseller-admins can create, delete, or modify accounts, and have all the privileges of an account admin in the accounts defined in the COS authentication service.

The user privileges default to –N/– –normal if –A/– –admin and –R/– –reseller_admin options are omitted.

–N/– –normal Specify that the user has no special privileges for the specified account.

This option can also be used to withdraw the existing privileges of a user.

The user privileges default to –N/– –normal if a privilege option is not specified.

–g/get

<account>:<user> Get an authentication token and storage service URL for the specified user.

–K/– –key <user key> Specify the user’s key (password).

This option may be omitted if the user does not have a key.

–e/– –endpoint Retrieve the service endpoint catalog associated with the user account.

–i/info

<account> Retrieve account information, including the service endpoint catalog.

<account>:<user> Retrieve detailed user information.

–d/delete

<account> Delete the specified account.

Note Only an empty account can be deleted.

<account>:<user> Delete the specified user.

Table D-1 cos-swauth Subcommands and Their Options




– The authentication token can also be specified by using the COS_AUTH_TOKEN environment variable.

• –a <storage-url>

– The storage URL returned by the cos-swauth get operation.

– The storage URL can also be specified by using the COS_STORAGE_URL environment variable .

• –v/– –verbose

– Display detailed information as part of the output.

• –h/– –help <subcommand>

– Display help information for the specified subcommand.

• sub-command <options>

– The subcommands include –i/info, –l/list, –s/set, –g/get, –u/update, –d/delete.

– Each subcommand is associated with a set of relevant options. Table D-2 provides a listing of the subcommands and their options.

Table D-2 cos-swift Subcommands and Their Options


–i/info

No container or object specified. Display metadata for the storage account in the storage URL.

<container name> Display metadata for the specified storage container.

<container name>/<object name> Display metadata for the specified storage object.

–l/list

No container specified. List the containers of the storage account in the storage URL.

<container name> List the objects of the specified storage container.

<container name>/<object path> Form a prefix by appending a slash to the specified object path if the path does not end with a slash.

Then, list the objects of the specified storage container whose names have this prefix.

–L/– –limit <value> Restrict the number of entries listed to less than or equal to the specified value.

–m/– –marker <value> List the storage containers or objects with names greater than the specified marker value.

–e/– –end_marker <value> List the storage containers or objects with names less than the specified marker value.

–p/– –prefix <value> List the storage containers or objects whose names begin with the specified prefix.

–d/– –delimiter <character> List the storage containers or objects whose names do not have the specified delimiting character.

Note If the –p/– –prefix <value> option is used, the delimiting character may be present in the specified prefix.



–V/– –Verbose If no container name is specified, display the metadata of the containers belonging to the storage account in the storage URL. For each container, the container name, the number of subordinate objects, and the total content size of all subordinate objects are displayed.

If a container name is specified, display the metadata of the objects in that storage container. For each object, the object name, MD5 checksum, content type, content size, and modification timestamp are displayed.

–s/set

<container name> Create a container with the specified name.

<container name>/<object name> Create an object with the specified name within the mentioned container.

–m/– –meta <name>:<value> Create a custom metadata item with the specified name and value. This option may be repeated multiple times.

–r/– –read <ACL> Specify a read access control list (ACL) for a container. Specifying an empty string ("") as the ACL will cause the existing ACL to be deleted.

–w/– –write <ACL> Specify a write access control list (ACL) for a container.

–f/– –file <name> Upload the named file as object content. If an existing object is specified, this file will replace prevailing object content.

–T/– –type <content type> Specify the content type of an object. The specified content type must be a valid mime type.

–u/update

No container or object specified. Update the metadata of the storage account in the storage URL.

<container name> Update the metadata of the specified container.

<container name>/<object name> Update the metadata of the specified object in the container.

–m/– –meta <name>:<value> Create a custom metadata item with the specified name and value. This option may be repeated multiple times.

–r/– –read <ACL> Specify a read access control list (ACL) for a container. Specifying an empty string ("") as the ACL will cause the existing ACL to be deleted.

–w/– –write <ACL> Specify a write access control list (ACL) for a container.

–g/get

<container name>/<object name> Retrieve the content of the specified object.

–f/– –file <name> Store the retrieved object content in the specified file.

Note If this option is not used and file is not specified, the object content will be directed to stdout.

–d/delete





<container name> Delete the specified container.

Note Only an empty container can be deleted.

<container name>/<object name> Delete the specified object.




cisco cloud object storage release 3.0.1 user guide€¦ · contents vi cisco cloud object storage...

Documents