J2UL-2073-01ENZ0(00)November 2015

FUJITSU SoftwareServerViewCloud Monitoring Manager V1.0

 Overview

Trademarks LINUX is a registered trademark of Linus Torvalds.The OpenStack® Word Mark and OpenStack Logo areeither registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation in theUnited States and other countries and are used withthe OpenStack Foundation's permission. We are notaffiliated with, endorsed or sponsored by the OpenStackFoundation, or the OpenStack community.Red Hat is a trademark or a registered trademark of RedHat Inc. in the United States and other countries.Java® is a registered trademark of Oracle and/or itsaffiliates.Python and PyCon are trademarks or registeredtrademarks of the Python Software Foundation.ServerView and Systemwalker are registered trademarksof FUJITSU LIMITED.Other company names and product names aretrademarks or registered trademarks of their respectiveowners.

Copyright FUJITSULIMITED 2015

All rights reserved, including those of translation into otherlanguages. No part of this manual may be reproducedin any form whatsoever without the written permission ofFUJITSU LIMITED.

High Risk Activity

The Customer acknowledges and agrees that the Product is designed, developedand manufactured as contemplated for general use, including without limitation,general office use, personal use, household use, and ordinary industrial use, but is notdesigned, developed and manufactured as contemplated for use accompanying fatalrisks or dangers that, unless extremely high safety is secured, could lead directly todeath, personal injury, severe physical damage or other loss (hereinafter "High SafetyRequired Use"), including without limitation, nuclear reaction control in nuclear facility,aircraft flight control, air traffic control, mass transport control, medical life supportsystem, missile launch control in weapon system. The Customer shall not use theProduct without securing the sufficient safety required for the High Safety Required Use.In addition, FUJITSU (or other affiliate's name) shall not be liable against the Customerand/or any third party for any claims or damages arising in connection with the HighSafety Required Use of the Product.

Export Restrictions

Exportation/release of this document may require necessary procedures in accordancewith the regulations of your resident country and/or US export control laws.

Contents

Overview 3

ContentsAbout this Manual........................................................................... 4

1 What is CMM?.................................................................................. 6

1.1 Key Features.......................................................................................................... 6

1.2 Components........................................................................................................... 7

1.3 Users and Roles.................................................................................................... 8

2 Monitoring.......................................................................................10

3 Log Management........................................................................... 12

Glossary ......................................................................................................... 14

About this Manual

Overview 4

About this ManualThis manual is an introduction to FUJITSU ServerView Cloud Monitoring Manager - hereafterreferred to as Cloud Monitoring Manager (CMM).The manual is structured as follows:

Chapter Description

What is CMM? on page 6 Introduces CMM, its key features, components, andusers.

Monitoring on page 10 Describes the basic concepts and features ofmonitoring.

Log Management on page 12 Describes the basic concepts and features of logmanagement.

Glossary on page 14 Defines the central terms relevant for CMM.

Readers of this ManualThis manual is written for everybody interested in CMM. It offers a basic introduction for readerswho do not know CMM and for those who have already started using it. The manual does notrequire any special knowledge.

Notational ConventionsThis manual uses the following notational conventions:

Add The names of graphical user interface elements like menu options are shownin boldface.

init System names, for example command names and text that is entered fromthe keyboard, are shown in Courier font.

<variable> Variables for which values must be entered are enclosed in angle brackets.

[option] Optional items, for example optional command parameters, are enclosed insquare brackets.

one | two Alternative entries are separated by a vertical bar.

{one | two} Mandatory entries with alternatives are enclosed in curly brackets.

AbbreviationsThis manual uses the following abbreviations:

CMM Cloud Monitoring Manager

IaaS Infrastructure as a Service

ICMP Internet Control Message Protocol

OS Operating System

About this Manual

Overview 5

OSS Open Source Software

PaaS Platform as a Service

SaaS Software as a Service

Available DocumentationThe following documentation on CMM is available:• Overview: A manual introducing CMM. It is written for everybody interested in CMM.• CMM Operator's Guide: A manual for operators describing how to install, operate, and maintain

CMM.• OpenStack Operator's Guide: A manual for OpenStack operators describing how CMM

supports them in monitoring OpenStack services as well as managing the available log data.• Tenant User's Guide: A manual for tenant users describing how CMM supports them in

monitoring their services and virtual machines.

1: What is CMM?

Overview 6

1 What is CMM?As more and more applications are deployed on cloud systems and cloud systems are growing incomplexity, managing the cloud infrastructure is becoming increasingly difficult. Cloud MonitoringManager (CMM) helps mastering this challenge by providing a sophisticated Monitoring as aService solution that is operated on top of OpenStack-based cloud computing platforms.CMM allows enterprises to manage, track, and optimize their cloud infrastructure and the servicesthey provide to end users. It offers a suite of monitoring and analytics tools aimed at improving thehealth and performance of cloud systems.CMM assembles and presents metrics and log data in one convenient access point. It offers anintegrated view of cloud resources, based on its seamless integration with OpenStack. Whilebeing flexible and scalable to instantly reflect changes in the cloud infrastructure of an enterprise,CMM provides the required ways and means to ensure multi-tenancy, high availability, and datasecurity.

1.1 Key FeaturesCMM is an out-of-the-box solution for monitoring OpenStack-based cloud environments. It isprovided as a cloud service to users.The core of CMM is Monasca, an open source Monitoring as a Service solution that integrateswith OpenStack. The key features of CMM form an integral part of the Monasca project. CMMextends the source code base of the project through active contributions.Compared to the Monasca community edition, CMM provides the following added value:• Packaging as a commercial enterprise solution• Enterprise-level support

The key features of CMM address public as well as private cloud service providers. They include:• Monitoring• Log management• Integration with OpenStack

MonitoringCMM is a highly scalable and fault tolerant monitoring solution for OpenStack-based cloudinfrastructures.The system operator of the cloud infrastructure and the service providers do not have to carefor system monitoring software any longer. They use CMM to check whether their services andservers are working appropriately.CMM provides comprehensive metrics with reasonable defaults for monitoring the status, capacity,throughput, and latency of cloud systems. The CMM users can set their own warnings and criticalthresholds. Combined with a notification system, these alerting features enable them to quicklyanalyze and resolve problems in the cloud infrastructure.For details, refer to Monitoring on page 10.

Log ManagementWith the increasing complexity of cloud infrastructures, it is becoming more and more difficult andtime-consuming for the system operator to gather, store, and query the large amounts of log datamanually. To cope with these problems, CMM provides centralized log management features.

1: What is CMM?

Overview 7

Based on configurable agents that collect log data from all services and servers in the cloudinfrastructure, the log data can be accessed from a single dashboard. Integrated search, filter, andgraphics options enable operators to isolate problems and narrow down potential root causes.CMM thus provides valuable insights into the log data, even with large amounts of data resultingfrom highly complex environments.For details, refer to Log Management on page 12.

Integration with OpenStackCMM is integrated with OpenStack core services. These include:• OpenStack Horizon dashboard for visualizing monitoring metrics and log data• OpenStack user management• OpenStack security and access control

1.2 ComponentsThe following illustration provides an overview of the main components of CMM:

OpenStackOpenStack is the underlying platform technology of CMM. It allows for the deployment andmanagement of Infrastructure as a Service (IaaS) platforms.For details on OpenStack, refer to the OpenStack documentation.

CMM ServiceThe CMM Service is the central CMM component. It is responsible for receiving, persisting, andprocessing monitoring data, as well as providing the data to the users.The CMM Service relies on Monasca. It uses Monasca for high-speed metrics querying andintegrates the streaming alarm engine and the notification engine of Monasca. For details, refer tothe Monasca Wiki.

CMM Horizon PluginCMM comes with a plugin for the OpenStack Horizon dashboard. The CMM Horizon pluginextends the main dashboard in OpenStack with a new view for monitoring. This enables CMM

1: What is CMM?

Overview 8

users to access the monitoring and log management functions from a central Web-basedgraphical user interface. Monitoring metrics and log data are visualized on a comfortable andeasy-to-use dashboard.For details, refer to the OpenStack Horizon documentation.

CMM Metrics AgentA CMM Metrics Agent is a Python-based agent for gathering metrics and sending them to theCMM Service. The agent supports metrics from a variety of sources as well as a number of built-insystem and service checks. A CMM Metrics Agent can be installed on each virtual or physicalserver to be monitored.The agent functionality is fully integrated into the source code base of the Monasca project. Fordetails, refer to the Monasca Wiki.

CMM Log AgentA CMM Log Agent is needed for collecting log data and forwarding it to the CMM Service forfurther processing. It can be installed on each virtual or physical server whose log data is to beretrieved.The agent functionality is fully integrated into the source code base of the Monasca project. Fordetails, refer to the Monasca Wiki.

1.3 Users and RolesCMM users can be grouped by their role. The following user roles are distinguished:• A tenant user acts as a service provider in the OpenStack environment. He books virtual

machines in OpenStack to provide services to end users or to host services that he needs forhis own development activities. CMM helps tenant users to ensure the quality of their servicesin the cloud.For details on the tasks of the tenant user, refer to the Tenant User's Guide.

• The OpenStack operator is a special tenant user. He is responsible for administrating andmaintaining the underlying OpenStack platform and ensures the availability and quality of theOpenStack services (e.g. Heat, Nova, Cinder, Swift, Glance, or Keystone).For details on the tasks of the OpenStack operator, refer to the OpenStack Operator's Guide.

• The CMM operator is responsible for administrating and maintaining CMM. He provides thecloud monitoring services to other users and ensures the quality of the CMM Service.For details on the tasks of the CMM operator, refer to the CMM Operator's Guide.

User ManagementCMM is fully integrated with Keystone, the identity service which serves as the commonauthentication and authorization system in OpenStack.The CMM integration with Keystone requires any CMM user to be registered as an OpenStackuser. All authentication and authorization in CMM is done through Keystone. If a user requestsmonitoring data, for example, CMM verifies that the user is a valid user in OpenStack and allowedto access the requested metrics.CMM users are created and administrated in OpenStack:• Each user assumes a role in OpenStack to perform a specific set of operations. The

OpenStack role specifies a set of rights and privileges.

1: What is CMM?

Overview 9

• Each user is assigned to at least one project in OpenStack. A project defines a set of resourceswhich can be accessed by the assigned users.

For details on user management, refer to the OpenStack documentation.

2: Monitoring

Overview 10

2 MonitoringThe monitoring solution of CMM addresses the requirements of large-scale public and privateclouds where high numbers of physical and virtual servers need to be monitored and hugeamounts of monitoring data need to be managed. CMM consolidates metrics, alarms, andnotifications, as well as health and status information from multiple systems, thus reducing thecomplexity and allowing for a higher level analysis of the monitoring data.CMM covers all aspects of a Monitoring as a Service solution:• Storage of monitoring data in a resilient way.• Multi-tenancy architecture for submitting and streaming metrics. The architecture ensures the

secure isolation of tenant data.• Horizontal and vertical scalability to support constantly evolving cloud infrastructures. When

physical and virtual servers are scaled up or down to varying loads, the monitoring solution canbe adapted accordingly.

• Graphical user interface for all CMM user groups that is seamlessly integrated into the cloudinfrastructure. Based on OpenStack Horizon, the user interface visualizes the health and statusof the cloud resources and enables user access to all monitoring functionality.

Basic Usage ScenarioThe basic usage scenario of setting up and using the monitoring services of CMM looks asfollows:

2: Monitoring

Overview 11

The CMM operator is responsible for providing the monitoring services to the tenant users andthe OpenStack operator. This enables the tenant users and the OpenStack operator to focus onoperation and ensure the quality of their services without having to carry out the tedious tasksimplied by setting up and administrating their own system monitoring software. The CMM operatoruses the monitoring services himself for monitoring the operation of CMM.Tenant users monitor the services they provide to end users or the services they need for theirdevelopment activities. They ensure that the physical and virtual servers on which their servicesare provided are up and running as required.The OpenStack operator monitors physical and virtual servers, hypervisors, and services of theunderlying platform. In addition, he is responsible for the middleware components, for example,RabbitMQ or database services.

MetricsA CMM Metrics Agent can be installed and configured on each physical and virtual server wherecloud resources are to be monitored. The agent is responsible for querying metrics and sendingthe data to the CMM Service for further processing.Metrics are self-describing data structures that are uniquely identified by a name and a set ofdimensions. Each dimension consists of a key/value pair that allows for a flexible and concisedescription of the data to be monitored, for example, region, availability zone, service tier, orresource ID.The CMM Metrics Agent supports various types of metrics including the following:• System metrics, for example, CPU usage, consumed disk space, or network traffic.• Host alive checks. The agent can perform active checks on a host to determine whether it is

alive using ping (ICMP) or SSH.• Process checks. The agent can check and monitor a process, for example, the number of

instances, memory size, or number of threads.• HTTP endpoint checks. The agent can perform up/down checks on HTTP endpoints by

sending an HTTP request and reporting success or failure to the CMM Service.• Service checks. The agent can check middleware services, for example, MySQL, Kafka, or

RabbitMQ.• OpenStack services. The agent can perform specific checks on each process that is part of an

OpenStack service.

Alarms and NotificationsCMM supports GUI-based alarm and notification management. Template-based alarm definitionsallow for monitoring a dynamically changing set of resources without the need for reconfiguration.While the number of underlying virtual machines is changing, for example, this ensures theefficient monitoring of elastic cloud services. Notifications can be configured in order to informCMM users when an alarm is triggered.As a CMM user, you can:• Monitor the status of services and servers.• Create, update, and delete alarms.• Create, update, and delete notifications.• View, share, and explore visualizations of the monitoring data.

3: Log Management

Overview 12

3 Log ManagementServices and servers in a cloud infrastructure generate huge amounts of log data, from simplyannouncing that everything is healthy to detailed information on events or processes. The logs aredistributed over many physical and virtual servers. CMM collects, stores, and queries these logsand makes them accessible to CMM users in one central place.CMM covers all aspects of an enterprise-class log management solution:• Storage of log data in a resilient way.• Horizontal and vertical scalability to support constantly evolving cloud infrastructures. When

cloud resources are scaled up or down to varying loads, the log management solution can beadapted accordingly.

• Comfortable dashboard for visualizing and analyzing the log data. Based on OpenStackHorizon, the dashboard is seamlessly integrated into the cloud infrastructure. It enables useraccess to all log management functionality.

Basic Usage ScenarioThe basic usage scenario of setting up and using the log management services of CMM looks asfollows:

The CMM operator is responsible for providing the log management services to OpenStackoperators. He uses the log management services himself for retrieving log data on CMM.

3: Log Management

Overview 13

The OpenStack operator uses the log management services for retrieving the log data of theOpenStack services and the middleware components that make up the underlying OpenStackplatform.

Collecting Log DataA CMM Log Agent can be installed and configured on each physical and virtual server wherelog data is to be retrieved. The agent is responsible for collecting the log data from the cloudresources and sending them to the CMM Service for further processing.CMM stores the log data in a central database. The central database forms the basis forvisualizing the log data for the CMM users.

Analyzing Log DataCMM supports advanced data analysis and visualization of the log data in a variety of charts,tables, or maps. Visualizations can easily be combined in dynamic dashboards that displaychanges to search queries in real time.As a CMM user, you can:• Submit search queries and filter the search results.• Save search queries and reload them.• Construct visualizations of search results.• Save and share visualizations, and combine them into dashboards.• Save dashboards and reload them.

Glossary

Overview 14

GlossaryApplicationA software, including procedures and documentation, which performs productive tasks for users.

CloudA metaphor for the Internet and an abstraction of the underlying infrastructure it conceals.

Cloud ServiceThe availability of services to users on demand via the Internet.Cloud services are designed to provide easy, scalable access to applications, resources, andservices. They can scale dynamically to meet their user's needs. With the service providersupplying the hardware and software necessary for the service, there is no need for users toprovision or deploy their own resources or allocate IT staff to manage the service.

CMM OperatorA person responsible for maintaining and administrating CMM.

ElasticsearchAn open source application that provides a highly scalable full-text search and analytics engine.CMM uses Elasticsearch as the underlying technology for storing, searching, and analyzing bigvolumes of log data.

GrafanaAn open source application for visualizing large-scale measurement data. CMM integrates withGrafana for visualizing the CMM monitoring data.

Infrastructure as a Service (IaaS)The delivery of computer infrastructure (typically a platform virtualization environment) as aservice.

KibanaAn open source analytics and visualization platform designed to work with Elasticsearch. CMMintegrates with Kibana for visualizing the CMM log data.

LogstashAn open source application for receiving, processing, and outputting logs. Logstash provides apipeline for storing, querying, and analyzing logs. It uses Elasticsearch as a back-end data storeand Kibana as a front-end reporting platform.

MetricsSelf-describing data structures that allow for a flexible and concise description of the data to bemonitored. Metrics values represent the actual monitoring data that is collected and presented inCMM.

Glossary

Overview 15

MonascaAn open source Monitoring as a Service solution that integrates with OpenStack. It forms the coreof CMM.

OpenStack OperatorA person responsible for maintaining and administrating OpenStack, the underlying platformtechnology of CMM.

Platform as a Service (PaaS)The delivery of a computing platform and solution stack as a service.

ServiceGenerally, a discretely defined set of contiguous or autonomous business or technical functionality,for example an infrastructure or Web service.

Software as a Service (SaaS)A model of software deployment where a provider licenses an application to customers for use asa service on demand.

Tenant UserA person with limited access to cloud resources in OpenStack.