Operations Management and Monitoring of a Lync Server 2010 Environment

Operations Management and Monitoring of a Lync Server 2010 EnvironmentLync Server 2010

Published: March 2014Author: Indranil Dutta, Greg Stemp, James Birkinshaw

Abstract: This document describes the operational processes, tasks, and tools required for you to maintain a Lync Server 2010 environment. It explains how to manage Lync Server 2010 according to the Microsoft Operations Framework (MOF) model and it will help you design an efficient operational management environment, which includes implementing processes and procedures to maintain an efficient working environment.

MICROSOFT CONFIDENTIAL1

This document is provided as-is. Information and views expressed in this document, including URL and other Internet website references, may change without notice.Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.Copyright 2014 Microsoft Corporation. All rights reserved.

Table of Contents1Introduction and Overview12Best Practices for Lync Server Environments12.1Capacity and Availability Management22.1.1Capacity Management22.1.2Availability Management32.2Change Management42.2.1Managing the Timing of Changes62.3Configuration Management62.3.1Implementing Configuration Management72.3.2Tools Used for Configuration Management72.3.3Relationship with Change Management72.4System Administration82.4.1System Troubleshooting82.4.2System Troubleshooting Process82.4.3Issue Management Tools92.4.4Centralized vs. Decentralized Administration102.5Service Level Agreements102.5.1External Customers102.5.2Internal Customers112.5.3Typical Criteria112.6Documentation112.6.1Document Management Systems112.6.2Databases113Using Administrative Tools123.1Using Topology Builder123.2Using Lync Server Control Panel123.3Lync Server 2010 Logging Tool133.4Lync Server Management Shell143.5Lync Server Best Practices Analyzer144Operations Management154.1Standard Procedures164.2Emergency Procedures164.3Daily Tasks164.3.1Performing Physical Environmental Checks174.3.2Performing and Monitoring Backups184.3.3Checking Disk Usage194.3.4Checking Event Logs204.3.5Monitoring Lync Server 2010 Performance214.3.6Monitoring Operating System244.3.7Central Management Store Replication Status264.3.8Monitoring Network Performance264.3.9Scanning for Viruses and Checking Virus Definitions284.3.10Viewing and Analyzing Monitoring Server Reports294.3.11Validating Voice Number Normalization and Routing324.3.12Validating Address Book Access334.3.13Validating Address Book Web Query364.3.14Viewing Status of Pools384.3.15Monitoring Back End Lync Server 2010 Storage Performance384.3.16Monitoring Group Chat394.3.17Validate Domain Name System (DNS) Settings for DNS Load Balancing404.3.18Running Synthetic Transactions414.4Weekly Tasks964.4.1Archive Event Logs964.4.2Create Reports964.4.3Incident Reports964.4.4Check IIS Logs and Performance974.4.5Generate Database Reports974.4.6Check for Security and Lync Server Updates974.4.7Run the Lync Server 2010 Best Practice Analyzer984.4.8Review SLA Performance Figures984.4.9Review System Center Operations Manager Management Pack and Quality of Experience Reports984.4.10Generating and Viewing Database Reports for Enterprise Pools984.4.11Running Bandwidth Utilization Analyzer994.5Monthly Tasks1004.5.1Viewing Status of Global Settings for a Forest1004.5.2Viewing Edge Server Settings1154.5.3Check Lync Server 2010 Server Certificates1174.5.4Check Trunk Configuration against a Phone Number1174.5.5Check voice normalization rule1174.5.6Test Telephone Number against a Voice Policy1184.5.7Test telephone Number Against a Voice Route1184.5.8Test Voice Configuration1184.5.9Test Voice Rules, Routes, and Policies1184.5.10Security Checks1194.5.11Capacity Planning1194.5.12Disaster Recovery Test1194.6As-Needed Tasks1204.6.1Backup (and Restore) Policies or Configuration Settings1225Operations Checklists1225.1Daily Operations Checklist1235.2Weekly Operations Checklist1305.3Monthly Operations Checklist1326Monitoring Lync Server 2010 with System Center Operations Manager1356.1Introduction to the Microsoft Lync Server Management Pack1356.1.1Getting the Latest Management Pack and Documentation1357Operational Dependencies1358Troubleshooting1389Key Health Indicators (KHIs)138

Introduction and OverviewThis document describes the operational processes, tasks, and tools required for you to maintain a Microsoft Lync Server 2010 communications software environment. It explains how to manage Lync Server 2010 according to the Microsoft Operations Framework (MOF) model and it will help you design an efficient operational management environment, which includes implementing processes and procedures to maintain an efficient working environment. There is also a Lync Operations guide in the TechNet Library, but this guide differs in providing specific tasks at daily, weekly, and monthly intervals, and checklists to track ongoing operational issues. Both guides can and should be used in the ongoing operation of your Lync implementation, and where appropriate, this guide refers to the TechNet Library document.Understanding MOFMOF is a collection of best practices, principles, and models that provide organizations technical guidance about the management of IT assets, such as daily Lync Server 2010 operations. Following MOF guidelines can help you achieve mission-critical production system reliability, availability, supportability, and manageability for Microsoft products. For more information, see Microsoft Operations Framework 4.0. Learning about best practices for Lync Server 2010We recommend that you implement practical and proven procedures to manage Lync Server 2010. Using tried, tested, and documented methods of managing operations may be more efficient than developing your own methods.Separating operations into daily, weekly, and monthly processesDocument the required operational tasks that you will regularly perform. Documenting how you perform tasks helps to ensure that your information is preserved when there is a change in your operational environment such as when new technologies are deployed or staff changes occur. We recommend that operational tasks be separated into manageable workloads where tasks are performed on a daily, weekly, and monthly basis. Daily tasks would focus efforts on the functioning of a system, and monthly tasks would focus more on ensuring the long-term health of a system. This document is applicable to both environments deploying only instant messaging/presence (IM/P) components or IM/P with Enterprise Voice. When tasks or checklist items are specific to Enterprise Voice, this is mentioned and if your environment does not include Enterprise Voice the portion may be skipped.Deploying the tools required for operating Lync Server 2010Many tools are available to help troubleshoot issues, automate tasks, and help monitor and maintain the Lync Server 2010 environment. Define a standard set of tools for your organization so the tasks performed by the operations team are performed accurately, efficiently, consistently, and in a controlled manner. You should also implement processes to track incidents and major configuration changes.Best Practices for Lync Server EnvironmentsFor the benefit of readers not already familiar with the basics of server management in general, we provide an overview of server management practices. Readers already familiar with server management may choose to skip this section.Best practices are recommendations that are based on the knowledge and experience that IT professionals have gained across many environments. They provide standard procedures for typical tasks that your Lync Server administrators must accomplish daily, and list the tools that they should use to manage a Lync Server environment. Typical tasks for Lync administrators include the following:Capacity and Availability ManagementDefine how and what to measure to predict future capacity requirements and to report about the capacity, reliability, and availability of your systems. You must make sure that servers that are running Lync Server are sized to handle the load on the system, and that unplanned downtime is kept under the levels defined in the service level agreement (SLA). Additionally, you will have to upgrade hardware to continue to meet the defined requirements. Change Management and Configuration ManagementControl how changes are made to your IT systems. This should include testing, application feedback and contingency plans, documentation of all changes, and approval from management if issues occur. Keep a record of your software and hardware assets and their configurations.System AdministrationOutline standard methods for doing administrative tasks such as database administration and site administration. Security AdministrationHave a detailed policy and plan that protects data confidentiality, data integrity, and data availability of your IT infrastructure. This includes day-to-day activities and tasks that are related to maintaining and adjusting the IT security infrastructure. System TroubleshootingOutline methods for dealing with unexpected issues, including steps to prevent similar issues in the future.Service Level AgreementsMaintain a set of goals for the performance of your IT systems and regularly measure performance against these goals.DocumentationDocument standard procedures, such as configuration information and lessons learned, and make them available to the staff members that need them. As changes to the configuration are made, update the documentation accordingly.Capacity and Availability ManagementThe purpose of capacity management and availability management is to measure and control system performance. We recommend that you implement capacity management and availability management procedures so that you can measure and control system performance. You need to know if the system is available and if it can handle the current and the projected demands by setting baselines and monitoring the system to look for trends.Capacity ManagementCapacity management involves planning, sizing, and controlling service capacity to help ensure that the minimum performance levels specified in your SLA are exceeded. Good capacity management helps to ensure that you can provide IT services at a reasonable cost and still meet the levels of performance defined in your SLAs with the client. These criteria can include the following:System Response TimeThis is the measured time that the system takes to do typical actions. Examples include the time it takes for the audio/video server role to process audio/video traffic, the time that it takes for a client to create and join a conference, or the time taken for presence to be updated in all watcher clients.Storage CapacityThis is the capacity of a storage system, whether it is a content database, a backup device, or a local drive. Examples include the maximum amount of storage space to be provided per site and the amount of time that backups should be stored before being overwritten.Adjusting capacity is frequently a case of ensuring that enough physical resources are available, such as disk space and network bandwidth. The following table lists typical resolutions for capacity-related issues.Typical resolutions for capacity-related issuesIssuePossible resolution

Remote users having poor audio/video performanceCheck to see if appropriate bandwidth is available on the WAN links and if QoS is enabled and properly configured. Check QoE data.

Overall response of the Lync environment is slow.Run tests to check that the existing front-end servers are capable of dealing with the load. Introduce a new front-end server if required.Check SQL database response times and fix the causes for the delays (for example, improve disk I/O).

Troubleshooting in greater detail is covered in the Lync Server Networking Guide. Most of the Call Quality Methodology discussion in that document is transferable to Lync 2010.Capacity is affected by system configuration and depends on physical resources such as network bandwidth. For example, if a Lync environment is configured to perform a full backup nightly , care must be taken to help ensure that the impact on the interactive performance experienced by end users is minimized.Capacity management is the process of keeping the capacity of a system within acceptable levels and addresses the following issues:Reacting to changes in requirementsCapacity requirements have to be adjusted to account for changes in the system or the organization. For example, if your environment decides to start using Enterprise Voice, the number and placement of Mediation Servers and public switched telephone network (PSTN) gateways will be crucial. Should you be doing Session Initiation Protocol (SIP) trunking or direct SIP, the overall design will substantially be modified to provide the best Enterprise Voice performance. Predicting future requirementsSome capacity requirements change predictably over time. By tracking trends you can plan upgrades in advance. For example, available bandwidth between various Lync sites will need to be monitored to create a baseline. This baseline will allow you to predict when you will need to add additional bandwidth to these links as user count in these remote sites increases with time. Availability ManagementAvailability management is the process of ensuring that any IT service consistently and cost effectively delivers the level of consistent, reliable service that is required by the customer. Availability management is concerned with minimizing loss of service and with ensuring that appropriate action is taken if service is lost. In a Lync environment, you may be concerned about whether the Enterprise Voice service is available, whether users can join scheduled conferences, and so on. An SLA defines an acceptable frequency and length of outages and allows for certain periods when the system is unavailable for planned maintenance.If you have to provide reports to your management about the availability of systems, or if you have financial or other penalties associated with missing availability targets, you must record availability data. Even if you do not have such formal requirements, it is a good idea to at least know how frequently a system has failed in a certain time period; for example, system availability in the last 12 months and how long it took to recover from each failure. This information will help you measure and improve your teams effectiveness in responding to a system failure. It can also provide you with useful information if there is a dispute.Measures related to availability are as follows:AvailabilityThis is typically expressed as the time that a system or service is accessible compared to the time that it is down. It is typically expressed as a percentage. (You may see references to three nines or five nines. These refer to 99.9 percent or 99.999 percent availability.)ReliabilityThis is a measure of the time between failures of a system and is sometimes expressed as mean (or average) time between failures (MTBF).Time to RepairThis is the time taken to recover a service after a failure has occurred and is often expressed as mean (meaning average) time to repair (MTTR).Availability, reliability, and time to repair are related as follows:Availability = (MTBF MTTR) / MTBFFor example, if a server fails twice over a six-month period and is unavailable for an average of 20 minutes, the MTBF is three months or 90 days and the MTTR is 20 minutes. Therefore, Availability = (90 days 20 minutes) / 90 days = 99.985 percentAvailability management is the process of ensuring that availability is maximized and kept within the parameters defined in SLAs. Availability management includes the following processes:Monitoring Examining when and for how long services are unavailable.ReportingAvailability figures should be regularly provided to management, users, and operations teams. These reports should highlight trends and identify areas that are doing well and areas that require attention. The report should summarize compliance with targets set in the SLAs.ImprovementIf availability does not meet targets that are defined in the SLAs or where the trend is toward reduced availability, the availability management process should plan remedial steps. This should include working with other responsible teams to highlight reasons for outages and to plan remedial actions to prevent a recurrence of the outages.Capacity and availability measurements are repetitive tasks that are ideally suited to automated tools and scripts such as Microsoft System Center Operations Manager, which is discussed later in this document.Change ManagementChanges to your IT environment are inevitable. Changes include new technologies, systems, applications, hardware, tools, processes, and changes in roles and responsibilities. An effective change management system lets you introduce changes to your IT environment quickly and with minimal service disruption. A change management system brings together the teams involved in modifying a system. For example, deciding to take advantage of the Office Web Apps. This is an integrated Lync Service application that enables users to read and edit documents within a browser. The implementation of this service, after you have gone into production, requires the involvement of several teams:Test TeamThis team load-tests the Office Web Apps on a test server, in the process providing information about the expected usage patterns and expected performance of the production servers. Lync AdministratorsThis team determines the deployment strategy and scripts the installation where possible. The team is responsible for ensuring that the change is deployed on the production environment, and it is responsible for administration afterwards. The team must understand the effect of the changes and incorporate them in procedures before the changes are put into production Network TeamThis team is responsible for changes to firewall rules that allow access from the Internet to the internal Lync pool servers. The team is also responsible in working with the Lync administrators for ensuring that the amount of available bandwidth can support the additional load.Security TeamThis team assesses security and minimizes risks. The security team must review known vulnerabilities and help to ensure that security risks are minimized.User Acceptance TeamThis team is composed of users who are willing to test the system and offer feedback for improvements.The change management process defines the responsibilities of each team and schedules the work to be performed, incorporating checks and tests where they are required. Change controls will vary depending on the complexity and expected effect of a change. They can vary from automatic approval of minor changes, to change review meetings, to full project-level reviews. To illustrate this better, the groups of changes are discussed in this section.Major ChangesMajor changes have a global effect on the system and may require input from various teams. An example of this is upgrading to Lync Server 2010. Major changes affect many different teams and perhaps different systems. The change management process will probably include one or more change review meetings to inform the teams that will be involved in the change or be affected by the change.Significant ChangesSignificant changes require significant resources to plan, build, and implement. Appropriate change controls should be introduced to help ensure that the effect of the change is understood, deployment procedures are tested, and the rollback and contingency plans are ready. An example of a significant change is deploying a new cumulative update.Minor ChangesMinor changes do not significantly affect the IT environment, for example, modifying certain Lync policies via the Microsoft Lync Server 2010 Control Panel.Standard ChangesStandard changes are performed regularly and are well understood and documented. The change management process should review all changes to the procedure, but it should not, for example, be involved in creating every content database.The following example of change management examines how different teams interact and the actions that are performed when a new service pack is deployed. These actions are organized and managed by the change management process.Raise a change requestThe security team has assessed the latest service pack and confirmed that it resolves a possible vulnerability in the production system. The team raises a change request to have the new cumulative update applied to all servers running Lync Server.Service pack release notes reviewThe Lync administrator team reviews the service pack release notes to identify the effect on the system.A series of lab tests is doneThe Lync administrator team must perform test updates on a server in a non-production environment to decide whether the service pack can be applied successfully without affecting any of the installed applications and server systems. If there are third-party or internally created applications that interface with Lync Server in a production environment, these should be also tested. These tests can also be used to estimate the time required to perform the upgrades. Users are informed of the outageThe Lync administrator team, communications team, or user help desk informs all affected users about the planned maintenance cycle and how long the service will be unavailable.A full backup of Lync is performed before the upgradeThe Lync administrator team must make sure that there is a valid backup in place to be able to revert to the original system state if the service pack installation fails. We recommend that the backup be restored to a standby server to have this system readily available if there are issues. The cumulative update is deployedThe Lync administrator team does the installation during the planned maintenance cycle.Managing the Timing of ChangesWe recommend that you implement a procedure for scheduling changes to avoid disruptions in overlapping sections of your work. For example, two teams may both be planning a minor change to a system. One team may be applying a cumulative update on a pool while another team is migrating legacy users into that pool. Neither team is affected by the changes that the other team is planning, and each team may not necessarily know about changes that the other team is planning. If both changes occurred at the same time, there could be issues implementing the changes. Also, if there are issues after the changes have been applied, for example, if the user migration fails, it may be difficult to decide which change should be rolled back. There should be regular maintenance periods set up between IT and management to test the changes and accept them.Configuration ManagementConfiguration management is the process of recording and tracking hardware and software assets and system configuration information. It is generally used to track software licenses, maintain a standard hardware and software build for client computers and servers, and define naming standards for new computers. Configuration management generally covers the following categories:HardwareThis category tracks the pieces of equipment that the IT organization owns, where equipment is located, and who uses equipment. This information enables an organization to plan and budget for upgrades, maintain standard hardware builds, report on the value of IT assets for accounting purposes, and help prevent theft.SoftwareThis category tracks software that is installed on each computer, the version numbers, and where the licenses are held. This information helps plan upgrades, ensure that software is licensed, and detect the existence of unauthorized (and unlicensed) software.Standard BuildsThis category tracks the current standard build for the client computers and servers and whether the client computers and servers meet this standard. The existence and enforcement of standard builds helps support staff because the staff is required to maintain only a limited number of versions of each piece of software.Cumulative Updates and HotfixesThis category tracks which service packs are tested and approved for use and which computers are up to date. This information is important to minimize the risk of computers being compromised and to detect users who have installed unapproved updates.System Configuration InformationThis category tracks the function of a system, the interaction between system elements, and the processes that depend on the system running smoothly. For example, third-party proxy server may be configured on a single server. The proxy servers dependence on this server should be understood and contingency plans may be required if there is a failure. If the proxy server can be configured to also communicate with another front-end server, dependencies and contingency plans will probably change.Implementing Configuration ManagementAfter you determine the purpose of your configuration management exercise and decide what items need managing, you need to implement configuration management by collecting data and reporting data. The simplest approach for small organizations is to collect data manually (number and model of client computers, operating system, software installed) and save it in an Office Word or Office Excel document. For larger, more complex, and constantly changing systems, the discovery of assets and collection of detailed information must be automated. Decide what information is relevant to your organization and record it in a database.The configuration management database is a useful tool for support staff and management in the following areas:Security AuditsThe database enables you to identify servers running Lync Server and client computer systems that need to have hotfixes applied or that have missed the installation of a service pack or the latest antivirus updates.Software InstallationIdentifying client software versions and tracking them will aid administrators in planning version updates and new installs as well as assisting with licensing documentation and compliance. Configuration InformationIf you maintain an up-to-date list of all settings that have been modified from their default, then you will be able to troubleshoot issues quickly and more effectively.Planning UpgradesIf a capacity review reveals that additional storage space is required on your Lync database servers, its important to know if each server has an internal RAID controller. If they do, then are they the same model? Do they have the same number of disks installed? The configuration management database will indicate the type of disk that can be installed, the number, and the upgrade path in each case.Tools Used for Configuration ManagementThere are many tools to discover, audit, and report assets. Some of these tools are discussed in this section.Automated ScriptsYou can write simple scripts to report items like the operating system, service pack level, and existence of software on a specific set of computers. You can write these scripts to an organizations exact requirements; however, the required number of scripts and their complexity can make scripts expensive to create and maintain.Automated ToolsDepending on the size of your business and your organizational needs, you may want to consider using automated tools. Tools such as System Center Configuration Manager incorporate standard report templates (such as service pack level) and also enable you to create customized reports, for example, for a custom application. The System Center Operations Manager can also be used to report on hardware and software configurations. Relationship with Change ManagementConfiguration management is closely related to change management. Configuration management identifies the need for change and identifies and records that a change has occurred. For example, the configuration management database can be used to identify servers that require a hotfix. Change management then defines the process for applying the hotfix.Conversely, if a new cumulative update is rolled out, the change management process should supply this information to the configuration management system. The configuration management tools will probably need to be configured to identify the new software so that they can discover and track where and when the software is deployed.System AdministrationSystem administration includes the day-to-day administrative tasks, both planned and on-demand, that are required to keep an IT system operating smoothly. Typically, system administration tasks are covered by written procedures. These procedures help to ensure that the same standard tools and methods are used by all support staff.In a Lync environment, typical system administration tasks include backing up and archiving pools, monitoring logs, creating and managing users, and updating antivirus software.System TroubleshootingAn organization must be prepared to deal with unexpected issues and should have a procedure to manage issues from the point at which they are reported until their resolution. Information about how support staff diagnosed an issue should be recorded and used in the future to avoid unnecessarily repeating completed work.System Troubleshooting ProcessThe following diagram shows the system troubleshooting process and the interactions with other operations roles.System Troubleshooting Flowchart

Classify and PrioritizeThis task is typically performed by the service desk. For example, an issue may be grouped as a software issue or a hardware issue. The issue is then routed to the appropriate support team for investigation. The rules for determining the priority of an issue, together with the time to respond and time to resolve, are typically defined in the SLA.Investigate and DiagnoseThe appropriate support team diagnoses the issue and proposes changes to resolve the issue. If the solution is simple and does not require change control, the solution can be applied immediately. If the solution is not simple, a request for change should be raised and the proposed work should be managed by the change management process, frequently under a fast-track procedure. Any changes that are made should be recorded using the configuration management process.Close and RecordAfter testing the resolution, the issue should be closed. If there are lessons to be learned from the issue, an entry should be created in the knowledge base.Review and Trend AnalysisPeriodic reviews of recent issues should be performed to identify issue trends. For example, if your users are experiencing frequent issues with slow logons to their Lync sites, network bandwidth issues may be the cause. Issue resolution times and the effect of any outages on system availability should be reviewed and compared with the SLA. The person who liaises with the customer on service issues, such as an account manager, should be informed of any significant issues.Issue Management ToolsService desk tools enable staff to record, classify, and prioritize new issues. Tools will then provide the workflow processes to manage the issue service request through investigation and diagnosis, often by more than one support team. Tools, which will frequently provide reports about resolution times and historical trends, may also include a knowledge base database, which can be used to search through past issues.The Microsoft Knowledge Base is a useful record of support issues that have been encountered by Microsoft. For more information, see the Microsoft Support website (http://go.microsoft.com/fwlink/?linkid=14898).Third-party software typically requires customization to suit the organizations needs, such as the organization of teams, reporting requirements, and measures required by the SLA.

Centralized vs. Decentralized AdministrationRoles and responsibilities for performing system administration tasks depend on whether the organization follows a centralized model, a decentralized model, or a combination of both.Centralized ModelIn a centralized model, one or several administrative groups maintain complete control of the whole Lync Server environment. This administrative model is similar to a data center where all administration tasks are performed by a single information technology group. Roles and responsibilities within the team should be defined according to experience and expertise.Decentralized ModelDecentralized organizations are located in several geographic locations and have servers running Lync Servers and teams of administrators in different locations. For example, there may be local administration staff and one or more servers running Lync Server 2010 for each country/region. Alternatively, there may be a pool of servers running Lync Server 2010 and an administrative team for North America and one for Europe. Sometimes, you may want administrators to be responsible only for their own geographical area and restrict permissions to administer resources in other areas.Lync Server also allows you to delegate specific administrative tasks to specific individuals or groups by using role-based access control (RBAC). RBAC allows administrators to delegate specific user rights and permissions to other administrators to perform a subset of the administrative tasks possible. With RBAC, the users ability to carry out specific administrative tasks depends on the RBAC roles that are assigned to the user. RBAC provides a list of cmdlets that the user is allowed to run based on the RBAC roles that the user is a member of.Service Level AgreementsThe SLA is a document that defines the services that your customer expects from you. The complexity and content of this document depends largely on whether customers are internal (within your environment) or external.External CustomersIf your customer is external, the SLA may be part of a legal contract with financial incentives and penalties for performance that falls inside or outside defined levels of service. Defining these levels of service should be part of the overall contract negotiation.As with all contracts, its important that both parties understand expectations. The SLA defines these expectations. The contents of the document should change infrequently and only because of negotiations with the customer.Internal CustomersIf your customer is internal, you may still want to define the services that are expected of operations teams and of IT systems. The SLA may be created by the operations staff and intended as a set of goals for the availability of IT services within your organization. Alternatively, performance levels may be set by management and used as benchmarks when assessing staff performance.Typical CriteriaSLAs include sections that define criteria of minimum levels of availability, support, and capacity.AvailabilityDefine the hours and the operating systems on which sites and other Lync services will be available. Any routine maintenance that affects service availability should be defined. Define external factors that affect service, for example, the loss of Internet connectivity.SupportDefine the hours when support for a system will be available. Specify methods for customers to contact support staff, how incidents are grouped, and target time to respond and to resolve the incident. Define frequency and content of feedback to the customer.CapacityDefine the maximum allowed size of Lync sites and the steps to take if the limit is exceeded. Define the maximum allowed time to do standard tasks, such as the time to retrieve a document from a document library. Define the maximum number of users per Lync pool and agree to a process to increase capacity if more users are added.DocumentationThe MOF model is composed of many service management functions. Documentation about how and when tasks are performed can be shared with members of the same team or with other teams. The method of storing and sharing documentation can vary according to the type of function. For example, the procedures for system administration may be stored as Word documents because they are likely to be printed and referenced frequently. Configuration management information may be automatically generated and stored in a database for easy searching and indexing. Some documentation may be sensitive and should be restricted.Document Management SystemsA documentation management system acts as a central repository for documents and helps to ensure that only the latest revision of a document is available. You can also consider archiving the older version of the document for reference purposes. Lync Server provides functionality suitable to this task. DatabasesSeveral tools and management functions have been discussed that are suited to using databases. The configuration management process is likely to use automated processes that store large amounts of data that require indexing and searching. Support staff may search a database of past issues and resolutions when troubleshooting new issues.It is likely that there will be different databases being used for different purposes. Decide if these databases should be linked or consolidated. For example, if the service desk identifies several issues with a common theme (such as new software causing an issue with a particular network card), the support staff can query the configuration database to predict how many computers might be affected.Using Administrative ToolsThe Lync Server 2010 administrative tools consist of the following:Lync Server Deployment WizardUse to deploy Lync Server 2010.Lync ServerTopology BuilderUse to add any components to your deployment.Lync Server Control PanelUse for ongoing management of your deployment.Lync Server Logging toolUse to troubleshoot problems in your deployment.Lync Server Management ShellUse to manage your deployment at the command line.Lync Server Best Practice Analyzer Lync Server 2010, Best Practices Analyzer is a diagnostic tool that gathers configuration information from Lync Server 2010 environments and determines whether the configuration is set according to Microsoft best practices.The administrative tools are installed by default on each server on which you installed Lync Server 2010. Additionally, you can install the administrative tools on other computers, such as administrative consoles, to facilitate planning and deployment for your organization. You also use most or all of these tools to manage your deployment. For details about installing Topology Builder and the other administrative tools, see Topology Builder Requirements for Installation, Publishing, and Administration in the Deployment documentation and Install Lync Server Administrative Tools in the Operations documentation.When you manage your deployment, you use the following two tools primarily: Topology Builder and Lync Server Control Panel.NoteIn Lync Server 2010, the Lync Server Management Shell is a new method of administration and management. Lync Server Management Shell is a powerful management interface, built on the Windows PowerShell command-line interface that includes a comprehensive set of cmdlets that are specific to Lync Server 2010. With Lync Server Management Shell, you gain a rich set of configuration and automation controls. Topology Builder and Lync Server Control Panel both implement subsets of these cmdlets to support management of Lync Server 2010. The Lync Server Management Shell includes cmdlets for all Lync Server 2010 administration tasks, and you can use the cmdlets individually to manage your deployment. For details, see Lync Server Management Shell in the Operations documentation.Using Topology BuilderIf you need to make changes to your topology after your initial deployment (for example, to add a server to your topology), you must run Topology Builder to make the changes, and then publish the topology again prior to deploying the new component in your topology. Documentation on Topology Builder can be found in the deployment documentation at http://technet.microsoft.com/en-us/library/gg398788(v=ocs.14).aspx.Using Lync Server Control PanelYou can use Lync Server Control Panel to perform most of the administrative tasks required to manage and maintain Lync Server 2010. Use either of the following procedures to open Lync Server Control Panel so you can make changes to the configuration settings for your deployment.NoteYou can use a user account that is assigned to the CSAdministrator role to perform any task in Lync Server Control Panel. You can use other roles to log on to Lync Server Control Panel to administer specific functionality, dependent on the task you need to perform. For example, you can use CSArchivingAdministrator to administer Archiving in Lync Server Control Panel. For details about roles, see Role-Based Access Control in the Planning documentation. For information about the roles that can be used to perform a specific procedure, see the documentation for the procedure.To open Lync Server Control Panel using the administrative URLFrom a user account that is assigned to the CsAdministrator role or other role that has appropriate user rights and permissions for the task to be performed, log on to any computer in your internal deployment.Start Lync Server Control Panel: Open a browser window, and then enter the Admin URL provided by your organization.To open Lync Server Control Panel on a computer running Lync Server 20101. From a user account that is a member of the CsAdministrator role or other role that has appropriate user rights and permissions for the task to be performed, log on to a computer on which you have installed Lync Server 2010.Start Lync Server Control Panel: Click Start, click All Programs, point to Administrative Tools, point to Microsoft Lync Server 2010, and then click Lync Server 2010 Control Panel.Further documentation about Lync Server Control Panel is available at http://go.microsoft.com/fwlink/?LinkId=392057&clcid=0x409. Lync Server 2010 Logging ToolMicrosoft Lync Server 2010 Logging Tool integrates a range of logging and tracing functionality. It facilitates troubleshooting by capturing logging and tracing information from the product while the product is running. Logging Tool is installed with the Lync Server administrative tools. You can use Logging Tool to run debug sessions on any Lync Server role.Lync Server 2010 Logging Tool generates log files on a per-server basis, so it must be actively running and tracing on each computer for which you want to generate a log.Note:To view and analyze log files on a computer other than the one on which the logs were captured, you can run the Lync Server 2010 Logging Tool on the other computer by installing the Lync Server administrative tools on that computer. OCSLogger.exe is the file that runs Logging Tool, which by default is installed to %ProgramFiles%\Common Files\Microsoft Lync Server 2010\Tracing. You can run OCSLogger.exe on computers running the Windows XP, Windows Vista, or Windows 7 operating systems as well as on Windows Server 2003, Windows Server 2008 SP1, and Windows Server 2008 R2. To run OCSLogger.exe on Windows Vista or Windows 7, you must run the application in the Run as Administrator mode.To Start Lync Server 2010 Logging ToolDo one of the following:Open a command prompt (Command Prompt, Windows PowerShell, or Lync Server Management Shell). Navigate to the folder where OCSLogger.exe is installed (by default, %ProgramFiles%\Common Files\Microsoft Lync Server 2010\Tracing), type OCSLogger.exe, and then press Enter.On the Start menu, open the Run dialog box. Type the full path and file name, by default %ProgramFiles%\Common Files\Microsoft Lync Server 2010\Tracing\OCSLogger.exe), and then click OK.On the Start menu, click inside the Search box. Type OCSLogger.exe, and then press Enter.For further details about the Logging Tool, see the Lync Server 2010 Logging Tool documentation in the TechNet Library at http://go.microsoft.com/fwlink/p/?LinkId=199265.Lync Server Management ShellLync Server 2010 introduces a large set of new and improved features compared to what was available in Microsoft Office Communications Server 2007 R2. One improvement is the way in which you manage your implementation. For example, theres a new user interface, called the Lync Server Control Panel, which represents a big shift from what most people are used to with the Microsoft Management Console (MMC). The other major improvement to manageability is the inclusion of Windows PowerShell.Windows PowerShell allows you to manage Microsoft applications from the command line. It includes a command-line environment, product-specific commands, and a full scripting language. Windows PowerShell was first introduced as a downloadable release for the Windows operating system late in 2006, and was incorporated as the command-line interface for manageability of Microsoft Exchange Server 2007. From that point it continued to grow, and it has been incorporated into most of the Microsoft Server products, the most recent of these being Lync Server 2010. Lync Server 2010 introduces close to 550 product-specific cmdlets that you can use to manage every aspect of your deployment.The following sections contain a list of cmdlets and their descriptions. This information is also available directly from the command line. Simply type the following at the Lync Server Management Shell command prompt:Get-Help -FullFor example, to retrieve help from the command prompt on the New-CsVoicePolicy cmdlet, type the following:Get-Help New-CsVoicePolicy -FullThings to know about Windows PowerShell in Lync Server 2010:To run the Lync Server 2010 cmdlets, open the Lync Server Management Shell.Lync Server Management Shell is automatically installed on every Lync Server Enterprise Edition Front End Server or Lync Server Standard Edition.New and updated information, sample scripts, and help for getting started and learning more about Windows PowerShell and Lync Server cmdlets is available at the Lync Server 2010 Windows PowerShell Blog, http://go.microsoft.com/fwlink/?LinkId=203150.For detailed cmdlets documentation:Lync Server 2010 Cmdlets by Category at http://technet.microsoft.com/en-us/library/gg398306.aspxLync Server 2010 Cmdlets Index at http://technet.microsoft.com/en-us/library/gg398867.aspxLync Server Best Practices AnalyzerLync Server 2010 Best Practices Analyzer is a required tool in the sections that follow. Make sure it is installed shortly after implementing Lync Server 2010.You can use the Best Practices Analyzer to identify and resolve issues with your Lync Server deployment. The Best Practices Analyzer gathers configuration information from Lync Server 2010 components.With the proper network access, the Best Practices Analyzer can examine servers running Active Directory Domain Services (ADDS), Exchange Server 2010 Unified Messaging (UM), and Lync Server. You can use the Best Practices Analyzer to do the following: Proactively perform checks, verifying that the configuration is set, according to recommended best practices.Automatically detect required updates to Lync Server 2010.Generate a list of issues, such as suboptimal configuration settings, unsupported options, missing updates, or practices that we do not recommend.Help you troubleshoot and fix specific issues.The Best Practices Analyzer provides the following features:Minimal installation prerequisites.Online documentation about reported issues, including troubleshooting tips.Configuration information that you can save for later review.State-of-the-art system analysis.Best Practices Analyzer uses a set of XML configuration files to determine the information to gather from your Lync Server environment. In addition to checking ADDS, it checks sources such as the Windows Server operating system registry and settings in Windows Management Instrumentation (WMI).Best Practices Analyzer compares the data it gathers with a set of predefined rules for the settings and configurations of Lync Server 2010 deployments.After comparing the collected data with the predefined rules, the tool reports issues. For every issue that it reports, Best Practices Analyzer provides information about what was found in the scanned Lync Server 2010 environment and the recommended configuration. Best Practices Analyzer also provides links to more detailed information about the specific issues.The Best Practices Analyzer gathers configuration information only from Lync Server 2010 components. You can use the previous version of the tool to scan Office Communications Server 2007 R2 and Office Communications Server 2007 environments. For details about downloading and installing the Lync Server 2010 version of the tool, see Lync Server 2010, Best Practices Analyzer at http://go.microsoft.com/fwlink/?LinkId=211230.

Operations Management Operations management involves administering an organizations infrastructure components, and includes the day-to-day administrative tasks, both planned and on-demand, that are required to keep an IT system operating smoothly.In a Lync Server 2010 environment, typical system administration tasks include backing up data, managing settings, monitoring system status and performance, and managing connectivity.Standard ProceduresSeveral resources can help you define what standard procedures are required in the organization, and how to perform them. Because each organization is unique, you may have to further customize and adapt these resources to suit everyday requirements.Standard operational procedures change, and documentation occasionally needs to be revised. As changes are made, the change management process, as defined in the Service Management Functions of the Microsoft Operational Framework, should identify how each change is likely to affect how and when administrative tasks are performed. Use the change management function to update and control the procedural documentation.We recommend that operational tasks be separated into manageable workloads, where tasks are performed on a daily, weekly, monthly, and as-needed basis. Daily tasks would focus efforts on aspects that are critical to the functioning of a system and monthly tasks would focus more on ensuring the long-term health of a system. The tasks that must be performed can be separated into the following categories:Daily Tasks Weekly Tasks Monthly Tasks As-Needed Tasks When preparing documentation for operations management, use checklists to help ensure that the required tasks are performed at the appropriate time. For detailed information about preparing operations checklists, see the sample checklists located in Operations Checklists.Frequently, change management takes over where system administration finishes. If a task is covered by a standard procedure, it is part of the system administration function. If there is no standard procedure for a task, it should be handled by using the change management function.Emergency ProceduresEmergency procedures are on-demand procedures that fall outside the standard procedures described above and should be handled by the change management function (refer to the MOF for a description on the "change management" function). These procedures usually relate to:Implementing change as part of a troubleshooting effortto resolve an identified system issue.Resolve an identified security threatsuch as emergency security patches or software updates.Impact to the business should be evaluated to determine whether a change should be implemented in an ad hoc manner or channeled through the official change management procedures as defined by the Changing Quadrant guidance.We recommend implementing emergency change procedures. For example, document how to deal with these types of procedures and assist with reducing a state of confusion when an emergency change is presented.Daily TasksTo help ensure the availability and reliability of the Lync Server 2010 deployment, you must actively monitor elements that are critical to the functioning of the system, which includes the physical platform, the operating system, and all important Lync Server 2010 services. Preventive maintenance and proactive monitoring will help you identify potential errors and issues that may negatively affect the Lync Server 2010 deployment. Monitoring the Lync Server 2010 deployment involves checking for issues with connections, services, server resources, and system resources. WindowsServer operating systems2008, 2003, and 2003 R2, together with Microsoft System Center Operations Manager 2007, and Lync Server 2010 provide you with many monitoring tools and services to help ensure that the Lync Server 2010 organization is running smoothly. When these technologies are implemented together, administrators will have the ability to receive alerts when or before issues occur. The key advantages to daily monitoring are:Meeting the performance and availability requirements of defined SLAs.Successfully completing specific administrative tasks, such as daily backup operations, and checking server health.Detecting and addressing issues, such as bottlenecks in the server performance, or need for additional resources before they affect productivity.Daily maintenance tasks assist the administrative team to define or establish a criteria or baseline for normal systems operations within the organization, and to detect any abnormal activity. It is important to implement these daily maintenance tasks so that the administrative team can capture and maintain data about the Lync Server 2010 infrastructure, such as usage levels, possible performance bottlenecks, and administrative changes. To help organize the performance of daily tasks, see Daily Operations Checklist.Performing Physical Environmental ChecksBefore checking the performance, availability, and functionality of the Lync Server 2010 deployment, you should check the physical environment. For example, the server room temperature might need to be lowered, or a network cable might need to be replaced. For best results, perform the following physical environmental inspections:Physical security measures Physical security protection such as locks, doors, and restricted-access rooms must be secured. Check for any unauthorized and forced entries and signs of equipment damage.Temperature and humidity High temperature, poor air flow, and humidity can cause hardware components to overheat. Check temperature and humidity to help ensure that the environmental systems such as heating and air conditioning can maintain acceptable conditions and function within the hardware manufacturer's specifications. When new equipment has recently been installed, also check that air flow both to and from the servers is unimpeded and meets manufacturer spec.Devices and components The Lync Server 2010 organization relies on a functioning physical network and related hardware. Make sure that routers, switches, hubs, physical cables, and connectors are operational.The specifics on how to perform these checks will depend greatly on your installation site and the server hardware that was chosen. The first time you perform this check, refer to the hardware documentation and make a note of the desired parameters for future reference.Desired server space environment

ParameterDesired value or range

Temperature

Humidity

Front of server facesHot aisle / cold aisle

Unimpeded exhaust clearance

Performing and Monitoring BackupsYour business priorities should drive the specification of backup and restoration requirements for your organization. Performing backups of the servers and data is the first line of defense in planning for a disaster. Computers that run Lync Server 2010 services or server roles must have a copy of the current topology, current configuration settings, and current policies before they can function in their appointed role. Lync Server is responsible for ensuring that this information is passed along to each computer that needs it. The Export-CsConfiguration and Import-CsConfiguration cmdlets are used to back up and restore your Lync Server topology, configuration settings, and policies during a Central Management store upgrade. The Export-CsConfiguration cmdlets enable you to export data to a .ZIP file; you can then use the Import-CsConfiguration cmdlet to read that .ZIP file and restore the topology, configuration settings, and policies to the Central Management store. After that, the replication services of Lync Server will replicate the restored information to other computers running Lync Server services. The ability to export and import configuration data is also used when initially configuring computers located in your perimeter network (for example, Edge Servers). When configuring a computer in the perimeter network, you must first perform a manual replication using the CsConfiguration cmdlets: you will need to export the configuration data using Export-CsConfiguration and then copy the .ZIP file to the computer in the perimeter network. After that, you can use Import-CsConfiguration and the LocalStore parameter to import the data. You only need to do this once; after that, replication will take place automatically.Who can run this cmdlet: By default, members of the following groups are authorized to run the Export-CsConfiguration cmdlet locally: RTCUniversalServerAdmins. To return a list of all RBAC roles, this cmdlet has been assigned to (including any custom RBAC roles you have created yourself), run the following command from the Windows PowerShell prompt:Get-CsAdminRole | Where-Object {$_.Cmdlets match "Export-CsConfiguration"}All SQL 2008 Back End databases should be backed up as per SQL best practices using the DBExport command.Regular testing of the Disaster Recovery Plan for your Lync Server 2010 infrastructure should be performed in a lab environment that mimics the production environment as closely as practicable. Refer to the Monthly Tasks for details about Disaster Recovery Testing.Note that the backup frequency can be adjusted, based on your Restore Point and Recovery Point objectives. As a best practice, take regular, periodic snapshots throughout the day. Generally, you should perform full backups every 24 hours. Backup all SQL Back End databases using the dbexport command.Checking Disk UsageHard disks drives are a critical component of the Lync Server 2010 deployment. Without sufficient free disk volume, neither the operating system nor the Lync Server 2010 databases can function correctly. You must monitor the Lync Server 2010 back-end database statistics daily to help ensure that servers do not run out of disk space, and to prepare to add storage resources as required.Apart from checking space on disks hosting the operating system, program files, database, and transaction logs (Lync Server 2010 Back End), you should also monitor usage of the file system that includes disk space for file shares containing the following important data:Meeting contentMeeting content metadataMeeting compliance logsApplication data files (used internally by the application server component)Group Chat Server Web service and compliance folders (to store files uploaded to the Group Chat Web service)Group Chat compliance XML files (containing Group Chat compliance records)Update files (for Device Update Service)Address Book filesLync Server 2010 needs hard disk space to store its databases and transaction logs as well as files on file shares previously listed. You should monitor the disk space regularly to help ensure that the Lync Server 2010 deployment is not negatively affected because of insufficient storage resources. Compare and maintain statistical information about available disk space on each Lync Server 2010 volume and expected growth of the databases and transaction log files. This helps with capacity planning and adding storage when the storage resources are required. To accommodate troubleshooting and disaster recovery situations, we recommend that available free volume space be equal or greater than 110 percent of the size of database.You can check free disk space by using the following methods:1. System Center Operations Manager System Center Operations Manager can be used to alert administrators when volume space is constrained.Running a scriptMonitor disk space by running a script that sends you an alert message if the available hard disk space falls below 20 percent. You can find a sample script on Microsoft Script Center on TechNet , look at: http://gallery.technet.microsoft.com/scriptcenter/site/search?query=hard%20disk%20alert&f%5B0%5D.Value=hard%20disk%20alert&f%5B0%5D.Type=SearchText&ac=5Windows ExplorerUse WindowsExplorer to check for disk space on volumes that store Lync Server 2010logs and databases. Checking Event LogsYou can use Windows Event Viewer to view event logs and obtain information about service failures, replication errors in the ADDS, and warnings about system resources such as virtual memory and disk space. Event Viewer is included with Microsoft Windows Server 2008 and 2012. In the Lync Server 2010 Logging Tool, when you end the debug session, click Analyze Log Files to view the log files by using the Snooper tool.Event Viewer maintains logs about application, security, and system events on the computer. Both Lync Server 2010andWindows report warnings and error conditions to the event logs. Therefore, make sure that you review event logs daily.Use Event Viewer to: View and manage event logs. Obtain information about hardware, software, and system issues that must be resolved.Identify trends that require future action.A server that is running a WindowsServeroperating system records events in three types of logs:Application logsThe Application log contains events logged by applications or programs. Developers determine which events to log. For example, a database program might record a file error in the Application log. Most Lync Server 2010Server-related events appear in the Application log.Security logsThe Security log records events such as valid and invalid logon attempts as well as events related to resource use such as creating, opening, or deleting files or other objects. For example, if logon auditing is enabled, attempts to log on to the system are recorded in the Security log.System logsThe System log contains events logged by Windows system components. For example, the failure of a driver or other system component to load during startup is recorded in the System log. The event types logged by system components are predetermined by the server.Lync Server 2010The logging tool records significant events related to authentication, connections, and user actions. After enabling diagnostic logging, you can view the log entries in Event Viewer.NoteWe do not recommend using the maximum logging settings unless you are instructed to do this by Microsoft Product Support Services. Maximum logging drains significant resources and can give many false positives, that is, errors that get logged only at maximum logging but are really expected and are not a cause for concern. We also recommend that you do not enable diagnostic logging permanently. Use it only when troubleshooting.Within each Event Viewer log, Lync Server 2010 Server records informational, warning, and error events. Monitor these logs closely to track the types of transactions being conducted on the Lync Server 2010 servers. You should periodically archive the logs or use automatic rollover to avoid running out of space. Because log files can occupy a finite amount of space, increase the log size (for example, to 50MB) and set it to overwrite, so that the Lync Server 2010Server can continue to write new events.You can also automate event log administration by using the following tools and technologies:System Center Operations Manager System Center Operations Manager 2007 monitors the health and use of Lync Server 2010 servers. Lync Server 2010 Management Pack for Operations Manager 2007 SP1 extends Operations Manager by providing specialized monitoring for servers that are running Lync Server 2010.The Lync Server 2010 Management Pack for Operations Manager monitors Standard and Enterprise Edition of Lync Server 2010. This release also incorporates the Quality of Experience (QoE) Management Pack which was previously a separate management pack. Monitored types are event log entries, performance counters as well as stateful monitoring of QoE. This version of the management pack only monitors Lync Server 2010, and cannot be used to monitor Office Communications Server 2007.The management pack provides the following features: Alerts indicating service impacting eventsAlerts indicating configuration, and other non-service impacting issuesState monitoring of Lync Server servicesProduct knowledge: Cause and resolution of identified issuesFor more information about Lync Server 2010 Management Pack, refer to Monitoring Lync Server 2010 with System Center Operations Manager.Event CombThe Event Comb tool gathers specific events from the event logs of several computers to one central location. It lets you report on only the event IDs or event sources it specifies. For more information about Event Comb, see the Account Lockout and Management Tools website.Event triggers In Windows Server 2008 you have the ability to "Attach a Task to This Event" within the Windows Event Viewerwhere an administrator can either run a program, send an email message or display an on-screen message. For more information about this feature, see the Windows Server 2008 R2 topic Run a Task in Response to a Given Event.You can also use command-line tools such as Eventtrigger.exe to create and query event logs and associate programs with particular logged events. By using Eventtriggers.exe, you can create event triggers that run programs when specific events occur. For more information about event triggers, see the WindowsServer2003topic New command-line tools.Monitoring Lync Server 2010 PerformanceLync Server 2010 performance is affected by various factors such as user profiles, system architecture, software, hardware components, third-party integration points such as gateways and telephony equipment, network connectivity and performance, Windows Active Directory service configuration and performance as well as the Windows operating system functionality.At the core of a Lync Server 2010 deployments performance is the server software and hardware it is implemented on. As an example, it is essential that a front-end server has sufficient hardware resources to cope with the expected (short-term) user load. If a respective front-end server is required to provide services to 10 thousand users, then an adequately configured server is essential to meet the expected load requirements to ultimately help ensure the best possible end-user experience.Monitoring server performance is thus critical to gauge whether the implemented server infrastructure have adequate hardware resources for the day-to-day peak-load requirements. Monitoring server performance helps identify system bottlenecks allowing administrators to apply corrective action before the end-user experience is affected. The performance data should be used for long-term capacity planning.While detailed information on all performance objects and counters to be observed is linked to in Monitoring Lync Server 2010 with System Center Operations Manager 2007, a few performance counters you should follow can provide administrators a quick view of the system performance:To track overall system health of the front-end server, a good starting point is to check Processor\% Processor Time. The value should be below 80 percent at all times.To track the performance of the back end Microsoft SQL Server database software instance used by the Front End pool, monitor the following performance counters:LC:USrv 00 DBStore\Usrv 002 Queue Latency (msec)LC:USrv 00 DBStore\Usrv 0 04 Sproc Latency (msec)The healthy server at steady state should show