spotlight on sql server getting started guide

Spotlight on SQLServer 8.0

Getting Started Guide

2011 Quest Software, Inc.ALL RIGHTS RESERVED.

This guide contains proprietary information protected by copyright. The software described inthis guide is furnished under a software license or nondisclosure agreement. This software may beused or copied only in accordance with the terms of the applicable agreement. No part of thisguide may be reproduced or transmitted in any form or by any means, electronic or mechanical,including photocopying and recording for any purpose other than the purchasers personal usewithout the written permission of Quest Software, Inc.

The information in this document is provided in connection with Quest products. No license,express or implied, by estoppel or otherwise, to any intellectual property right is granted by thisdocument or in connection with the sale of Quest products. EXCEPT AS SET FORTH INQUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FORTHIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMSANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTSINCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESSINTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE ORINABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THEPOSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respectto the accuracy or completeness of the contents of this document and reserves the right to makechanges to specifications and product descriptions at any time without notice. Quest does notmake any commitment to update the information contained in this document.

If you have any questions regarding your potential use of this material, contact:

Quest Software World HeadquartersLEGAL Dept5 Polaris WayAliso Viejo, CA 92656email: [email protected]

Refer to our Web site (www.quest.com) for regional and international office information.

Patents

Protected by U.S. Patent #6,901,582. Additional patents pending.

Trademarks

Quest, Quest Software, the Quest Software logo and Spotlight are trademarks and registeredtrademarks of Quest Software, Inc in the United States of America and other countries. For acomplete list of Quest Softwares trademarks, please seehttp://www.quest.com/legal/trademark-information.aspx. Other trademarks and registeredtrademarks are property of their respective owners.

Spotlight on SQL Server 8.0

Getting Started Guide

Monday, 16 May 2011

Table of Contents

Use Spotlight on SQL Server 5

Diagnose CPU 5

Diagnose IO 6

Diagnose Blocking 8

View Historical Data 10

Understand Playback 11

Identify Expensive SQL 12

Turn on and configure SQL Analysis 12

Real-Time Data and Expensive SQL 14

Data Over a Selected Time Frame and Expensive SQL 15

Filter Displayed and Collected Data 16

Report on Database Growth 18

Analyze Wait Statistics 20

Find Long Running SQL 21

Configure the Virtualization Overhead Guage 22

Connections to VMware Servers 23

Troubleshoot Spotlight on SQL Server 24

Installation Issues 24

Remote installation of the diagnostic server fails 24

Diagnostic Server Issues 24

Connection Issues 25

Problems connecting to a SQL Server Instance 26

WMI Issues 26

Error 0x800706BA 26

Error 0x80070005 27

Error 0x80041010 27

Spotlight on SQL Server Getting Started GuideTable of Contents

4

Error 0x80041017 29

Miscellaneous Issues 29

Monitor Multiple SQL Server Instances 29

Unable to perform certain operations 29

Missing SQL Server Performance Counters 30

Session Trace Errors 30

Playback 31

Connections to SQL Servers 32

Appendix: Contact Quest 35

Contact Quest Support 35

Spotlight Support 35

Contact Quest Software 36

About Quest Software 36

Index 37

Use Spotlight on SQL Server

Diagnose CPUSpotlight continually monitors CPU usage on the target server and alarms if it crosses athreshold. The thresholds are:

CPU Usage Alarm severity

0 67% Normal

67 80% Low

80 93% Medium

93 100% High

A CPU alarm indicates that a bottleneck on CPU is likely. This bottleneck could exist for anumber of reasons expensive queries could be running, excessive compilation or recompilationcould be occurring, or a process on the server, not even related to SQL Server, is hogging theCPU. With Spotlight, you can investigate the causes of a bottleneck both in real-time and in theimmediate past.

When Spotlight detects that there is a bottleneck on the CPU, an alarm is raised. You can seealarms on the Spotlight home page, and in Spotlight Today.

1

Spotlight on SQL Server Getting Started GuideUse Spotlight on SQL Server

6

Our first order of business is to find out what process is using all of the CPU.

From the Spotlight Home Page From Spotlight Today

Click Diagnostics. Select the alarm and click Diagnose.

This opens the Diagnostics drilldown. The Diagnostics drilldown helps you answerquestions such as:

l Why is CPU usage higher than usual?

l Is the SQL Server causing high CPU usage?

l If it is the SQL Server, is it optimizing queries or executing them?

The first grid on the CPU page shows possible causes of CPU pressure and the likelihood thateach is contributing to high CPU usage on the server being monitored. Click an item in the gridto display an explanation of the diagnosis and supporting information.

Diagnose IOSustained high I/O wait time rates are a good indicator that of a disk subsystem bottleneck andthat I/O device service times will be degraded. When Spotlight detects unusually high diskactivity on the SQL Server machine it raises an alarm.

Spotlight looks at SQL Server I/O wait times to determine if there is an IO bottleneck. A valuelarger than 15 ms/IO indicates a possible bottleneck.

The thresholds for the I/O wait time alarm are:


7

I/O Wait Time Alarm severity

0 - 15 Normal

15 - 50 Medium

50 - infinity High

After detecting a potential disk bottleneck, Spotlight displays an alarm on the home page and inSpotlight Today.

The first thing we need to do is identify the process that is generating high amounts ofI/O activity.

From the Spotlight Home Page From Spotlight Today

Click Diagnostics |IO. Select the alarm and click Diagnose.

This opens the I/O Diagnostics drilldown. The I/O Diagnostics drilldown helps you answerquestions such as:

l Why is the I/O higher than usual?

l Is the high I/O due to paging activity?

l Is the SQL Server causing the high I/O?

l If the SQL Server is causing the high I/O, is it through query executions, scan operators,PAGE_IO_LATCH or LOG waits?


8

The first grid on the I/O page shows the possible causes of high I/O and the likelihood of eachcontributing to this instance of the alarm. Click an item in the grid to display an explanation ofthe diagnosis and supporting information.

Diagnose BlockingBlocking in SQL Server occurs when a session that was performing some task is unable toprogress because it must wait on a resource that is currently being used by another session. Theresource that is being waited on can be either a physical structure like a lock on a table or aninternal SQL Server resource such as a latch.

Excessive blocking can be a major cause of poor application performance since it reduces thethroughput of the system. Often a user of an application does not realize that they are waiting ona resource held by another user. From their point of view, it often seems like their application hasstopped responding.

When diagnosing blocking, you want to start your investigation by answering thefollowing questions:

l Who is waiting on what?

l How long have they been waiting?

l What SQL was running while they were waiting?

Spotlight not only alerts you to blocking issues in your database but also helps you answer theabove questions so you can quickly resolve the problem causing the blocking.


9

When Spotlight detects a blocking issue, an alarm is displayed on the home page.

The first thing we want to do is look at the Blocking drilldown. The Blocking drilldown showsdetails about the current blocks and in particular, answers the questions stated above.

On the Spotlight homepage, click the Blocked Processes button, and then click theBlocking drilldown.

The Blocking grid shows the sessions that are either blocked or are causing other sessions to beblocked. The hierarchy in the tree diagram makes it easy to see which session is causing theblocking. In addition, the resource on which the sessions are waiting is displayed. You can askSpotlight to resolve these to actual object names by clicking on the ? button. The time spentwaiting and the SQL last executed are also displayed.


10

From here, you can view session details.

1. Select the entry in the grid.

2. Click View Session Details.

View Historical DataNote: This feature has limited support in the Desktop Edition. A maximum of four hoursplayback data is available. Scheduling cannot be configured.

In Spotlight, you can view events and data collected in the recent past as though they werehappening in real-time. You can do this via the Playback ribbon The Spotlight home page andpage drilldowns show details in playback identical in format to that displayed in "live" state, soyou can view historical data in the same way as you view a live connection.

To playback time

1. Select a connection from the Spotlight browser.

2. Select the Monitor ribbon tab.

3. Click to show:

Alarms Description

Playback

Select an alarm or moment in time in the past to return to and viewin the Spotlight home page and page drilldowns.


11

Alarms Description

RealTime

Return to present time the Spotlight home page and pagedrilldowns.

Rewind Go back in time on the Spotlight home page or page drilldown.Click the associate arrow to define how far back to travel: 5minutes, 10 minutes, 1 hour or 1 day.

Skip Starting from the past, skip forward in time on the Spotlight homepage or page drilldown. Click the associate arrow to define how farforward to skip: 5 minutes, 10 minutes, 1 hour or 1 day.

Play Starting from the past, step forward in time through the alarms insequence. Show on the Spotlight home page or page drilldown.Click the associate arrow to define the speed of play.

Understand Playback

Note: This feature has limited support in the Desktop Edition. A maximum of four hoursplayback data is available. Scheduling cannot be configured.

Click Playback and a new control opens.

Control Description

Select the date. Use the back and forward buttons to change the date.

Click the time to playback to.The alarms list scrolls to show those alarms occurring at the selected time.Note:The time scale is colored according to the most severe alarm for theconnection.

The alarms occurring on the selected date.Click an alarm to playback to the time the alarm occurred.Note:The color indicates the severity of the alarm.


12

Identify Expensive SQLIdentifying expensive SQL can be a time consuming process. Use the SQL Analysis feature inSpotlight to identify SQL that is consuming resources on your server.

With SQL Analysis, you can quickly answer questions such as:

l What SQL has executed the most?

l What SQL consumed the most CPU or I/O in total?

l What SQL consumed the most Average CPU or I/O?

l What SQL took to longest time to execute?

Answering these questions will help you to direct your tuning efforts more productively.

SQL Analysis uses either ROWSET or Server Side SQL Server trace to collect raw data onSQL statement execution. It then aggregates this data by similar SQL statements so thatpotentially thousands or even millions of rows of raw data are distilled down into anaggregate picture of workload.

You can set filters at point of collection so that only the events that you are interested in arecollected and when viewing the data, you can also filter the data to narrow the focus downeven further.

Turn on and configure SQL Analysis

1. Click Configure | SQLAnalysis Filters

2. Choose one of the following from the connection list:

l To change the setting for an individual connection, select the connection name.

l To change the setting for all connections, select Default Settings.

3. Select Override the default settings.

4. Select Enable SQL Analysis.

5. Click Data Collection.

6. Configure the following fields on the Data Collection page:

Option Description

SQL Analysis isscheduled to collectevery n minutes

Note: This feature is not available in the Desktop Edition.How often Spotlight collects SQL Analysis data. Thedefault is six hours.


13

Option Description

To change how often Spotlight collects SQL Analysisdata

a. Click the link.

b. Select Override the default settings for thecollection 'SQL Analysis'.

c. Under Store reporting data in the SpotlightStatistics Repository, click the Store data link.

d. Set the collection schedule. Data can be collectedat regular intervals or at a specific time on one ormore days.

Use continuousRowset trace

Select to use Rowset trace to collect SQL Analysis data.Rowset trace is convenient though may affect throughputon busy servers. It should be used with care. It is thedefault collection method.

Use sampling ServerSide trace

Select to use sampling Server Side trace to collect SQLAnalysis data.Server Side trace is more suitable for busy servers thanRowset trace. Server Side trace requires additionalconfiguration.

Run trace for or untiltrace file reaches

Server Side trace will run until one of these conditions ismet.Specify the maximum amount of time Server Side traceshould run for.Specify the maximum size of the trace file.

Server trace filelocation (on the SQLServer host)

Type a location for the server trace file. The path isrelative to the SQL Server instance.For example, C:\Trace Files.

Retrieve data throughSQ Server

Select to process trace files on the SQL Server instance.This option has the potential to affect throughput on busyservers.Use this option when there are permission or firewallsettings preventing the Diagnostic Server machine fromretrieving files from the SQL Server machine.This is the default option.

Retrieve data from thefile system

Select to process trace files on the Diagnostic Server. TheDiagnostic Server machine must have the appropriate


14

Option Description

permissions required to connect to the SQL Serverinstance.This option is not available when configuring defaultsettings.

Trace file location(from DiagnosticServer host)

Type the location of the server trace files on the SQLServer instance. The location should be a shared folder theDiagnostic Server can connect to.For example, \\computername\Trace Files.This option is not available when configuring defaultsettings.

Real-Time Data and Expensive SQL

You can use the SQL Analysis grid when load testing in a testing environment. It shows SQLAnalysis data in real time.

If you are trying to answer the question I am simulating application workload in my testenvironment and I want to look at a breakdown of SQL statements in real-time. I am interested inwhat SQL statements are consuming the most average CPU right now., you would want to lookat an aggregate of SQL statement executions (workload) in real-time. The SQL Analysis grid inthe SQL Activity drilldown allows you to do exactly that.

1. Select a SQLServer connection from the Spotlight browser.

2. Click Monitor |SQLActivity.

3. Select SQLAnalysis.

4. Do the following:

To identify Sort the grid by

SQL that consumed the most CPU Total CPU

The average CPU consumed by a particularstatement

Average CPU

SQL that was executed the most Execution Count

SQL with the highest logical I/O Average Reads

Tips:


15

l Use filters to refine the data shown in the grid. See "Filter Displayed and Collected Data"(page 16) for more information.

l Some columns are hidden by default. To view hidden columns, right-click the grid headerrow and select Organize Columns.

l Results are cleared when you leave the SQL Analysis grid. Click Clear Grid to clear thegrid manually.

l To wrap a column onto multiple lines, right-click the data content of the grid and selectProperties | Options.

Note: Word wrap may degrade the performance of Spotlight.

l If Quest SQL Optimizer is installed, you can use it to tune the non-conforming SQL thathas been identified. To do this, click Optimize SQL.

Data Over a Selected Time Frame and Expensive SQL

Note: This feature is not available in the Desktop Edition.

The SQLAnalysis - Workload view shows the aggregate workload over a selected time period.Use the SQL Analysis - Workload view when diagnosing slow response times.

If you are trying to answer the question What SQL statement consumed the most CPU overmonth end processing?, you would want to look at an aggregate of SQL statement executions(workload) over a time range. The SQL Analysis Workload view allows you to do exactly that.

1. Select Reporting and Trending |Views | SQLServer Performance | SQLAnalysisWorkload in the Spotlight Browser.

2. Click Report | Select Connections and select at least one connection.


16

3. Select a Time Range.

4. Click Refresh.

5. Do the following:

To identify Sort the grid by

SQL that consumed the most CPU Total CPU

The average CPU consumed by a particularstatement

Avg CPU

SQL that was executed the most Execution Count

SQL with the highest logical I/O Avg Reads

Tip: Use filters to refine the data shown in the grid. See "Filter Displayed and CollectedData" (page 16) for more information.See "Filter Displayed and Collected Data" (page 16)for more information.

Filter Displayed and Collected Data

To filter the data displayed

Start from the data displayed in the SQLAnalysis grid or the SQLAnalysis Workload view

SQLAnalysis grid SQLAnalysis Workload view

1. Click Change Filter.

2. Select Filter results.

1. Click Filter Data.

2. Select Filter results.

To add a filter to the list

1. Click Add.

2. From the Column list, select the column you want to base your filter on.

3. From the Condition list, select a condition to apply to the column.

4. In the Value field, type a value to filter events by. You can use the percent sign (%)wildcard with the is like/is not like condition.


17

Tips:

l You cannot use wildcard characters for the Database name when connected to a SQLServer 2000 instance.

l To turn off filtering, clear the Filter results checkbox.

l To edit or delete a filter select it in the filter list and click the appropriate button.

To filter the data collected

1. Click Configure | SQL Analysis Filters.

2. Choose one of the following from the connection list:

l To change the setting for an individual connection, select the connection name.

l To change the setting for all connections, select Default Settings.

3. Select Override the default settings.

4. Ensure the Filters page is selected.

l Click Add.

Tips:

l You can use the percent sign (%) and underscore (_) as wildcards.

l You cannot use wildcard characters in the Database name field whenconnected to a SQL Server 2000 instance.

l When filtering on a string field such as DatabaseName orApplicationName, use only the "LIKE" and "NOTLIKE" operators. Theother operators apply only to numeric values.

Tips:

l Use the Arrow buttons to change the order of the filters.

l Edit or delete a filter by selecting it in the filter list and clicking theappropriate button.

l Filtering affects the average, minimum, maximum, and last values displayed inthe grid, as calculations are performed only on data retrieved from the SQLServer instance.

l To turn off filtering, on the Filters page, click Clear. This removes all filters.

5. You can specify the number and type of SQL Executions retrieved from the SQL Serverinstance


18

a. Select the Advanced page.

b. In the Aggregate the top n SQL statements field, enter the maximum number ofSQL executions to be retrieved from the SQL Server instance.

c. From the Sorting by list, specify the criterion you want Spotlight to use todetermine what the "top" SQL statements are. Choose from Average Duration,Average CPU, or Average IO.

Spotlight aggregates the data collected and then stores only the aggregated data.

Report on Database GrowthNote: This feature is not available in the Desktop Edition.

To report on database growth across multiple SQL servers, use the Large Databases view inSpotlight Reporting and Trending. The Large Databases view shows the largest databases on theselected SQL Servers. With Spotlight, you can view this data in a number of different ways.

1. Select Reporting and Trending |Views | SQLServer Environment |Large Databasesin the Spotlight Browser.

2. Click Report | Select Connections and select at least one connection.

3. Select a Time Range.

4. Click Refresh.

By default, the data is shown as a bar chart, listing each database on the selected servers in orderof size. You can change the way the data is displayed using the Chart View button, locatedabove the chart.


19

For example, if you wanted to view the growth of your largest databases over time you canchange the chart to a line graph. To do this:

1. Click the Chart View button and then select Line from the list.

2. Click Refresh.

To look at the space consumption proportionally you could use a pie chart. ClickChart View |Pie.

You can use the data and chart from the view in your own reports by exporting the view. To dothis, click Export and choose a location to save the files. Spotlight saves the chart as a JPG fileand the grid as a CSV file.


20

Analyze Wait StatisticsWhen troubleshooting SQL Server performance issues, one area you can focus your investigationon is waits. Long wait times can be an indicator of performance bottlenecks so identifying wherewaits are occurring may assist you in your tuning efforts.

Spotlight not only provides an easy way to see if your SQL server has a problem with waits butalso identifies on which particular resource waits are occurring. Spotlight takes the raw data fromSQL Server and automatically calculates the rate of wait over time so you have a more currentperspective.

To start your investigation into waits, go to the SQL Activity drilldown and click the WaitStatistics tab.

Look at the Wait Time Breakdown chart. Spotlight breaks waits down into categories based onthe type of wait. The chart shows the amount of time used for each wait category as a proportionof the total CPU time available. If the chart is showing a large proportion of waits or signal waitsthen your SQL server is spending time waiting and not doing useful work. Ideally, the WaitTime Breakdown chart will show a high proportion of either CPU time or unused CPU time,meaning the server is either doing useful work or is idle.


21

To see the data from the Wait Time Breakdown chart shown over time, look at the Wait TimeTrend chart.

The Waits Detail grid shows detailed waits statistics for the SQL Server instance beingmonitored. It shows all wait types in SQL Server and breaks down the statistics into signal timeand resource wait time. (SQL Server provides only wait time and signal wait time.) By default,the Waits Detail grid is sorted by Wait Time Rate (ms/s) which allows you to immediately seewhich wait types are being waited on right now.

Find Long Running SQLThe SQL - Long Running SQL alarm alerts you to SQL that has been running longer than aspecified time frame. Because this time frame is unique to each environment, the SQL - LongRunning SQL alarm, by default, is not configured.

ScenarioYou work in an OLTP environment and often get complaints aboutdelays in server processing. You want to be notified when SQL on aparticular server is running longer than expected so you can investigatewhat is causing the delay. You configure Spotlight to raise a mediumseverity alarm when it detects SQL running for longer than five minutes.When the SQL - Long Running SQL alarm is raised, you use theSessions drilldown to view the long running SQL.

To configure the Alarm: SQL - Long Running SQL

1. Click Configure |Alarms.

2. Select a connection to configure. Select Default Settings to apply the configuration to allconnections.

3. Select the alarm SQL - Long Running SQL.

4. Select Override the default settings for the alarm "SQL - Long Running SQL".

5. Click Add Severity and select a severity.


22

Scenario: Click Add Severity and select Medium.

6. Select the check box of the new severity.

7. Copy and paste the text in the Description cell from the Normal severity to the newseverity.

8. In the Start cell, type the duration in seconds, of how long the SQL should run for beforethe alarm is raised.

Scenario: In the Start cell, type 300 .

Tip: You can also configure different alarm severities using keys. In the SQL - Long RunningSQL alarm, the key is the SPID. See "Configure Keyed Alarms" in the online help for moreinformation.

Configure the Virtualization Overhead GuageNote: This feature is not available in the Desktop Edition.

The Virtualization Overhead gauge shows the percentage of CPU that is unavailable to thisvirtual machine because it is being consumed either by other virtual machines or byVMware itself.

In a virtual environment, the physical CPU of a host is shared by virtual machines and theVMware Hypervisor. This means there can be multiple processes wanting to use the physicalCPU of the host at the same time. The result is that virtual machines may be ready to run buthave to wait to be scheduled on a CPU. Vmware call this ready time. The VirtualizationOverhead gauge shows the amount of ready time as a percentage of the theoretical maximumCPU available to the virtual machine.

To see virtualization data on Spotlight Home Pages

1. There needs to be a connection to the VMware Server. See "Connections to VMwareServers" (page 23) for more information.

2. In the Spotlight Browser, right-click the Windows connection and select Properties.

3. In the Host field, select the connection to the VMware Server.

4. In the VM Name field, select the name of the virtual machine.

The Virtualization Overhead gauge now displays data on the corresponding Windows HomePage and SQL Server Home Page.

Tip: SQL Server connections automatically use the virtualization data from their correspondingWindows connections. You do not need to configure the SQL Server connection once theWindows connection is configured.


23

Connections to VMware Servers


To create a connection:

Note: Use the following procedure to create a VMware connection to the ESX or VirtualCenterserver that hosts the Windows machine.

1. Click Configure | Connections.

2. In the Connection Bar on the left, click VMware.

3. Double-click Add new connection.

4. Complete the connection details:

Field Description

Address The IP address or hostname of the ESX server or VirtualCenter serveryou want to connect to.

User A valid user account on the target system.Note:For Spotlight to monitor a VMware server, the username specifiedmust have at least a read-only role for the ESX server or VirtualCenterserver that you want to connect to.

Password The password required to connect the specified user to the server.

Troubleshoot Spotlight on SQL Server

Installation Issues

Remote installation of the diagnostic server fails


Remote installation of the Diagnostic Server fails if the user has both a domain account and alocal account with the same user name.

Additionally, if the user uses a domain account that does not have permission to access theremote machine (that is, the account does not have administrator privileges) then the remoteinstallation will not complete.

Diagnostic Server IssuesA Diagnostic Server on a separate machine continuously performs the collection of Spotlightdata. The collected data can then be displayed whenever a Spotlight client is started on amachine that has access to the specified Diagnostic Server.

You may encounter the following situation(s):

Unable to establish a Spotlight connection to a SQL Server instance

When the Diagnostic Server is unable to establish a Spotlight connection to a specified SQLServer, you may need to upgrade the current installation of Microsoft's Data AccessComponents (MDAC) on:

l The machine where the Spotlight client is installed AND

l The machine where the Diagnostic Server is installed

For more information on MDAC and MDAC installation, go to the Microsoft Download Centerat http://www.microsoft.com/downloads/en/default.aspx and search for "MDAC".

Unable to connect to the Diagnostic Server

Note:This section is not relevant to the Desktop Edition.

If you are unable to create a connection between the Spotlight client and Diagnostic Server,check the following:

2

Spotlight on SQL Server Getting Started GuideTroubleshoot Spotlight on SQL Server

25

l If a user is not a member of either the Quest Diagnostic Users or Quest DiagnosticAdministrators groups on the Diagnostic Server they will be unable to connect to theDiagnostic Server. When this happens a "Diagnostic Server Unavailable" page isdisplayed on the Spotlight on SQL Server client with the message "Userauthentication failed".

See "Update the Quest Diagnostic User Groups" in the online help for more informationon how to add a user to the Quest Diagnostic user groups.

l Ensure the Spotlight client and Diagnostic Server are running the same version ofSpotlight on SQL Server. When upgrading Spotlight on SQL Server you must upgradeboth the Spotlight client and Diagnostic Server. For more information, see "UpgradeSpotlight on SQL Server" in the Spotlight on SQL Server Deployment Guide.

Unable to collect data

If, after making a connection, alarms are raised related to inability to collect data, checkthe following:

l The domain is included in the user name when specifying Windows OS (not SQL Server)connection details. For example, domain\johnsmith, instead of johnsmith.

l If the monitored host is the same as the machine running the Diagnostic Server, select theWindows authentication option (under OS Connection Details) on the Details page,which is displayed when you create a new connection.

l If the monitored host is in a different domain to the machine running the DiagnosticServer, specify the host name as a fully qualified address (for example,machine1.domain.company.corp). You can specify the host name on the Details pagewhen you create a new connection.

Tip: If the host name is not fully qualified, the connection may work but onlyintermittently. It is recommended that the host name be fully qualified

l If you get errors executing "LogicalDisk" or "PagingFile" queries, start a command prompton the monitored machine and execute the command "diskperf -y" then reboot.

l Do not use Windows authentication for connections remote to the Diagnostic Servermachine unless you have configured the Diagnostic Server process to run under a userwith access to the remote machine. (By default the Diagnostic Server runs under the"Local System" account, which will not have privileges on remote machines).

If you have done all of the above and there are still connection issues, reboot the DiagnosticServer machine (this is sometimes needed due to credentials conflicts in cached Windowsconnections). This must be a full machine reboot, not just a restart of the Diagnostic Server.

Connection IssuesSpotlight on SQL Server retrieves its data from the SQL Server via the Diagnostic Server. TheDiagnostic Server also uses WMI queries to retrieve performance counter information from the


26

Windows server on which SQL Server is running.

The Diagnostic Server cannot connect to a server unless it can access all of this information. Thismeans you need to have the appropriate permissions on the server.

Also, the data must be retrievable over your network. Spotlight on SQL Server requires morethan just a specific port TCP/IP connection to SQL Server to do this. The port on the DiagnosticServer must be open. For more information, see the Spotlight on SQL Server Deployment Guide.This document is available in PDF from Start | Quest Software | Spotlight.

Problems connecting to a SQL Server Instance

If you have trouble connecting to a SQL Server instance, check that you can:

l Connect to the SQL Server using another tool such as Microsoft's SQL ServerManagement Studio, or sqlcmd.

l Connect to WMI using another tool such as Microsoft's WMI CIM Studio, or wmic.

If you have problems with any of these checks, you will need to investigate and resolve theseissues before you can connect to that server using Spotlight on SQL Server.

WMI IssuesThe Diagnostic Server uses Microsoft WMI technology to collect performance data. Thefollowing is a guide to troubleshooting WMI errors.

Error 0x800706BAThe RPC server is unavailable. Exception from HRESULT: 0x800706BA

Cause

The Diagnostic Server is not able to establish a connection with the target computer.

Solution

Check the following:

l The address of the server is entered correctly via the Spotlight Connection Manager.

l The Remote Procedure Call (RPC) service is running on the target computer.

l TCP Port 135 is open both to internal traffic on the Diagnostic Server and thetarget computer.

WMI opens an undetermined port in addition to port 135. This can be troublesome in afirewalled environment. Specifying a port range for WMI is recommended for this type ofenvironment. See "WMI connectivity" in the Spotlight on SQLServer Deployment Guide


27

for instructions on how to specify a port range. The use of port scanning software issometimes required to locate the ports needed to be opened.

Error 0x80070005Access is denied. Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)

Cause

The Windows user specified to monitor the target computer is unknown to the target or doesnthave administrator rights.

Solution

Check the following:

l If the connection is configured to use Windows Authentication, check that the userrunning the Diagnostic Server Service has administrator rights.

l If the connection is configured to use named credentials, check that the user hasadministrator rights.

Error 0x80041010WMI query failed: Invalid class. 0x80041010

Cause

The WMI class does not exist on the target machine

Solution

Try any of the following to recreate the WMIclasses:

Windows Connections

l Recreate the WMI classes by opening a command prompt window and running thefollowing command:

wmiadap /f

l Use the Microsoft Extensible Counter List (Exctrlst) utility to enable thefollowing counters:

Win32_PerfDisk, Win32_PerfNet, Win32_PerfOS and Win32_PerfProc.

See http://www.microsoft.com/downloads/details.aspx?familyid=7ff99683-b7ec-4da6-92ab-793193604ba4&displaylang=en for information.

Note: A reboot of the target machine is required.

SQL Server Analysis Services Connections


28

l Recreate the WMI classes by opening a command prompt window on the machinehosting the SQL Server Analysis Services instance. Run the following command:

wmiadap /f

l Unregister and re-register the WMI classes. To do so:

1. On the problematic machine with SQLServer Analysis Services installed,find out the SQL installation path. See below for the default installationpath. Your installation path may be different.

SQL Server Analysis Services 2008

C:\Program Files\Microsoft SQLServer\MSAS10.MSSQLSERVER\OLAP\bin\Counters

SQL Server Analysis Services 2005

C:\Program Files\Microsoft SQL Server\MSSQL.2\OLAP\bin

Note: You are looking for the directory that contains the following files:

For unamed instances:

MSSQLServerOLAPService

perf-MSSQLServerOLAPServicemsmdctr.ini

For named instances:

MSOLAP$Your_SSAS_NamedInstanceName

perf-Your_SSAS_NamedInstanceNamemsmdctr.ini

If you need to check the named instance name, use services.msc.

2. At the command prompt, change directory to the installation path.

For example, cd C:\Program Files\Microsoft SQLServer\MSAS10.MSSQLSERVER\OLAP\bin\Counters

3. Run the following commands to unload and load counters:

For unamed instances, run:

unlodctr MSSQLServerOLAPService

then

lodctr perf-MSSQLServerOLAPServicemsmdctr.ini

For named instances, run:

unlodctr MSOLAP$Your_SSAS_NamedInstanceName


29

then

lodctr perf-Your_SSAS_NamedInstanceNamemsmdctr.ini

Tip: Replace Your_SSAS_NamedInstanceName with the SQLServerAnalysis Services named instance name. If you need to check the namedinstance name, use services.msc.

4. Run the following command to parse all the performance libraries onthe system and refresh the performance counter classes on theproblematic machine:

wmiadap /f

Error 0x80041017WMI query failed: Invalid query. 0x80041017

Cause

The columns returned by the WMI class are incomplete or unexpected.

Solution

Recreate the WMI classes using:

wmiadap.exe /f

See http://msdn.microsoft.com/en-us/library/aa394528(VS.85).aspx for more information.

Miscellaneous Issues

Monitor Multiple SQL Server Instances

This version of Spotlight on SQL Server is designed to monitor a maximum of 100 SQL Serveror Analysis Services or Replication instances and 100 Windows servers. Monitoring more thanthis recommended limit may result in poor performance or product instability.

Unable to perform certain operations

If, whilst using the Spotlight on SQL Server client, a message is displayed stating:

You are not connected to the Diagnostic Server as a member of the Quest DiagnosticAdministrators group. Only members of that group can perform the requested operation.

and the user needs to be able to perform the operation that led to this message, the Spotlight onSQL Server client user must be added to the Quest Diagnostic Administrators group on theDiagnostic Server.


30

See Update the Quest Diagnostic user groups in the Spotlight on SQL Server DeploymentGuide, for more information.

Missing SQL Server Performance Counters

Spotlight on SQL Server uses the SQL Server virtual table sysperfinfo (SQL Server 2000) andsys.dm_os_performance_counters (SQL Server 2005 and 2008) to retrieve data for many of itsdisplays. In some rare cases, this table may not contain information. When this is the case,Spotlight on SQL Server will not be able to collect the data it requires, and will display "0" formany of its components. Most obvious will be the Memory icons on the home page, which willshow 0 MB of memory used by SQL Server. Also, many of the flows on the home page willshow no activity, and many drilldowns will show incomplete information.

Spotlight on SQL Server raises an informational alarm shortly after connecting if it detects thatthe sysperfinfo or sys.dm_os_performance_counters table contains no data.

You can confirm the existence of this problem by running the following SQL in Query Analyzerfor SQL Server 2000:

select * from master..sysperfinfo

or by running the following SQL in SQL Server Management Studio for SQL Server2005 and 2008:

select * from sys.dm_os_performance_counters

If this query returns no records, then your SQL Server performance counters are missing andSpotlight on SQL Server will not be able to operate correctly.

Sometimes, for a variety of reasons, the SQL Server Performance Monitor counters will not showup as they should. Often, but not always, this problem can be fixed by following these steps:

1. At the command prompt, type the following: unlodctr.exe MSSQLServer

2. Then type: lodctr.exe \binn\sqlctr.ini

3. Reboot the server.

More information can be found in the SQL Server Knowledge Base at msdn.microsoft.com.

Session Trace Errors

If errors are displayed on the Session Trace page (SQL Activity drilldown | Sessions page), installthe following on the Diagnostic Server:

l SQL Server Management Tools

When diagnosing SQL Server 2000 or SQL Server 2005 instances, the SQL Server 2005Management Tools are required to use the Session Trace functionality in Spotlight. Whendiagnosing SQL Server 2008, the SQL Server 2008 Management Tools are required.


31

l Windows XP Service Pack 2

Applies only if the Diagnostic Server is running on Windows XP.

PlaybackThis feature has limited support in the Desktop Edition as a maximum of four hours playbackdata is available.

Playback data is collected at scheduled intervals, in response to some alarms, and also whileviewing a drilldown using the Spotlight client. This means that history data may not beavailable for a drilldown in Playback mode. You can change the rate at which data is collected.See "Configure Scheduling" in the online help for more information.

Connections to SQL ServersTo view an existing connection:

Select the connection from the Recent connections or Connections |SQLServer menu inthe Spotlight Browser.

This opens the Spotlight Home Page.

To add multiple SQLServer instances via discovery or by importing them from a file

1. Click Configure |Connections.

2. In the Connection Bar on the left, click SQLServer.

3. In the Connection list, double-click Discover.

4. Complete the steps in the wizard.

Action Information

Discovery Note: The Discovery Wizard locates and displays the SQL Servers onthe network that you may want the Diagnostic Server to monitor. Ifthe wizard discovers no instances of SQL Servers on the network, onereason may be that the SQL Server components that enable Spotlightto discover those servers have not been installed. See "Install SQLServer Client Tools" in the online help for more information.

Importfrom file

To import SQLServer instances from a file, each line in the file mustcontain a reference to a single SQLServer instance with optional username and password details. The format is:

Instance name[,User[,Password]]

If no user name is specified, Windows authentication is used.

Limitations This version of Spotlight on SQL Server is designed to monitor amaximum of 100 SQL Server or Analysis Services or Replicationinstances and 100 Windows servers. Monitoring more than thisrecommended limit may result in poor performance or productinstability.

3

Spotlight on SQL Server Getting Started GuideConnections to SQL Servers

33

To add an individual connection

1. Click Configure |Connections.

2. In the Connection Bar on the left, click SQLServer.

3. In the Connection list, double-click Add new connection.

4. Complete the connection details:

Field Description

Target SQL Server Instance

Address Enter the connect string used to link to a server (that is, the ServerName, Server Instance Name, or IP address).

WindowsAuthentication(usingDiagnosticServercredentials)

Select this option to use the user name and password configuredon the Diagnostic Server to connect to the SQL Server instance.

Database User A valid user that enables Spotlight to connect to the target system.

Password The password required to connect the specified user to the targetsystem.

Linked OS Connection

Connection The IP address, hostname, or URL of a Windows machine youwant to connect to.If you do not want to monitor an associated Windows machine,select Do not monitor.

Create Click to create a new connection to a Windows machine.You will need to complete the following:AddressThe IP address, hostname, or URL of the Windows machine youwant to connect to.Use Diagnostic Server credentialsSelect this option to use the user name and password configuredon the Diagnostic Server to connect to the Windows machine.User

Spotlight on SQL Server Getting Started GuideConnections to SQL Servers

34

Field Description

A valid user account on the target system.Note: Spotlight on Windows needs to use an account with theprivileges required to retrieve server information, query theregistry, and access WMI and performance monitor objects. Anaccount with administrative rights to the target machine allowsthis.PasswordThe password required to connect the specified user to theWindows machine.

Appendix: Contact Quest

Contact Quest SupportQuest Support is available to customers who have a trial version of a Quest product or who havepurchased a Quest product and have a valid maintenance contract. Quest Support providesunlimited 24x7 access to SupportLink, our self-service portal. Visit SupportLink athttp://support.quest.com.

From SupportLink, you can do the following:

l Retrieve thousands of solutions from our online Knowledgebase

l Download the latest releases and service packs

l Create, update and review Support cases

View the Global Support Guide for a detailed explanation of support programs, online services,contact information, policies and procedures. The guide is available at: http://support.quest.com.

Spotlight Support

When contacting Quest Support about a Spotlight product, please have the version and buildnumber handy. You can obtain both by choosing the About option from the Help menu in therelevant Spotlight application. If your question is about an error message, write the messagedown and have it available for the technical support representative.

Whenever you have a question about Spotlight, click Support Bundle or Contact Support onthe Help menu. This creates a file called SpotlightSupport.zip. This file contains a snapshot ofyour Spotlight installation. Send this file and your request for assistance to Quest Software([email protected]).

Note: You can find SpotlightSupport.zip in different locations under different operating systems.

l Windows XP: C:\Documents and Settings\\Application Data\QuestSoftware\Spotlight\Default\Support

l Vista: C:\Users\\AppData\Roaming\Quest Software\Spotlight\Default\Support

If you find that information in the Help system or printed user guides is incorrect, unclear, orincomplete, please report the problem to Quest Support.

Spotlight on SQL Server Getting Started GuideAppendix: Contact Quest

36

Contact Quest Software

Email [email protected]

Mail

Quest Software, Inc.World Headquarters5 Polaris WayAliso Viejo, CA 92656USA

Web site www.quest.com

See our web site for regional and international office information.

About Quest SoftwareNow more than ever, organizations need to work smart and improve efficiency. Quest Softwarecreates and supports smart systems management productshelping our customers solve everydayIT challenges faster and easier. Visit www.quest.com more information.

Index

A

about Quest Software 36

B

blocking 8

C

connect 32

connection limit 29

connections

issues 25

troubleshoot 25

Contact Quest Software 36

counters 30

CPU usage 5

Customer support 35

D

database growth 18

Diagnostic Server

issues 24

E

error codes, WMI 26

expensive SQL 12

H

historical data 10

I

installation issues 24

IO bottleneck 6

M

MDAC 24

monitor 32

P

performance counters missing 30

Playback 10

Q

Quest Software 36

Quest support 35

R

rowset trace 13

S

server side trace 13

session trace errors 30

SQL

expensive 12

long running 21

SQL Analysis

configure 12

filter data 16

SQL Server

connection 32

connection limit 29

Spotlight on SQL Server Getting Started GuideIndex

38

performance counters 30

Support 35

T

Technical support 35

troubleshoot

connections 25

Diagnostic Server 24

installation 24

performance counters 30

Playback 31

session trace 30

unable to perform certain operations 29

user groups 29

WMI 26

V

Virtualization Overhead gauge 22

VMware

connection 23

W

wait statistics 20

WMI issues 26

Use Spotlight on SQL ServerDiagnose CPUDiagnose IODiagnose BlockingView Historical DataUnderstand Playback

Identify Expensive SQLTurn on and configure SQL AnalysisReal-Time Data and Expensive SQLData Over a Selected Time Frame and Expensive SQLFilter Displayed and Collected Data

Report on Database GrowthAnalyze Wait StatisticsFind Long Running SQLConfigure the Virtualization Overhead GuageConnections to VMware Servers

Troubleshoot Spotlight on SQL ServerInstallation IssuesRemote installation of the diagnostic server fails

Diagnostic Server IssuesConnection IssuesProblems connecting to a SQL Server Instance

WMI IssuesError 0x800706BAError 0x80070005Error 0x80041010Error 0x80041017

Miscellaneous IssuesMonitor Multiple SQL Server InstancesUnable to perform certain operationsMissing SQL Server Performance CountersSession Trace ErrorsPlayback

Connections to SQL ServersAppendix: Contact QuestContact Quest SupportSpotlight Support

Contact Quest SoftwareAbout Quest Software

Index

spotlight on sql server getting started guide

Documents

quest software logo

software license

sale of quest products

sql server

expensive sql

sql analysis

loss of information

long running sql