isilon advisor user guide - 162.249.33.144
TRANSCRIPT
Page 1
Isilon Advisor User Guide
THIS TECHNICAL DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS ANDTECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.
Copyright © 2009 - 2016 Dell Inc. All rights reserved. Dell and the Dell EMC logo are trademarks of Dell Inc. in the United States and/or otherjurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.
Page 2
Isilon AdvisorTopics
Welcome to Isilon Advisor
Running Isilon AdvisorLog Set Collection (Gather)Command Line Interface (CLI)
User Interface
Settings
Check DescriptionsCluster ChecksCluster Configuration ChecksNode ChecksNode Status ChecksDisk Checks
Running an Analysis
Analysis ResultsExporting ResultsGenerating a ReportSharing and Receiving Results
Previous Runs
Upgrade Advisor
FAQ
Requesting Access to the Log Processor (Internal Users only)
Support
Page 3
Welcome to Isilon Advisor
Description
The EMC Isilon Advisor is a smart, fast and effective application that enables customers to self-support common Isilon issues and acceleratetime-to-resolution. This is the same application used by EMC Isilon Technical Support Engineers and Field Representatives to resolve servicerequests. Use it to diagnose, troubleshoot and proactively avoid issues by analyzing the current health of your cluster and listing items that requireattention.
Items that require attention can range from simple checks to critical alerts. They are listed in order of importance and color-coded for quick visualreference ( : critical, : needs attention, : no problem found). More importantly, Isilon Advisor provides links to documentation onred yellow greenhow to resolve the issues identified and if further assistance by EMC Support or Field is needed, a summary of each check can be extracted to aflat file and attached to a Service Request (SR). This information will aid the EMC team in resolving your issue faster.
Isilon Advisor performs the following checks:
Pre-upgradePost-upgradeGeneral HealthFirmwareCustom Check
Features
Proactively identify issues and potential solutionsRun reports on 50+ different checksTake preventative action on known issuesGain visibility into the health of your environment in minutes
Benefits
Accelerate Time-to-ResolutionIA enables you to resolve documented known issues without engaging EMC. You can resolve issues immediately after beingdiagnosed therefore eliminating one business day or longer from the troubleshooting process.
Proactively Avoid IssuesThe built-in Health Check feature diagnoses current issues and identifies potential ones. Leverage years of collective knowledgefrom EMC and routinely run the Health Check to ensure good hygiene.
Reduce Capital and Operational ExpensesBy resolving issues early you reduce or eliminate future critical issues that may result in data loss, data unavailability anddowntime. Faster time-to-resolution equates to less time spent troubleshooting and faster restoration to peak performance andavailability.
Quickly Assess Your EnvironmentThe Isilon Advisor provides visibility into the overall health of your environment within a matter of minutes, enabling you tosustain optimal cluster health with minimal overhead and time spent.
Intended Users
Isilon Advisor is intended for Isilon customers, EMC personnel, and business partners.
Page 4
Running Isilon Advisor
Overview and Download
The EMC Isilon Advisor is a smart, fast and effective application that enables customers to self-support common Isilon issues and acceleratetime-to-resolution. Use it to diagnose, troubleshoot and proactively avoid issues by analyzing the current health of your cluster and listing itemsthat require attention.
See Isilon Advisor for additional information and a link to the download.
Running the Application
Isilon Advisor is a self-contained single file executable and is not installed. Simply copy the appropriate executable from the zip file into thedirectory of your choice and execute it. This launches both a command window to run the web server and your default browser usinglocahost:{port} and the default ports of 8080, 8081 and 8088. Optionally, you can specify a specific port using the Command Line Interface (CLI).
When executed for the first time, an IsilonAdvisor directory is created in one of the following locations:
Windows: C:\Users\<username>\IsilonAdvisor\Linux: /home/<username>/IsilonAdvisor/
Scripting
As of version 2.3.0, scripting is supported through the .Command Line Interface (CLI)
Space Considerations
Log sets can be gigabytes in size and are contained in a tarball which has to be unfurled, so plan accordingly for the necessary free space forboth the tarball and the unfurled data.
Log Sets
You must use full log sets gathered with the isi_gather command. For collection details, refer to .Log Set Collection (Gather)
OneFS Support
The following OneFS versions are supported:
7.1.x 7.2.x8.0.x
Operating System Support
The following operating systems are supported:
Windows 7 and 10Ubuntu 15.x or 16.x (Linux)
Browser Support
The following browsers are supported:
Chrome 39 or laterFirefox 34 or laterSafari 7 or later
The minimum free space required to run Isilon Advisor is 4GB
In addition, tgz files require that you must have enough space to uncompress the entire log set and still have free space left
Page 5
Beware that unsupported browsers, such as Internet Explorer (IE) may cause erratic and undesirable behavior with inputs andformatting.
Page 6
1. 2. 3.
1. 2. 3.
1. 2.
Log Set Collection (Gather)
Where are my Isilon logs?
Compressed log sets can be found at "/ifs/data/Isilon_Support/pkg/"
The will have a naming convention of "log folder IsilonLogs-[clustername]-YYYYMMDD-hhmmss.tgz"
Collecting Logs
See the below procedure for how to gather logs and configuration files from the cluster using the OneFS web administration interface and the isi_, or for a demonstration of these procedures, see the video below.gather_info command
Procedure
You can gather logs using the OneFS web administration interface or the command-line interface.
OneFS web administration interface
OneFS 7.x
Log in to the OneFS web administration interface.Click .Help > DiagnosticsClick Start Gather.
It takes several minutes to gather the logs. When the process is complete, the logs are automatically uploaded to the Isilon FTP site. Thecompleted log gather is listed in the Archived Info Manager section.
OneFS 6.0 - 6.5
Log in to the OneFS web administration interface.Click .Help > Diagnostics > Gather InfoClick .Start Gather
It takes several minutes to gather the logs. When the process is complete, the logs are automatically uploaded to the Isilon FTP site. Thecompleted log gather is listed in the section.Archived Info Manager
Command-line interface
Open an SSH connection on any node in the cluster and log on using the "root" account.Run the following command:
isi_gather_info
It takes several minutes to gather the logs. When the process is complete, the logs are automatically uploaded to the Isilon FTP site.
NOTEIf problems are encountered during the upload process, the package will need to be sent manually to .ftp.isilon.com
You can find the package in the following location, where <timestamp> is the date and time that the script was run:
/ifs/data/Isilon_Support/pkg/IsilonLogs-mycluster-<timestamp>.tgz
For detailed instructions for uploading files to , see , .ftp.isilon.com How to upload files to Isilon Technical Support 16759
NOTEFor a list of the options that can be used with the isi_gather_info command, run the following command from the command line:
isi_gather_info -h
Page 7
1. 2. 3.
Command Line Interface (CLI)
Using the CLI enables users to create a script to run Isilon Advisor against multiple logs, automatically run a specific check against those logs,and export the results. It also enables a user to use an alternate port to run multiple instances of Isilon Advisor at the same time, or because thedefault ports 8080, 8081 and 8088 are already in use.
Parameter Value(s) Required Description
-check "Health Check", "Firmware Check", "PreUpgradeCheck", "PostUpgrade Check"
Yes This is the check that the application will run.
-output Path to where the output should be placed Yes This is the folder where the results will be placed.
-logset Path to the log set or a folder name Yes Specifies if a folder in which case all files located in that folder, or a singlefile in which case that single file will be processed.
-exportResults yes | no No If set to yes, then the exported results will be triggered and placed in thespecified folderoutput
-srNumber 0 - 999999999 No Optional and contains the SR that is being used for this analysis.
-help N/A No Displays a list of options.available
-port 1024 - 65535 No This is the port that the IsilonAdvisor will use. If no value is specified, theapplication will use 8080, 8081 or 8088.
i.e. http://localhost:{port}
Examples
Run a health check and export results to a file
IsilonAdvisor_2. -check= srNumberx.x_os "Health Check" -output="c:\data\report_output" -logset="c:\data\logs\logfile123.tgz" -exportResults=yes -=12345678
Runs the Health Check analysis on the log file "c:\data\logs\logfile123.tgz" and place the results and the exported results into the"c:\data\report_output" folder.
Run Isilon Advisor on an alternate port
IsilonAdvisor_2. -port=45007x.x_os
Runs the IsilonAdvisor on port 45007 and start up the default browser on http:\\localhost:45007.
Display the CLI parameters
IsilonAdvisor_2. -helpx.x_os
Displays the help information for the IsilonAdvisor CLI and exit.
Validation
If the required parameters passed are in error or not present, the application will not start in any form.If the port range is not valid the application will not start.If the port passed is already in use, the application will not start.
Page 8
User Interface
Responsive Web Design
The user interface uses a responsive web design which is an approach aimed at providing an optimal viewing experience across multipleplatform/devices: desktop, mobile and tablet. It provides easy reading and navigation with a minimum of resizing, panning, and scrolling. Forexample, when the browser window changes from landscape to portrait, the form fields reflow to match the changes in dimensions.
Navgation Bar
Item Description
Home Display the with the and .home page inputs available analyses
Settings Global settings.
PreviousRuns
Previous runs displays a list of the that have been run. Selecting for a previous check displays the previous analyses Resultsresults from that check without needing to re-analyze the logs.
Help Display the for answers to .FAQs commonly asked questions
Support How to for bugs, questions, and enhancement requests. request support
? Page level help for the displayed page.
Selecting a Log Set
Enter the or use the button to navigate to it.Log Set Location* browse
*Internal users attempting to access a log set on the Log Processor will be prompted for their DELL EMC SSO credentials and must have beenapproved for access.
See for requesting access.Requesting Access to the Log Processor (Internal Users only)
Selecting an Analysis
Click the appropriate to navigate between analyses. tab
Running an Analysis
See Running an Analysis
Page 9
Reviewing Results
Analysis Overview
Click the and click the "+" or text to expand the overview "-" or text again to close the overview.
Cluster Results Summary
Summarizes critical and warning items as well as installed patches for quick and easy reference.
Open / Close All
Click or to open or close all of the analysis results.Open All Close All
Page 10
Sort / Filter By
Use and to organize and filter the analysis results.Sort By Filter By
Displaying Details
For any results row, click the and display the details and click the . "+" or text to expand the results "-" or text again will close the details
Copy and Paste
Copying and pasting of details is not available via the right-click context menu so just use the standard Windows shortcut keys for copy, Ctrl-C Ct for paste.rl-V
Page 11
Settings
Global Settings
Setting Description
Reports Folder Location for creation of the Isilon Advisor reports.
The default location is in subdirectory in the directory.Reports IsilonAdvisor
Windows: C:\Users\<username>\IsilonAdvisor\Reports\Linux: /home/<username>/IsilonAdvisor/Reports/
Page 12
Check Descriptions
See each type of check for the appropriate descriptions.
Cluster Checks
Cluster Configuration Checks
Node Checks
Node Status Checks
Disk Checks
Page 13
Cluster Checks
Check Name Description
ActiveUpgrade
Checks for running upgrade processes on the cluster. During an upgrade, major changes or administrative tasks to the clustershould not be performed until upgrade is completed.
Cluster Health Reports the overall health of the cluster per OneFS.
Cluster Name Reports the name of the cluster.
Cluster Size Reports the total cluster size of the cluster per OneFS.
Cluster Time
Reports time drift, cluster time, and time sync. If the cluster time is off, or if nodes are reporting a time drift, authentication andsubsequent access to the cluster can be affected.
Time Drift reports the largest amount of time between two nodes. The IA will report if the time drift exceeds 60 seconds.Cluster time is determined by nodes in quorum (50% + 1 nodes) having the same time to the nearest minute. The IA willprovide a notification if the cluster time cannot be determined. This can happen if the time is not synchronized on at least50% + 1 of the nodes.Time Sync shows the cluster's time sync configuration and will warn if the cluster is not configured to sync its time with anexternal NTP server.
ContactInformation
Reports the primary and secondary contact information configured on the cluster to ensure notifications are reaching theproper personnel.
EMCTechnicalAdvisories(ETA)
Reports on conditions that meet active EMC Technical Advisories (ETAs). ETAs alert customers to potential data unavailableor data loss scenarios.
InstalledPatches
Reports installed patches and checks whether the cluster has any out-of-date patches installed.
Job Status Checks the status of all on-cluster maintenance jobs enabled on the cluster.
Notification Reports how email notifications are configured on the cluster to ensure notifications are reaching the proper personnel.
OneFSVersion
Reports the OneFS version that is running on the cluster.
Open Events Reports all open events as seen by OneFS
OpenSMMaster
Reports which node is the subnet master.Checks for InfiniBand switch configuration errors.Checks for misconfigured InfiniBand network settings.
Storage Pools Reports the storage pool configuration and any issues.
UpgradeService
Checks the upgrade service status and reports any issues.
Configuration See .Cluster Configuration Checks
Page 14
Cluster Configuration Checks
Check Name Description
ESRS Reports whether ESRS is enabled or disabled.
File SharingConfiguration
Reports isi_auth output and checks for errors.
Gateway Priority Checks the priority on all routable gateways and flags any gateways that share the same priority level. When gatewaysshare the same priority level, the network could experience degraded performance.
Global NamespaceAcceleration (GNA)
Reports the Global Namespace Acceleration (GNA) status and hardware qualifications for enabling GNA. Flags anycustom sysctls, debug sysctls, and efs.lbm.leak_freed_blocks in sysctl.conf file.
HDFS Configuration Checks to see if HDFS is enabled or disabled and reports status and configuration.
Installed Licenses Reports the license status of software modules for the cluster.
Isi Snapshot D Log Checks for Snapshot errors.
Isi Sync Policy Reports the SyncIQ policies configured on the cluster.
NFS Status Lists all NFS exports that are configured on the cluster.
Report Snapshot Info Reports the Snapshot details for the cluster.
SmartConnectIQ Checks and reports the configured subnets, pools, node members, and devices for SmartConnectIQ and AdvancedSmartConnectIQ.
Service PrincipalNames (SPN) List
Lists the service principal names (SPNs) for Kerberos authentication.
SupportIQ Status Checks whether SupportIQ is enabled or disabled.
Page 15
Node Checks
CheckName
Description
Boot DriveCheck
Checks SSD boot drives and partition mirroring status and reports any error conditions that could indicate a failing or failed bootdrive.
NodeCounts
Reports mismatched nodes by counting each node type found in the cluster. If any single node type has less than three nodes,that node type will be flagged as being mismatched. Also flags nodes that are EOSL.
NodeFirmware
Checks for firmware versions and flags any nodes below current firmware levels.
Node Health Checks the health of each node and flags any that nodes that report status other than OK.
NodePartitions
Flags any nodes with any writeable partition at 92%+ full.
Node Size Checks for free space on HDDs and SSDs and reports any node with 10% or less space remaining.
NodeUptime
Checks the node uptime and reports any nodes with an uptime of 180+ days.
NVRAMBatteryCheck
Checks the NVRAM battery status to determine if the batteries are charged sufficiently to preserve the contents of the NVRAMand reports any issues.
Status See .Node Status Checks
Page 16
Node Status Checks
Check Name Description
CTO Report The CTO reports the serial number, configuration number, product number, and the AsBuilt Record (ABR) for all nodes inthe cluster at the time the nodes were built. This helps to verify proper hardware in the event new hardware is needed.
DMI Log Flags any nodes on which DIMM errors are present.
Hardware Status Checks for any devices that are reporting errors.
Kernel OpenFiles
Reports a count of open files by node.
Lwiod Logs Reports errors in the Lwiod logs.
NetstatConnections
Reports connection counts to and from each node and reports any listen queue overflow errors. This can be used toidentify performance issues or network hardware issues.
NetworkInterface Card(NIC) Status
Reports the NIC status of each node.
Routing Tables Displays each node's network routing table. This can be used to identify network issues.
Var Crash Reports the capacity of the /var/crash partition on the cluster.
Page 17
Disk Checks
CheckName
Description
Bay Health Reports drive bay health status of each node.
DiskFirmware
Checks for firmware versions and flags any drives below current firmware levels.
Load Checks the disk load of all disks in each node. Heavy disk load will affect performance.
IDI Errors Reports any IDI errors found. IDI errors are Isilon Data Integrity errors and have the potential to indicate data loss orunavailability.
Page 18
1. 2. 3. 4.
Running an Analysis
Workflow
Select a log set*Click an analysis typeFor a , click the appropriate checkscustom checkRun the analysis
Log Set
Enter the or use the button to navigate to it. Log Set Location* browse
*Internal users attempting to access a log set on the Log Processor will be prompted for their DELL EMC SSO credentials and must have beenapproved for access.
See for requesting access.Requesting Access to the Log Processor (Internal Users only)
Analysis Type
Click the appropriate to select the type of analysis to run. tab
Analysis Description
Pre-upgrade Check Determines whether or not the cluster is ready for an upgrade, and is based on the Health Check, but with tighterthresholds.
Post-upgradeCheck
Determines if an upgrade is still in progress, and is based on the Health Check, but adds the upgrade process check.
Health Check Checks on the state of the cluster, including the nodes and disks.
Firmware Check Checks whether node and disk firmware are current.
Custom Check Based on the Health Check with some additional checks, and enables selection of any or all checks.
Custom Check Analysis
When selecting a custom check, you can select which group(s) or specific check(s) you want to run. Just click the appropriate group(s), specificcheck(s) or to run all of these checks.Check All
Page 19
Run the Analysis
Click on Run <check type> Check
Page 20
Analysis Results
All results are color-coded for quick visual reference ( : critical, : needs attention, : no problem found).red yellow green
Status Summary
A summary of the cluster, node and disk conditions is displayed and depicts the highest level risk found for each with a summary of the number ofcritical and warning items and the number of tests completed.
Status Description
Critical Items that need to be investigated and addressed. These items indicate a condition that could potentially cause dataunavailability or data loss and should be addressed immediately.
Warning Indicate a configuration or state that should be investigated to determine the appropriate action. This may or may not meanthat an action is required. This may be items that are not in accordance with engineering standards or best practices.
Informational Items that do not need to be investigated or addressed.
Overview
The analysis overview contains a label for the analysis type run and once expanded provides an overview of which log was processed, the clusterit came from and optionally a SR# (service request number) associated with this analysis. As Isilon Advisor (IA) analyzes the cluster and nodestatus based on known issues and best practices, the IA health status may not agree with the ISI (OneFS) health status. This is in part due to thefact that IA has more current information and thus more checks than OneFS.
Field Description
Log Path Items that need to be investigated and addressed. These items indicate a condition that could potentially cause dataunavailability or data loss and should be addressed immediately.
SR# Indicates a setting is misconfigured or that the cluster is experiencing a problem. These items require attention andinvestigation as soon as possible.
ClusterName
Name of the cluster for the associated log.
Log Date Timestamp from the associated log.
Page 21
Cluster/NodeName
The name of the cluster and node for the associated log.
ISI Status The health status as seen by OneFS.cluster or node
IA Status The cluster or node health status as seen by Isilon Advisor.
CRIT - Items that need to be investigated and addressed. These items indicate a condition that could potentially causedata unavailability or data loss and should be addressed immediately.
- WARN Indicates a setting is misconfigured or that the cluster is experiencing a problem. These items should beinvestigated as soon as possible.
- OK Items that do not need to be investigated or addressed.
Cluster Results Summary
Summarized list of just the critical and warning items and the installed patches for a quick and easy reference.
Field Description
Check Name Name of the check that was run.
Result Result summary of the check that was run. For installed patches, a detailed list of the patches.
KB / Docu Link List of relevant reference materials
Results
The initial view summarizes all results into brief, single line results.
Page 22
Details
To view details for any result, click and expand a that result. When possible the details in highlight the item found, and the links take you to reddocumentation with additional information about your item.
Page 23
1. 2. 3. 4.
Exporting Results
Use exported results to share Isilon Advisor results with others, such as support or coworkers. The exported results are contained in a single zipfile and can be opened in any of the and does not require Isilon Advisor to be installed. Exported results are displayed usingsupported browsersthe same results view as seen in Isilon Advisor.
Click the button Export ResultsSelect a for the exported results folderExport is saved to the folderCopy and the to the desired receiver send zip file
Browser Support
The following browsers are supported:
Chrome 39 or laterFirefox 34 or laterSafari 7 or later
Beware that unsupported browsers, such as Internet Explorer (IE) may cause erratic and undesirable behavior with inputs andformatting.
Page 24
1. 2. 3. 4.
Generating a Report
Use reports to share Isilon Advisor results with others, such as support, coworkers or customers. Word reports contain both summary anddetailed level results, while PowerPoint reports contain only summary level results.
Click the button Generate ReportSelect a report typeSelect a for the reportfolder Report is saved to the folder
Page 25
1.
1. 2. 3.
1. 2.
a. b.
Sharing and Receiving Results
Share Exported Results
This method works well for sharing the results with anybody, regardless of whether or not they have Isilon Advisor installed, and presents thesame results view as seen in Isilon Advisor.
Sharing
See Exporting Results
Receiving
Unzip the zip file to the desired folderOpen the unzipped folderOpen the html file
Share Summary text file
This method works good for sharing the results with EMC support or field personnel who are familiar with the technical aspects of an IsilonCluster, such as EMC Isilon Support.
Sharing
Navigate to the appropriate directory for the clusterCopy from the appropriate directory and send it to the appropriate receiverAnalysisType_Check_yyyymmdd-hhmmss.txt
Windows: C:\Users\<username>\IsilonAdvisor\ReportsLinux: /home/<username>/IsilonAdvisor/Reports
Page 26
Previous Runs
Previous runs displays a list of the that have been run. Selecting for a previous check displays the results from that previous analyses Resultscheck without needing to re-analyze the logs.
Page 27
1. 2. 3.
Upgrade Advisor
Use Upgrade Advisor to generate pre-upgrade, upgrade and post-upgrade steps with links to relevant documentation.
Select the Current OneFS VersionSelect the Target OneFS VersionExpand and review the results
Page 28
FAQ
Use the for answers to .FAQs commonly asked questions
Page 29
1. 2. 3. 4. 5. 6. 7. 8. 9.
10. 11. 12.
Requesting Access to the Log Processor (Internal Users only)
Please log a request using the account portal to be added to LPXUsers group.
Go to http://accountportal.emc.comSelect ACTIONS, then Request Access from the dropdown menuFrom the User Detail tab, go to the Find Profile section and search for your profileSelect NextFrom the Access Catalog tab, select "Isilon – Log Processor (LP)" from the Filter By dropdown listSelect "Isilon – Log Processor (LP)" from the "Isilon – Log Processor (LP)" dropdown listFrom the PROPEL Role Dictionary section, select "+" from the add columnEnter your justification and click OKSelect NextFrom the Verify request tab, select your access request and click SubmitAdditional information can be found hereYour request will be routed to the appropriate team for approval
Only required if you map the drive as HTTPS uses SSO validation
Page 30
Support
GS Tools Support has updated our support processes to better serveour users.
For GS Tools products only, you no longer have to create a Service Request (SR) simply use the"Contact Us" option on the NavBar and we will do that for you. Once the SR is created you willreceive a notification via email containing the SR number and a link to it. Please open the emailand and not to any of the emails. add any files or screenshots to the SR
Contact Us is available from our GSnavigation bar. Click onTools Central
the Call out icon to launch theContact Us form.
Click on " "Request Support
For "Request Support" thefollowing fields are required
Subject (mandatory) -be descriptive the tool orapplication name
Environment (optional) -Browser versions addedautomatically, addadditional informationsuch as operatingsystem depending onyour application beingdesktop or web based.
Contact Email (optional)- added automatically,can be changed ifrequired.
Phone Number (mandatory) - enter a phonenumber including countrycode for our support staffto contact you ifrequired.
Message (mandatory) -enter as muchinformation as possibleto explain the issue youare having and any
Page 31
application information,versions, history, actionsetc
Click Send
Confirmation message will bereceived, please follow theinstructions in the message.