nimbleos 5.2.1.800 release notes

NimbleOS 5.2.1.800 Release Notes

Published October, 2021Version 5.2.1.800.

Legal Notices

Copyright © 2021 by Hewlett Packard Enterprise Development LP

Notices

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise productsand services are set forth in the express warranty statements accompanying such products and services. Nothing herein should beconstrued as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors oromissions contained herein.

Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistentwith FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Com-mercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no controlover and is not responsible for information outside the Hewlett Packard Enterprise website.

Hewlett Packard Enterprise believes in being unconditionally inclusive. If terms in this document are recognized as offensive ornoninclusive, they are used only for consistency within the product. When the product is updated to remove the terms, this documentwill be updated.

Acknowledgments

Intel®, Itanium®, Pentium®, Intel Inside®, and the Intel Inside logo are trademarks of Intel Corporation in the United States and othercountries.

Microsoft® and Windows® are either registered trademarks or trademarks of Microsoft Corporation in the United States and/orother countries.

Adobe® and Acrobat® are trademarks of Adobe Systems Incorporated.

Java® and Oracle® are registered trademarks of Oracle and/or its affiliates.

UNIX® is a registered trademark of The Open Group.

All third-party marks are property of their respective owners.

Publication Date

Thursday October 21, 2021 16:47:53

Document ID

dhg1623281946928

Support

All documentation and knowledge base articles are available on HPE InfoSight at https://infosight.hpe.com. To register forHPE InfoSight, click the Create Account link on the main page.

Email: [email protected]

For all other general support contact information, go to https://www.hpe.com/us/en/services/nimble-storage.html.

Legal Notices ii

https://infosight.hpe.com

mailto:[email protected]

https://www.hpe.com/us/en/services/nimble-storage.html

Contents

NimbleOS 5.2.1.800..........................................................................................................................4Important Update Note............................................................................................................................................................................................................................4

Special Notes..................................................................................................................................................................................................................................................4

New Features in 5.2.1.800.....................................................................................................................................................................................................................8

Recent Release Features.........................................................................................................................................................................................................................8

Documentation..............................................................................................................................................................................................................................................9

Verified Update Paths............................................................................................................................................................................................................................11

Known Critical Issues..............................................................................................................................................................................................................................14

Resolved Critical Issues.........................................................................................................................................................................................................................28

Resolved Issues..........................................................................................................................................................................................................................................30

Known Issues...............................................................................................................................................................................................................................................30

Contents 3

NimbleOS 5.2.1.800

5.2.1.800Version:

Thursday October 21, 2021 16:47:53Revision:

The release notes describe the major changes, fixes, and known issues for this release of the NimbleOS. They do not includeall individual fixes and internal changes.

For technical support, contact HPE Nimble Storage Support at:

mailto:[email protected] (877-364-6253), option 2.

Important Update Note

Updating NimbleOS can involve an update to component firmware on the standby controller. This can cause an email alertand automated case indicating "Standby Controller Not Available" when the firmware update process takes longer than fiveminutes. This is expected behavior and does not affect data services. At the end of the software update, you can check statusof both controllers in the Web UI under Manage > Hardware. One controller will be ACTIVE and the other STANDBY undernormal operating conditions following a successful software update.

All third-party software notices can be found on HPE InfoSight (https://infosight.hpe.com) on the Resources > Documentationpage:

https://infosight.hpe.com/resources/nimble/docs

The Documentation page also includes the General Terms and Conditions document. You can display this document byperforming the following steps:

1 In the navigation pane on the HPE InfoSight Documentation page, scroll through the Document Type list and selectSupport Policy.

2 In the page that appears, select General Terms and Conditions. This document opens in a browser tab.

Special Notes

DescriptionNote

HPE Nimble Storage continues to qualify configurations between releases. TheValidated Configuration Matrix provides information about validated configurationsand is updated frequently. It is a good practice to check your system configurationagainst this online tool. The Validated Configuration Matrix tool is available onHPE InfoSight:

https://infosight.hpe.com/resources/nimble/validated-configuration-matrix

Critical:

NimbleOS 5.2.1.800 4

mailto:[email protected]




DescriptionNote

HPE Alletra 6000 and HPE Nimble Storage systems are built with a focus on dataintegrity. HPE has improved the heuristics of drive failure handling to furthersafeguard customer data in extraordinary situations where multiple drives fail ina short period of time. Starting with release 5.2.1.800 , if three drives in a RAID seton an All Flash array fail without time to rebuild, the array shuts down.

Prior to the 5.2.1.800 release, HPE Nimble Storage All Flash systems could experi-ence three simultaneous drive failures and keep functioning while rebuilding parity.Statistically, HPE’s AI data has shown that when a system has three drive failuresin rapid succession, it has a high likelihood of more failures because this oftenmeans the failures are not random, but correlated. As a result, HPE has decided toshut the All Flash Array down after three drive failures in the same RAID group (ormore specifically, if there is no parity left in system).

By changing the behavior of All Flash Arrays in release5.2.1.800, HPE reduces therare risk of data loss in the event that four disks drives fail in rapid succession.

Note: The behavior of Hybrid Flash arrays is unchanged. Hybrid FlashArrays currently shut down if a third drive in a RAID set fails.

Critical:

Arrays must be running NimbleOS 5.0.4.0 or later to update directly to NimbleOS5.2.1.800.

Critical:

An extended data services outage may occur with MS iSCSI initiator and Intel NICsusing the built-in Windows driver e1q60x64.sys (version 11.0.5.21/11.0.5.22).

If you encounter this problem, please update your system to use the latest Windowsdriver.

Critical:

A service outage may occur on Windows 2012 R2 hosts using Emulex or BroadcomFibre Channel HBAs with firmware/driver prior to 11.2. Update the Emulex orBroadcom firmware/driver to 11.2 or later

Critical:

Due to a known Red Hat Enterprise Linux bug 1002727, while running virtualizedin VMware ESX, manually rebooting the active controller in presence of heavy IOsusing the reboot --controller command on a Fibre Channel array may trigger anincorrect retry initiated by RHEL guests running the following kernel versions:

• 6.4 and earlier• 6.5 without the patch• 7.0 without the patch

This incorrect retry logic may lead to unexpected application behavior. In theseenvironments, we recommend the failover command instead.

Critical:

Due to a known Red Hat Enterprise Linux bug 3550561, unexpected applicationbehavior may occur on RHEL 7.5 hosts with kernel-3.10.0-862.3.2.el7 or derivativesusing Emulex FC FCoE HBAs (lpfc driver) and raw devices. To avoid this issue:

• If running RHEL 7.6, update to kernel-3.10.0-957.el7 or later.• If running RHEL 7.5z, update to kernel-3.10.0-862.25.3.el7 or later.

Critical:


DescriptionNote

As outlined in the current Validated Configuration Matrix, HPE Nimble Storagefully supports Windows guest operating systems on Microsoft Hyper-V, includingVirtual Fibre Channel (VFC) connectivity and multipathing with HPE Nimble StorageDSM and VSS support. However, Linux guest operating systems running in Hyper-V VFC configurations are not qualified.

Running Red Hat Linux guest operating systems with the “Linux Integration Ser-vices” kit installed, or with hv_storvsc drivers in such configurations can lead toRed Hat bug 1364282, which can cause an unexpected service outage.

Critical:

Starting with NimbleOS 5.1.1.0, the size of the software package now exceeds 2GB, which may lead to lengthier software download times. Previously, the sizes ofthe NimbleOS 5.0.x download packages were approximately 1.6 GB, and NimbleOS4.x packages were approximately 900 MB.

Important:

After completing the NimbleOS update for array groups configured for SynchronousReplication, download the corresponding version of the Synchronous ReplicationWitness software, and update the witness host.

Important:

Microsoft Offload Data Transfer (ODX) is not supported if the destination volumehas synchronous replication enabled.

Important:

As of vSphere 7.0, VMware has discontinued the flex client. Consequently, the HPENimble Storage vCenter Plugin no longer supports the flex plugin for vCenter 7.0.

Important:

You can enable deduplication for CS1000, CS3000, CS5000, CS7000, CS700, andCS500 arrays on a volume only if the corresponding storage pool has a Flash toDisk Ratio (FDR) greater than 4%. To calculate the FDR, obtain the "Total arraycapacity (MiB)" and "Total array cache capacity (MiB)" values by using the HPENimble Storage CLI command pool_name. This command returns the Pool capacity(MiB), which is the "Total array capacity (MiB)", and the Pool cache capacity (MiB),which is the "Total array cache capacity (MIB)".

Then perform the following calculation:

FDR = "Total array cache capacity (MiB)"/"Total array capacity (MiB)" * 100

If the array has sufficient capability for deduplication, the pool --info commandwill also show a value for dedupe capacity (MiB).

Note: On the HF20H, HF20, HF40, and HF60 platforms, pool --info dis-plays "N/A" as the value for dedupe capacity (MiB). This because youcan enable deduplication for the entire array.

Important:

For connections to the NimbleOS GUI, you must have port 5392 open for the GroupManagement IP address and both diagnostic IP addresses.

Important:

Numerous host integration toolkits are supported in NimbleOS 5.2.1.800. It isstrongly recommended that they be installed on all Windows, Linux, and VMwarehosts. For more information about supported toolkits, refer to the Validated Con-figuration Matrix, which is available on HPE Nimble Storage InfoSight:


Important:


https://infosight.hpe.com/org/urn%3Ainfosight%3A00000000-0000-0000-0000-000000000001/resources/nimble/software/NimbleOS/Synchronous%20Replication%20Witness

https://infosight.hpe.com/org/urn%3Ainfosight%3A00000000-0000-0000-0000-000000000001/resources/nimble/software/NimbleOS/Synchronous%20Replication%20Witness


DescriptionNote

HPE Nimble Storage recommends that you update to HPE Nimble Storage WindowsToolkit (NWT) 7.0.1 or later if you are using Microsoft VSS Synchronization andNimbleOS 5.1.4.200 or later.

Using application consistent snapshots with earlier versions of NWT and NimbleOS5.1.4.100 may result in the following error messages:

• In the host's VSS requestor log (C:\ProgramData\Nimble Storage\Logs\VssRe-questor.log):

PID:1996 TID:5752 ERR reqcommon. cpp:683 RequestStatus=QueryStatus(),Function=pAsync->QueryStatus(), Error=VSS_E_PROVIDER_VETO, rc=Syste-mError, ca=ContactSupport

• In the Windows event viewer:

event id 4100: EndPrepareSnapshots method: failed to find LUN s/n <SERI-AL_NUMBER> on connected arrays. Make sure that the Nimble array versionis compatible with this version of Nimble Windows Toolkit.

event id 4170: Nimble VSS provider is not compatible with the current versionof the Nimble array software(). Install appropriate version of the Nimble VSSprovider.

NWT 7.0.1 resolves this issue.

Important:

HPE Storage Connection Manager for VMware 7.0 is signed by VMware for ESXi7.x. It can be installed through the VMware Update Manager or esxcli commandwithout the --no-sig-check flag.

See the Connection Manager for VMware Release Notes 7.0 or later and the latestVMware Integration Guide for further details.

To locate the latest version of the guide, log in to HPE InfoSight. Choose Re-sources > HPE Alletra, Nimble Storage Documentation. In the left pane, clickIntegration Guide, then click Connection Manager for VMware. From the listdisplayed, choose the version of the guide that you want.

Important:


DescriptionNote

Various timeout values affect HPE Storage targets from Windows/Linux hosts.Before you update the NimbleOS, install the HPE Storage Toolkit for Windows orHPE Storage Linux Toolkit (NLT) on the host or tune the timeout values. Timeoutdetails for various operating systems can be found on HPE InfoSight under Re-sources > Documentation. From the HPE Nimble Storage Documentation page,locate the article you want.

The following Knowledge Base articles and Integration Guides explain how toconfigure and verify host timeout settings for the major supported operating sys-tems (OS):

• For Windows, refer to https://infosight.hpe.com/InfoSight/media/cms/ac-tive/sup_KB-000052_Windows_Host_Disk_Timeout_Values.whz.

In the context of Microsoft Windows, the following article should also be con-sidered:

https://infosight.hpe.com/InfoSight/media/cms/active/sup_KB-000246_MPIO_Timeout_Parameters_For_MSDSM_And_NimbleDSM_In_Win-dows_2012_R2doc_version_family.whz

• For VMware, refer to the Common Tasks and Best Practices > Host TimeoutValues section of the VMware Integration Guide.

• For Linux, refer to https://infosight.hpe.com/InfoSight/media/cms/ac-tive/sup_KB-000304_Linux_Host_Disk_Timeout_Values.whz.

Important:

vVol VMs cannot be claimed after deleted from the downstream array.

A vVol VM can be protected and may be subsequently replicated to a downstreamarray (as configured in the storage policy). In the case where this vVol VM isdeleted, a supported “claim” workflow allows us to claim this vVol VM on thedownstream array. This workflow is not supported at present if performed on asetup where the vCenter version is 6.5 or above due to validation failures on thevCenter.

VMware DCPN Ticket Reference:

https://dcpn.force.com/TechnicalRequestCaseRedesignPart-ner?Id=5000H00001JRKhf

Important:

New Features in 5.2.1.800

No new features were introduced in NimbleOS 5.2.1.800.

Recent Release Features

The following features were released in NimbleOS 5.2.1.x:

Fan-Out Replication

You may now use volume snapshot replication to replicate to two destinations simultaneously.

HPE Cybersecurity – Signed Updates

NimbleOS Releases are now digitally signed by HPE. Code signing ensures the authenticity of the provider (it is HPE) andthe integrity of the software download.

Fibre Channel Target Driven Zoning



https://infosight.hpe.com/InfoSight/media/cms/active/sup_KB-000052_Windows_Host_Disk_Timeout_Values.whz

https://infosight.hpe.com/InfoSight/media/cms/active/sup_KB-000052_Windows_Host_Disk_Timeout_Values.whz

https://infosight.hpe.com/InfoSight/media/cms/active/sup_KB-000246_MPIO_Timeout_Parameters_For_MSDSM_And_NimbleDSM_In_Windows_2012_R2doc_version_family.whz



https://infosight.hpe.com/InfoSight/media/cms/active/sup_KB-000304_Linux_Host_Disk_Timeout_Values.whz

https://infosight.hpe.com/InfoSight/media/cms/active/sup_KB-000304_Linux_Host_Disk_Timeout_Values.whz

HPE Nimble Storage arrays are now able to program the zones in the Fibre Channel (FC) fabric using information from theinitiator groups that have been configured. This removes the requirement for the administrator to program the FC zonesusing separate fabric management tools.

Array Upgrade for AFxxxx/CSxxxx to AFxx/HFxx (Offline)

HPE Nimble Storage now supports data-in-place upgrades from the previous generation of arrays to the currently shippingarrays. This version of the upgrade process requires a brief down time while the existing array chassis is replace with the newone, and the media is moved from the older array to the new array.

Support for 10,000 Volumes on AF40 Arrays

The limit on the number of volumes supported by an HPE Nimble Storage AF40 model array is now 10,000, up from 1,000volumes in previous NimbleOS releases.

Storage Class Memory

NimbleOS now supports new 1.5 TB storage class memory cards. Support is limited to HPE Nimble Storage AF60 and AF80model arrays.

Synchronous Replication: Witness OVA

The Peer Persistence feature requires an external witness. The Witness is available for download from InfoSight as a virtualmachine packaged as an OVA.

dHCI Unified Update

HPE Nimble Storage dHCI now provides an Update tab in the HPE Nimble Storage vCenter Plugin that allows you to performan automatic update when there is a new version of NimbleOS, ESXi, or HPE Nimble Storage Connection Manager for VMware.

dHCI Server Configuration Limits

The limit on the number of servers supported in a dHCI cluster has increased to 32.

dHCI Support for Intel and AMD Processors

dHCI adds support for ProLiant servers using AMD processors. It continues to maintain support for Intel-based ProLiantservers. The Validated Configuration Matrix provides information about which server models are supported.

Note: You can use either Intel-based ProLiant servers or AMD-based ProLiant servers in your dHCI configuration.You cannot use both in the same cluster.

Documentation

These Release Notes and other user documentation are available on HPE InfoSight:


You can manually reach the documentation page by logging onto HPE InfoSight and selecting Resources > Nimble Storage >Documentation.

Document Search Interface

There are several methods you can use to locate the documents you need.

The Nimble Storage Documentation page provides a search interface that allows you to search for information across alldocumentation, including support and knowledge base articles, best practices, solutions and integration guides, productdocumentation, and configuration matrices.

To go directly to a document, use the navigation pane on the left side of the Nimble Storage Documentation page. Thenavigation pane organizes documents into categories, including:

• Document Type• Nimble Software and Solutions• Software Version• Integration• Platform




You can use the page scroll bar to move up and down the navigation pane.

Third-Party Software Notices

All third-part software notices can be found in the Documentation Portal on HPE InfoSight.

Here are the steps to manually access the third-party software notices.

1 Log in to HPE InfoSight (https://infosight.hpe.com) .2 From the menu, select Resources > Nimble > Documentation.3 In the left navigation pane of the Documentation Portal, scroll through the Document Type section and select Support

Policy.4 From the list of documents, select General Terms and Conditions.

The document opens in a new browser tab.

Core User Documentation

The following is the core user documentation for NimbleOS:

• GUI Administration Guide• CLI Administration Guide• SNMP Reference• Command Reference• REST API Reference

If you are using an HPE Nimble Storage dHCI-enabled array, you should also check the dHCI Deployment Guides and GettingStarted Guide.

Workflow Documents

There are several workflow guides that contain procedures you can perform using either the CLI or the GUI. Each workflowguide covers a specific, frequently performed task related to HPE Nimble Storage products. Each task described by a workflowdocument is explained in detail in the GUI Administration Guide and the CLI Administration Guide.

Hardware

Documentation for all hardware components is available on HPE InfoSight. Click the Hardware Guide link in the DocumentType category. Hardware documentation includes array and expansion shelf installation quick start guides, installation,upgrade, and replacement guides, and comprehensive hardware guides.

Host Integration Guides

Host Integration Guides are available from HPE InfoSight. To locate these documents on the HPE InfoSight Documentationpage, scroll down the navigation pane to the section called Integration Guide.

Note: A single Host Integration Guide supports multiple version of NimbleOS and the companion Integration Toolkitsoftware packages. The version number listed on the guide might be different from the version numbers of theNimbleOS and Toolkit software packages that it supports.

NimbleOS 5.2.1.800 10


Verified Update Paths

Table 1: From Versions 5.x

From Versions 5.x

To VersionFrom Version

5.2.1.8005.2.1.700

5.2.1.8005.2.1.600

5.2.1.8005.2.1.500

5.2.1.8005.2.1.400

5.2.1.8005.2.1.300

5.2.1.8005.2.1.200

5.2.1.8005.2.1.100

5.2.1.8005.2.1.0

5.2.1.8005.1.4.200

5.2.1.8005.1.4.100

5.2.1.8005.1.4.0

5.2.1.8005.1.3.100

5.2.1.8005.1.3.0

5.2.1.8005.1.2.100

5.2.1.8005.1.2.0

5.2.1.8005.1.1.0

5.2.1.8005.0.10.100

5.2.1.8005.0.10.0

5.2.1.8005.0.9.100

5.2.1.8005.0.9.0

5.2.1.8005.0.8.100

5.2.1.8005.0.8.0

5.2.1.8005.0.7.300

5.2.1.8005.0.7.200

5.2.1.8005.0.7.100

5.2.1.8005.0.7.0

5.2.1.8005.0.6.0

5.2.1.8005.0.5.200

5.2.1.8005.0.5.0

5.2.1.8005.0.4.0

NimbleOS 5.2.1.800 11

From Versions 5.x


5.0.10.05.0.3.100

5.0.10.05.0.3.0

5.0.10.05.0.2.0

5.0.10.05.0.1.100

5.0.10.05.0.1.0


From Versions 4.x


5.0.10.04.5.6.0

5.0.10.04.5.5.0

5.0.10.04.5.4.0

5.0.10.04.5.3.0

5.0.10.04.5.2.0

5.0.10.04.5.1.0

5.0.10.04.5.0.0

5.0.10.04.4.1.0

5.0.10.04.4.0.0

5.0.10.04.3.1.0

5.0.10.04.3.0.0

5.0.10.04.2.1.0

5.0.10.04.2.0.0

5.0.10.04.1.0.0


From 3.x Versions


5.0.10.03.9.3.0

5.0.10.03.9.2.0

5.0.10.03.9.1.0

5.0.10.03.9.0.0

5.0.10.03.8.1.0

5.0.10.03.8.0.0

5.0.10.03.7.0.0

NimbleOS 5.2.1.800 12

From 3.x Versions


5.0.10.03.6.2.0

5.0.10.03.6.1.0

5.0.10.03.6.0.0

5.0.10.03.5.4.0

5.0.10.03.5.3.0

5.0.10.03.5.2.0

5.0.10.03.5.0.0

5.0.10.03.4.1.0

5.0.10.03.4.0.0

5.0.10.03.3.0.0

5.0.10.03.2.1.0

5.0.10.03.1.0.0


From 2.0.x VersionsFrom 2.1.x VersionsFrom 2.2.x, 2.3.x Versions

To VersionFrom VersionTo VersionFrom VersionTo VersionFrom Version

2.1.9.12.0.8.02.3.18.02.1.9.14.5.6.02.3.18.0

2.1.9.12.0.7.02.3.18.02.1.9.04.5.6.02.3.16.0

2.1.9.12.0.6.*2.3.18.02.1.8.04.5.6.02.3.15.0

2.1.9.12.0.5.02.2.9.02.1.7.04.5.6.02.3.14.0

2.1.9.12.0.4.02.2.9.02.1.6.04.5.6.02.3.12.*

2.2.9.02.1.5.04.5.6.02.3.9.*

2.2.9.02.1.4.04.5.6.02.3.8.0

2.2.9.02.1.3.04.5.6.02.3.7.0

2.2.9.02.1.2.04.5.6.02.3.6.0

2.1.9.12.1.1.04.5.6.02.3.4.0

2.1.9.12.1.0.04.5.6.02.3.3.0

4.5.6.02.3.2.1

4.5.6.02.3.2.0

4.5.6.02.3.1.0

3.9.3.02.2.11.0

3.9.3.02.2.10.0

3.9.3.02.2.9.0

3.9.3.02.2.7.*

NimbleOS 5.2.1.800 13

From 2.0.x VersionsFrom 2.1.x VersionsFrom 2.2.x, 2.3.x Versions


3.9.3.02.2.6.0

3.9.3.02.2.5.*

2.2.11.02.2.3.*

2.2.11.02.2.2.0

2.2.11.02.2.1.0

2.2.11.02.2.0.0


From 1.0.x VersionsFrom 1.3, 1.2, 1.1 VersionsFrom 1.4.x Versions


Contact Support1.0.7.*1.4.6.01.3.*.*2.1.9.11.4.12.0

Contact Support1.0.6.*1.4.6.01.2.*.*2.1.9.11.4.11.0

1.2.2.01.1.*.*2.1.9.11.4.10.0

2.1.9.11.4.9.0

2.1.9.11.4.8.0

2.1.9.11.4.7.0

1.4.12.01.4.*.*

Known Critical Issues

Known Critical Issues in Release 5.2.1.800

WorkaroundDescriptionTitleComponentID

Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Console

On Data Service startup followedby a failover, during the recov-ery process, it is possible to hita checksum mismatch. The DataService restarts when the mis-match is encountered. The ser-vice will make three attempts torestart, before disabling the ser-vice.

Data Service dis-abled on array dueto NVRAM Check-sum mismatch

Data ServiceAS-115397

Not applicableDuring disk index processing theautomated task manager reportsstatus to the Data Service. If thetask manager does not respondwithin the expected time of 30seconds, the Data Service willrestart to recover from the con-dition.

Data Service mayrestart unexpected-ly due to healthcheck failure


NimbleOS 5.2.1.800 14




On hybrid arrays, the flash cachecheckpoint task is run periodical-ly to reduce memory/NVRAMconsumption by the cachebuffers. This task is expected tocompleted within a specifiedamount of time. In rare scenar-ios, this task can be blocked andfail to complete in time. Theservice will restart to recoverfrom the condition.

Data services mayrestart unexpected-ly due to BlockUnique Storagetask timeout.


Not applicableThe data services daemon peri-odically synchronizes its persis-tent in-memory (NVRAM)structures with its other objectson flash. This process is requiredto complete within a reasonableamount of time. Under certainrare conditions, this process canget delayed and trigger a restartof the data services daemon.This can cause a momentarydelay in IO which the daemonrestarts. There should not beany disruption of IO, since thedaemon is designed to restartwithin any IO timeout limits.

Data service mayrestart unexpected-ly


Not applicableDuring internal deduplicationprocessing, in invalid memoryreference can cause a segmenta-tion fault, leading to restart ofData Service to recover from thecondition.

Data Service mayrestart due to in-valid memory refer-ence during dedupli-cation


Not applicableDuring the Fingerprint Index orDisk Index merge, updates aregenerated in Cache Index. TheData service will start a transac-tion for the merge and wait for1000 such updates to accumu-late before committing thempersistently. Due to certainworkloads, the wait may be along time before 1000 updatesare accumulated. The servicemay restart to recover from thiscondition.

Data Service mayrestart due to longtransaction waittime


NimbleOS 5.2.1.800 15



When planning to remove amember array from group,schedule a planned maintenancewindow and place all ESX hostsinto maintenance mode to mini-mize impact to availability. ESXtypically resumes connection tovVol datastores, and reconnectsto VMs, after a period of 15-30minutes automatically without amanual intervention.

Scaled vVol environments with500 vVol VDI VMs or more than5000 Nimble vVol volumes mayexperience IO disruption whenremoving a member array fromgroup. Symptom of problemwould appear as vVol datastoresbeing (inaccessible). Virtual Ma-chine status would also appearas (inaccessible).

Removing memberarray from multi-ar-ray group maycause IO disruptionto scaled vVol envi-ronments



When the Data Service detectsa metadata inconsistency, theservice may restart repeatedlyand hosts could experience un-expected application behavior.

Data Servicerestarts when de-tecting metadatainconsistency


Not applicableThe Data service may restartunexpectedly when a checksumverification failure occurs.



Contact HPE Nimble StorageSupport to identify and replacethe faulty SSD.

If the array encounters a faultySSD with multiple bad segments,the Log-structured File System(LFS) attempts to resolve theerrors. If the array is under highload and LFS has a high humberof block errors to resolve it mayexhaust the IORW buffers caus-ing the Data Service to restartrepeatedly.

Data service mayrestart repeatedlydue to faulty SSD


Not applicableData service may restart due arare race condition between theoperations of shutting down thedata service and resync of thestandby controller. When therace condition is detected theData service restarts to recoverfrom the condition.

Data service mayrestart due to arace condition


Not applicableWhile committing internaltransactions, Data Service mayhit a rare race condition. To re-cover from this Data Servicemight restart



Not applicableDuring snapshot replication ofa dedupe-enabled volume, thedownstream array file systemmay restart due to an out-of-memory condition.

Snapshot replica-tion of deduplica-tion-enabled vol-umes may lead toFile System restart


NimbleOS 5.2.1.800 16



Not applicableA disruption in network connec-tions can cause Data Service torestart unexpectedly.

Data Service mayunexpectedlyrestart


Not applicableIn environments with frequentnetwork checksum errors, inter-ruptions of Replication and DataService restarts may occur. Net-work errors that are not caughtby ethernet or TCP/IP check-sums can lead to an unexpectedfilesystem restart.

Data Service mayrestart due to repli-cation network er-rors


Review network devices toidentify and reduce networkchecksum errors.

In environments with frequentnetwork checksum errors, inter-ruptions of Replication and DataService restarts may occur.

Replication interrup-tions and Data Ser-vice restarts mayoccur due to net-work errors



In rare instances, the Data Ser-vice may restart unexpectedlywhile performing Volume Man-agement checksum computa-tions.

Data Service mayrestart unexpected-ly


Not applicableWhen attempting to perform apool merge operation, if thereare a large number of volumesthat must be striped across thepool, and one of the arrays hasa large number of pendingdeletes, then it is possible for theoperation to fail due to the DataService being overloaded.Symptoms of this behavior areif the pool merge operationhangs for several minutes andreturns the following message:The request could not be under-stood by the server.

Pool merge may faildue to too manypending deletes


Not applicable, on restart DataService would behave normally.

Generation delete operationsand NVRAM to disk data flushoperation can cause Data Ser-vice to restart due to healthcheck failure, as it can holdcheckpoint for a long time.

Data Service mayrestart due to vol-ume managerhealth check failure


NimbleOS 5.2.1.800 17



Not applicableIf the Data Service is restartedduring a volume move, it wouldresult in incorrect Bin Migrationstate to be recovered, causinganother Data service restart.This is a rare event since it re-quires snapshots to be taken aswell as a Data Service restartduring a volume move.

Data Service restartduring volumemove may causethe service torestart again.


Not applicableThe Data service may restartwhen Bin Migration is going on.This can happen when followingactivities happen together: 1. Binmigration is occurring for a vol-ume. 2. Group Management ser-vice restart (because of any rea-son). 3. Group Management ser-vice unable to re-sync with Dataservice after restart. This canlead to Data service restart butit will not impact bin migrationas after Data service restart binmigration will resumed automat-ically.

Rare race conditionbetween Data ser-vice and GroupManagement ser-vice cause Dataservice restart



Counters used for space account-ing may cause validation to fail.The array may see a blip inspace usage because of whichwrites may be denied and inworst case, volume may go of-fline.

Writes may be de-nied due to coun-ters in space ac-counting


Not applicableController may reboot unexpect-edly if there is high memory uti-lization for the java and/or jettyprocesses on the array.

Controller may un-expectedly restartdue to high memo-ry utilization

Host IntegrationAS-84499


HPE Nimble Storage has identi-fied a rare firmware defect in asubset of drives which can, un-der certain write intensiveworkloads, cause the array tounder-perform.

Performance affect-ing firmware defectin a subset of 6TBdrives.

PlatformAS-94961

NimbleOS 5.2.1.800 18



If after 20 minutes the controllersensors do not report goodstate, please contact HPE Nim-ble Support, supporting HPE Al-letra 6000 and Data ServicesCloud Console for assistance.

During boot up due to a knownIntel defect the controller sen-sors may report missing for aperiod of time in the array alerts.After about 15-20 minutes, itreturns to a valid state and thesensors should report validreadings again.

Controller sensorsmissing forAFxx/HFxx arrays

PlatformAS-86764

Not applicableThe Data Service may restartunexpectedly when array en-counters a disk returning baddata, and the disk has not beenmarked as failed.

Data Service mayrestart unexpected-ly when a disk re-turns bad data

PlatformAS-97925

Not applicableDuring internal file operations,processes may be waiting for alock to be released. If the waittime exceeds 30 seconds, a ser-vice health check may restartthe Data service to recover.

Data service mayrestart during whenfile operation time-out is exceeded

PlatformAS-86099

Not applicableIn rare instances, array failovermay occur when an IO error isencountered due to unexpectedmemory modification.

Array failover mayoccur unexpectedlyafter an IO error.

PlatformAS-127256


If the array M.2 boot device isfaulty, the array software updateprecheck and/or Diagnostics forNimble Array uploads to HPENimble Support may fail. Thismay prevent issues from beingproactively identified by HPENimble Support.

Array updateprecheck or Diag-nostics for NimbleArray uploads mayfail due to faultyboot device

PlatformAS-94153

Not applicableThe Data Service on the arraymay restart when timeout forinternal communication betweenarray controllers has been ex-ceeded. The service restarts torestore the communication.

Data Service mayrestart due tohealth check failure

PlatformAS-104517

Contact HPE Nimble StorageSupport to review disks for re-placement.

In rare instances, the Data Ser-vice may restart if the array hasmultiple bad drives which makeIO handling very slow.

Data Service mayrestart if the arrayhas multiple baddrives

PlatformAS-108793

Not applicableUnder rare circumstances, alarge number of hotplug eventscan lead to incorrect memorytracking, resulting in a false out-of-memory error and a crash ofthe controller.

Controller may re-boot unexpectedlydue to memorystarvation eventhough there is freememory

PlatformAS-129166

NimbleOS 5.2.1.800 19



Perform an array failover orcontact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Console

The Array Management Servicemay restart on the array andbecome unable to start due to alock held by the Data Service. Arestart of the Data Service or anarray failover will resolve the is-sue by restarting the service torelease the lock.

Array ManagementService may restartand become unableto start

PlatformAS-118815

Not applicableThe current thermal policy hasa limitation where individualPCIe components are not moni-tored on card-by-card basis. Asa result, the high level tempera-ture policy that is current imple-mented, is sometimes incapableof regulating the temperaturesof individual cards.

Limited thermalmonitoring policyfor PCIe compo-nents

PlatformAS-107489

Verify that the software updatepackage has been downloadedto the system and then attemptthe software update again usingthe software --resume_updatecommand from the CLI.

During software update fromNimbleOS 3.x versions to laterversion, a failure to extract thesoftware update package mayincorrectly return the errormessage /tmp is out of space.even if there is still space in thedirectory.

Incorrect softwareupdate error mes-sage /tmp is out ofspace.

PlatformAS-93456

In order to add more 4 or moreProliant Servers, the workaroundis to add up to 3 servers at atime from the plugin.

Currently, if customer plans toadd 4 or more Proliants serversin their dHCI deployment via theplugin, the operation fails.

Plugin: Cannot add4 or more serversin the dHCI deploy-ment

dHCIAS-104924

Not applicableThe Data Service on the arraymay restart if a Task Manage-ment Function request to aborta command is received while thecommand is still being complet-ed. The service restarts to recov-er from the condition.


SANAS-112859

Not applicableNormally Fibre Channel sessionlogin/logout happens veryquickly. Under certain condi-tions, it may take long enoughthat SCSI commands will timeoutleading and lead to brief dataunavailable situations.

IO may timeoutduring Fibre Chan-nel session login/lo-gout

SANAS-93191

NimbleOS 5.2.1.800 20



Review the Fibre Channel fabricfor any issues that may causelost frames.

In rare occasions during periodswhere a large number of FibreChannel commands are abortedand sessions are terminating,the Data Service may occur be-cause of inconsistent or dupli-cate reporting of command ter-mination status. The initial trig-ger of this issue is typically lostframes on the fabric due to con-gestion or hardware failure.

Data Service mayrestart due to FibreChannel commandstate inconsistency

SANAS-91812

Not applicableWhen an implicit logout of a Fi-bre Channel session occurs SCSIcommands from the previoussession may not be terminatedproperly causing a restart of thedata service. The service restartsto recover from the condition.

Data Service mayrestart after implicitlogout of FibreChannel Session

SANAS-94746

Not applicableIn rare instances, the controllerkernel may crash if the outstand-ing SCSI command arent proper-ly cleaned up in the Fibre Chan-nel driver during session termi-nation. The kernel crash willcause the controller to reboot torecover from the condition.

Controller may re-boot unexpectedlydue to impropercommand cleanupduring Fibre Chan-nel session termina-tion

SANAS-94636

Addressing the fabric issuesshould resolve this issue.

Fibre Channel fabric issues (er-rors, congestion, etc.) can pre-vent the array from transferringdata and status to the initiator.This can result in a large amountof aborted commands. In rarecases this may cause problemsfor the Fibre Channel driver andSCSI stack.

Initiator side FibreChannel Fabric in-stability may resultin a large numberof aborted SCSIcommands

SANAS-95159

Contact HPE Nimble StorageSupport

When host sends a SCSI writecommand with an invalid offset(the offset + write size >UINT64), the data service mayrestart if the write command hasimmediate data.

Data Service mayrestart when en-countering SCSIwrite commandwith invalid offsetand immediate data

SANAS-118808

Contact HPE Nimble StorageSupport.

The Event service may restartunexpectedly when attemptingto load a trusted certificate dueto an issue which impacts certifi-cate management. The servicemay not restart after encounter-ing the issue.

Event service mayrestart unexpected-ly

SecurityAS-117550

NimbleOS 5.2.1.800 21




The Event service may restartunexpectedly when attemptingto load a trusted certificate dueto an issue which impacts certifi-cate management. The servicemay not restart after encounter-ing the issue.


SecurityAS-115780

Not applicableGroup Management service mayrestart due to a race conditionbetween threads when a volumeaccess control list is removed.The service restarts to recoverfrom the condition.

Group Managementservice may restartdue to race condi-tion

System Manage-ment

AS-110030

The Group Management servicewill recover after the crash. Theworkaround would be to nolonger use an excessively longusername, and reduce the user-name to under 255 characters.

An excessively long usernameof over 255 characters will ex-ceed a character limit within thearray groups auditing frame-work. This would cause theGroup Management service torestart unexpectedly.

Group ManagementService restarts dueto excessively longusername

System Manage-ment

AS-82919

Not applicableAudit log entries exceeding thelimit of 24000, would initiate acleanup of older entries 500 ata time. This could cause thegroup management service torestart unexpectedly duringprocessing of the cleanup.

Group ManagementService may restartduring audit logclean up

System Manage-ment

AS-116981

Arrays must be running thesame version of NimbleOS be-fore attempting a group mergeoperation.

The Group Management servicemay restart during group mergeoperations if the arrays beingmerged are not using the sameversion of NimbleOS. The servicerestarts to recover from thecondition.

Group Managementservice may restartduring groupmerge

System Manage-ment

AS-92134

Not applicableIn some cases, when the BackupGroup Leader is transitioning toa member array role, the ArrayManagement Service mayrestart unexpectedly. This oc-curs if the array experiences atimeout when the Array Manage-ment Service is attempting tostop the Backup Group Leaderprocesses. The restart of theservice recovers from the condi-tion.

Array ManagementService restarts dueto health checktimeout during Au-tomatic Failover

System Manage-ment

AS-110455

NimbleOS 5.2.1.800 22



Please contact HPE NimbleStorage Support

The Automatic Switchover Ser-vice may restart unexpectedlywhen the witness is removed orAutomatic Switchover is dis-abled on an array group. In rareinstances, this may lead to anissue with a database entrywithin the array is not clearedsuccessfully. If this array resetupis attempted on this array, theoperation will fail when the Ar-ray Management Service encoun-ters this stale entry.

Array resetup failsdue to previouscomplications withArray SwitchoverService

System Manage-ment

AS-107367

Not applicableThe Automatic Switchover Ser-vice internally creates and closesthreads each time during Auto-matic Failover (AFO) quorumsetup and tear down. This maycause the service to eventuallycrash after reaching the maxi-mum thread limit. The systemrecovers automatically when theAutomatic Switchover Servicerestarts.

AutomaticSwitchover Servicerestarts due tothread limitations

System Manage-ment

AS-89701

This restart is non-disruptive tothe data on the array, and theArray Management recovers af-ter the restart occurs.

The arrays database system maybecome unavailable for a limitedtime when there is a failure insetting up the Backup GroupLeader. When attempting todiscover a new Backup GroupLeader, the Array ManagementService may restart due to a racecondition.

Array ManagementService restartsduring BackupGroup Leader dis-covery

System Manage-ment

AS-98953

A Manual Group Leader Failoverwill be required to restore FibreChannel connectivity to thehosts.

An Automatic Failover (AFO) ofthe Group Management Serviceswill not be initiated if all FibreChannel (FC) interfaces on theGroup Leader array fail on bothcontrollers.

No AutomaticFailover in theevent the host los-es all FC connectivi-ty to an array

System Manage-ment

AS-94737

A failover can be initiated in or-der to restart the Group Manage-ment Service. You may alsoContact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consoleto restart the service manually.

When performing a controllerupgrade to a high-end model,the object limits will still showthe lower limits if the GroupManagement Service is notrestarted.

Group ManagementService must berestarted to unlockadditional volumelimits after con-troller upgrade

System Manage-ment

AS-65615

NimbleOS 5.2.1.800 23



Not applicableThe Array Management servicewill query the IP Monitoring ser-vice (IMS) for group leader andbackup group leader IP informa-tion to update the informationin the ScaleOut Database. If IMSreturns does not return valid IPinformation the Array Manage-ment service will restart to recov-er.

Array ManagementService may restartif IP Monitoringservice returns in-valid data

System Manage-ment

AS-106369

Remove the VLAN tagging fromthe management network toprevent network connectivityissues in Peer PersistenceGroup.

When the array group is config-ured for Peer Persistence andthe management subnet is con-figured with VLAN tagging, thecharacter limit for the interfacecreated for the Secondary Man-agement IP will be exceeded.This can cause this SecondaryManagement IP to be unreach-able.

Backup GroupLeader may experi-ence network con-nectivity issues inPeer Persistencegroup

System Manage-ment

AS-115317

Not applicableThe Array Management Servicemay be unavailable during orshortly after Automatic GroupLeader Failover events. This maycause the CLI or GUI to be un-available for a brief period oftime until the service startupcompletes.

Array ManagementService may be un-available duringAutomatic Failover

System Manage-ment

AS-116943

Please contact HPE NimbleStorage Support.

When there are network commu-nication issues between theGroup Leader and Backup GroupLeader, the system goes into anout-of-sync condition. In rarecircumstances, even after net-work connection is restored, thearray group may still remain out-of-sync.

Array group re-mains out-of-syncfollowing networkrecovery.

System Manage-ment

AS-106276

Not applicableWhen a new internal databaseconnection is established, anumber of prepared_statementsare run at the beginning. If oneof the prepared_statements failsthe Group Management Servicewill restart.

Group ManagementService may restartdue to internaldatabase handling

System Manage-ment

AS-108145

NimbleOS 5.2.1.800 24



Not applicableGroup Management service mayrestart when internal processingfails on arrays with high snap-shot activity. This may causesnapshots to fail to be deleted.

Group Managementservice may restartunexpectedly

System Manage-ment

AS-94594


In arrays with snapshot replica-tion where a volume collectionis deleted and created with thesame name, a race conditionmay occur due to a stale refer-ence for the volume collection.This can cause the Group Man-agement service to restart re-peatedly.

Group Managementservice may restartdue to race condi-tion during volumecollection deletion

System Manage-ment

AS-108902

Not applicableThe Group Management servicemay restart during an arrayshutdown while processingREST request. No user opera-tions are impacted because thearray is already in the middle ofa shutdown. The shutdown pro-ceeds normally.

Group Managementservice may restartduring array shut-down

System Manage-ment

AS-61614


The Snapshot rate limit - Num-ber of snapshots that can betaken for a given minute - is in-troduced in 4.x for distributingsnapshot load to the system. Ifa group surpasses this snapshotrate limit in previous NimbleOSreleases, the software update to4.x is still allowed. However, agroup merge is prevented if thesource or destination array sur-passes the snapshot rate limit.The following error message canbe seen during the group mergevalidation: INFO: Rate of snap-shots in the merged groupwould be greater than support-ed snapshot rate. This could oc-cur if even if the source array(the array being added to theexisting group) has 0 volumesor volume collections.

Group merge fails ifone of the groupsviolates the snap-shot rate limit

System Manage-ment

AS-69136

NimbleOS 5.2.1.800 25



Use unique name when creatingsnapshot collection.

If an attempt is made to createa snapshot collection with samethe name of an existing snap-shot collection, an error indicat-ing object already exists will bereported and the Group Manage-ment service may restart. Theservice will recover upon restart.

Group Managementservice may restartdue to duplicatesnapshot collectionname

System Manage-ment

AS-94113

Not applicableA system management processcan restart when the system isunder heavy load. The systemrecovers automatically. The Dataservice is not affected.

Group ManagementService restarts un-der heavy load

System Manage-ment

AS-100254

Not applicableAutomatic Failover demotion ofa Backup Group Leader arraymay be delayed for multipleminutes when an internaldatabase is performing weeklygarbage collection. The GroupLeader is not able to demote theBackup Group leader during aweekly window when an internaldatabase is performing internalgarbage collection. This may lastfor multiple minutes. EventuallyBackup Group Leader demotesucceeds, and a new BackupGroup Leader is selected. TheGroup loses High Availabilityduring this time and the GroupManagement service may restartto recover from the condition.

Automatic FailoverBackup GroupLeader demote maybe delayed in rarecircumstances

System Manage-ment

AS-113375

Not applicableDuring internal space usage cal-culations, a deadlock can occurbetween REST API threads. Ifthe threads do not respond toGroup Management servicewithin expected timeout, theservice restarts to recover fromthe condition.

Group Managementservice may restartdue to health checkduring space usagecalculation

System Manage-ment

AS-94485

Not applicableThe Array Management servicemay restart due to a race condi-tion encountered during serviceshutdown. The service will recov-er after the restart.

Array Managementservice restart dur-ing service shut-down

System Manage-ment

AS-98124

NimbleOS 5.2.1.800 26



Not applicableCS235 / CS300 / CS500 /CS700 arrays are blocked fromupdating to HPE Alletra 60006.0.0.0 and later. After updatingto 6.0.0.x, these arrays haveshown a higher than expectedfrequency of unexpected con-troller reboots. This issue occurswhen the SAS HBA detectsfaulty states and triggers a con-troller reboot, resulting in anunexpected takeover event.

CS235 / CS300 /CS500 / CS700 ar-rays are deniedfrom updating to6.0.0.x

System Manage-ment

AS-123886

Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consolefor assistance in determining thecause of the failure.

If a software update precheckfails, in some cases it will returnonly the failure status withoutproviding additional informationabout the cause of the failure.

Software precheckfailures returngeneric error mes-sage

System Manage-ment

AS-87736

Not applicableThere is a possibility of sporadic,transient, active directory au-thentication failures. In thesecases, the system will recover onits own, requiring no user inter-action.

Intermittent loginfailures due to Ac-tive directorylookups failures

System Manage-ment

AS-92465


In the case of a large number ofFC sessions, a volume read RESTrequest can lead to high memoryusage which can cause groupmanagement service restart.

Group managementservice might getrestarted due tohigh memory usage

System Manage-ment

AS-120667

Not applicableIn rare occurrences, a customer-initiated reboot may cause akernel reboot on the active con-troller. This will cause a longerreboot cycle.

Graceful shutdowntakes longer thanexpected

System Manage-ment

AS-95169

Refresh the page to restorebutton functionality.

After performing a bulk updateoperation from the Manage >Data Storage > Volumes >volumename > Data Protec-tion tab in the GUI, the pagefooter may not update and pre-vious button may be unavailable.

Page footer in GUImay fail to updateafter bulk updateoperation

System Manage-ment

AS-97968

Use volume collections to checksyncRep volumes, or use otherfilters to meet the needs

There is currently no way to fil-ter volumes using SynchronousReplication within the array GUI.

Unable to Filter vol-umes using Syn-chronous Replica-tion

System Manage-ment

AS-92379

NimbleOS 5.2.1.800 27



Not applicableThe Automatic Switchover Ser-vice monitors connectivity be-tween arrays in a synchronousreplication group. After a periodof time the memory usage of theservice increases until it exceedsits limit, then the service restartsto recover.

AutomaticSwitchover servicemay restart due tomemory leak

System InternalsAS-104154

Not applicableDue to a race condition, the DataService can crash during agraceful shutdown causing unex-pected Data Services restartmessages to be generated. Thisshould not cause any I/O impactbecause the Data Service is al-ready in the process of shuttingdown.

Data Service canrestart unexpected-ly during shutdownprocess


Resolved Critical Issues

Resolved Critical Issues in Release 5.2.1.800


Not applicableDuring update from NimbleOSversions prior to 5.2.1.x to5.2.1.x or later, the fingerprintindex is converted to a newerformat. In extremely rare scenar-ios, during this conversion pro-cess, NimbleOS may use an incor-rect reference pointer that cancause the Data Service to restartrepeatedly.

Unexpected DataService restart dur-ing fingerprint in-dex conversion


Not applicableData Services may unexpectedlyrestart due to out of memorycondition while running snapreplication on dedupe enabledvolumes with high compressionratios.



NimbleOS 5.2.1.800 28

Resolved Critical Issues in Release 5.2.1.800


After updating to 4.x.x.x or5.0.x.x, wait at least 1 hour be-fore updating onto 5.1.x.x+ orcontact HPE Nimble StorageSupport.

After performing a software up-date from 3.x.x.x (or earlier) to4.x.x.x or 5.0.x.x and then updat-ing onto 5.1.x.x+ immediatelyafter the 4.x.x.x or 5.0.x.x updatecompletes, there may not beenough time for an internalmetadata operation to completein the short amount of time thearray is on 4.x.x.x or 5.0.x.x. Asa result, the Data Service can gointo a restart loop leading to anoutage.

Data Service mayrestart repeatedlyafter performingmultiple softwareupdates in shortsuccession


Not applicableFirmware update may fail onspecific 1.9 TB SSDs. This willresult in an alert but will not af-fect operation of the array.

Drive Firmware up-date may fail onspecific 1.9 TB SS-Ds

PlatformAS-124465


In rare circumstances, duringshelf activation the FC driverthreads may deadlock which cancause the array controller topanic reboot to recover from thelock.

Fibre Channel ar-rays may experi-ence unexpectedcontroller rebootduring shelf activa-tion

SANAS-123867

Not applicableFibre Channel HBAs may en-counter an infinite port dumploop. If a Fibre Channel card inan array controller encounters3 firmware driver dumps within10 minutes the port will bebrought down on the controllerto prevent repeat occurrenceswhich could result in data un-availability. If the array deter-mines connectivity is better onthe peer controller due to downports the array may failover toimprove connectivity.

Fibre Channel portmay be broughtdown if severalfirmware driverdumps are encoun-tered

SANAS-123993

None. Data Service restarts toclear the inconsistent state andrestores the normal operations.

Due to a bug in Fibre Channeldriver, Task Management Func-tions are not properly updatingthe identifier that is unique to aspecific SCSI command or TaskManagement Function. In rarescenarios, this leaves the com-mand processing state machinein fibre channel driver in an in-consistent state and data servicerestarts to recover from this sit-uation.

All SCSI commandsand Task Manage-ment Functionsmust have a uniqueidentifier.

SANAS-121954

NimbleOS 5.2.1.800 29

Resolved Issues

Resolved Issues in Release 5.2.1.800


Not applicableWhen trying to allocate datapages for many large FingerprintIndex Hash Table indirectblocks, the process may fail forAF80 and HF60 arrays withdeduplication enabled. As a re-sult, the Data Service can go intoa restart loop, which can lead toextended data unavailability.

Data Service mayrestart unexpected-ly for HF60 andAF80 arrays



During controller boot, firmwareupdate may fail to completecausing the controller to boot tomaintenance mode.

Replacement con-troller fails to bootto NimbleOS

PlatformAS-123039

Not applicableThe Process Management ser-vice (PMD) monitors servicememory allocations for the arraycontrollers. The Link Layer Dis-covery Protocol (LLDP) serviceis used to query network infor-mation for the array, and is notcurrently monitored by PMD.This can allow the LLDP serviceto leak memory until an out ofmemory condition is encoun-tered causing the controller re-boot to recover.

Controller may re-boot unexpectedlydue to out of memo-ry for Link LayerDiscovery Protocol(LLDP) service

PlatformAS-123482

Known Issues

Known Issues in Release 5.2.1.800


Not applicableA lost wake-up to an operationmay cause the operation block-ing the checkpoint which resultsin Data Service restart.



Not applicableA volume that has a chain of atleast three branches, commonin replication or snapshot re-store workflows, may triggerunexpected Data Servicerestarts upon branches beingtrimmed and deleted.

Data Service mayrestart duringbranch point dele-tion


NimbleOS 5.2.1.800 30



Contact HPE Nimble StorageSupport to increase the healthcheck timeout on the array

After updating the array toNimbleOS 4.x.x.x or later, a pro-cess runs to convert an on-diskdata-structure specific to Nim-bleOS by initiating an operationthat walks the existing tree ofthis data-structure. In some cas-es, this operation keeps runningeven after the conversion hascompleted successfully, leadingto health check timeout. Thisresults in a Data Service restart.

Data Service restartwhile converting in-ternal data-struc-tures in NimbleOS


Not applicableIn rare instances, the Data Ser-vice may restart when internaldatabase communication be-tween services is not available.The service restarts to restorethe communication betweenservices.

Data Service mayrestart unexpected-ly due to internaldatabase communi-cation


Not applicableThe Data Service may restartunexpectedly during a read op-eration when decryption anddecompression fails.



This issue is resolved when con-nectivity to the standby con-troller is resolved. This may re-quire a controller failover or arestart of the standby controller.

When iSCSI protocol is enabledon a Fibre Channel array, if thestandby controller is not runningor if the service that managesthe Fibre Channel ports is notreachable on the standby con-troller, then the File System ex-periences issues when attempt-ing to commit the pending iSCSItarget IP information internally.This causes the File System ser-vice on the array to restart unex-pectedly.

Unexpected filesystem servicerestart when en-abling iSCSI proto-col on a FibreChannel array


Not applicableA race condition between twothreads can occur while internalsensor information is being up-dated on the array. This cancause the Data Service to restartto recover from the condition.

Data Service mayrestart due to racecondition while up-dating internal sen-sor information


NimbleOS 5.2.1.800 31



If RAID is degraded for an ex-tended period and Data Servicerestarts occur, contact HPENimble Support, supporting HPEAlletra 6000 and Data ServicesCloud Console to assess adjust-ing allocated buffer resources.

When RAID is degraded, IOneeds to be reconstructed byreading from multiple disks, andan internal buffer may exhaustits allocated resources. In rarecases when multiple disks aredegraded, the Data Service mayrestart unexpectedly.

Data Service mayrestart unexpected-ly when RAID is indegraded mode


Not applicableThe Data Service on the arraymay restart after a deletion of4TB or greater amount of dataon the array which can cause amemory leak leading to scratchpool exhaustion. The servicerestarts to recover from thecondition.

Data Service mayrestart after dele-tion of largeamount of data


Not applicableSoftware update to NimbleOS5.2.1.0 and later triggers an up-date of on-disk data structures.Under certain rare conditions,this update could potentiallytake a long time and result in arestart of the Data Service. Thismay cause a momentary delayin IO but should not result in anysignificant disruption, as the da-ta services daemon is designedto restart efficiently before anyIO timeout can occur.



Data service will be available af-ter restart.

During NimbleOS metadatasync, in rare instances, the dataservice may restart unexpected-ly. The metadata sync operationitself wont be affected and therestart will reset the race condi-tion; the data service will stabi-lize after the restart.

Data ServiceRestart due to arace condition dur-ing metadata sync


NimbleOS 5.2.1.800 32



Not applicableAs part of the file system check-point process, internal indexesare required to merge to disk.The health check process usesa heartbeat mechanism to deter-mine if the merge is makingprogress. Under some circum-stances, the process performingthe merge of a batch of updatesto the index fails to heartbeatwithin a deadline. When thathappens, the Data Service mayrestart with a health check fail-ure.

Data Service mayrestart unexpected-ly with health checkfailure when inter-nal index datastructures take toolong to merge



The Data Service may restartunexpectedly if the VolumeManager health check exceedsexpected time out. The servicerestarts to clear the condition.

Data Service mayrestart due to vol-ume managerhealth check failure


Not applicableUnder certain conditions, theData Service may restart whena large number of internal index-es merge within a short time-span and when some of theseindexes need to merge multipletimes in the same checkpoint. Inthis case, there can be lock con-tention between operations re-sponsible for picking what tomerge and operations process-ing the merge, leading to healthcheck timeout.

Data Service mayrestart unexpected-ly with VM healthcheck failure due tolock contention.


Not applicableThe File System Service mayunexpectedly restart because itis not able to allocate memory.This can happen when the pro-cess in charge of evicting dataout of cache cannot evict fastenough, so the filesystem is un-able to allocate memory for newincoming data.

File System Servicemay restart due tomemory exhaustion


NimbleOS 5.2.1.800 33



Not applicableIn case of large random reads orlarge number of random reads,the IORW buffers are held untilafter the read is done to use thecompressed block to insert it in-to cache. All the individual seg-ment reads need to completebefore performing cache inser-tion. Until all reads complete, thearray does not release any IORWbuffers. In the event there are abunch of large random reads inthe system, they will sendbatches of reads to the diskscheduler. The disk scheduler isnot aware these batches ofreads are connected. Since thereare many volumes, the diskscheduler tries to be fair to allvolumes and select one DiskOpper volume every time it re-sumes. When the large reads donot complete, the IORW bufferswill be held causing a timeoutwhich results in a restart of theData Service to recover.

Data service mayrestart when seg-ment read opstimeout after wait-ing for IORWbuffers


The array group will recover it-self following the Data Servicerestart.

The Data Service may restartunexpectedly on an array groupwith Storage Class Memory andSynchronous Replication config-ured, during an automaticfailover (AFO) event. This occurswhen invalidating index data inmemory exceeds the timeout of30 seconds.

Data Service mayrestart duringmetadata invalida-tion


No action is required, the alertshould resolve a short time later.

The Group Management servicesynchronizes attributes with theData Service on the array. If theData Service does not send aresponse within 10 seconds, theGroup Management service willgenerate an alert indicating theattribute synchronization maybe delayed. The service will retrythe synchronization, and it willcomplete a short time later whenthe response from the Data Ser-vice has been received.

Attribute synchro-nization may be de-layed


NimbleOS 5.2.1.800 34



Not applicableUnder certain circumstances, anunexpected Data Service restartmay be triggered with a genericAsynchronous Task Managerhealth check failure. The DataService restarts and resumesoperation normally, withoutcausing loss of access from theconnected hosts.

Unexpected DataService Restart


Not applicableIn certain array setups whereasynchronous replication is inprogress, arrays might en-counter a situation where thepartner array has connectionsinterrupted (not limited to soft-ware update) and is unreachableby the other array. The otherarray is, however, unaware of itbecause some network devicehas intervened the reachabilitystatus of the partner array. Theother array needs to performcertain operations, which havea timeouts associated with them,and the array undergoes a DataService restart to recover thecondition.

Arrays with replica-tion might experi-ence Data Servicerestarts when repli-cation partner isunreachable



When trying to allocate datapages for many large FingerprintIndex Hash Table indirectblocks, the process may fail dueto insufficient memory. As a re-sult, the Data Service mayrestart to recover the condition.

Data Servicerestarts unexpect-edly while allocat-ing data pages


Not applicableDuring array software update to5.2.1.600, the Fingerprint IndexDedupe Domain locking mecha-nism may encounter a lock in aninconsistent state. This can leadto a deadlock. The Data Servicewill restart to recover from thiscondition.

Data Service mayrestart unexpected-ly during softwareupdate


NimbleOS 5.2.1.800 35



Not applicableDuring Fingerprint Index (FI)processing threads may becomedeadlocked with Garbage Collec-tion (GC) threads causing a racecondition and a health checktimeout for Automated TaskManager (ATM). The Data Ser-vice will restart if it does not re-ceive a response from the ATMthreads to recover from thecondition.

Data Service mayrestart due to racecondition duringFingerprint Indexprocessing


Not applicableThe Group Data service mayrestart unexpectedly when outof memory condition is encoun-tered. The service restarts toclear the condition.

Group Data Servicemay restart unex-pectedly


Not applicableWhile the Data service is process-ing Update Entries in back-ground, one of these back-ground operations may exceedthe expected timeout and theservice will restart to recover.



Not applicableWhen a read op finds partial da-ta in-core, it issues a media readto get remaining data. By thetime, media read returns, the in-core data is synced, and tree isreopened for deletion. The readdoes not expect tree to be indelete state and causes the DataService to restart.

Data Service mayrestart unexpected-ly due to race condi-tion


Not Applicable, the Data Servicewill resume normal operationafter restart.

Generation delete loads a largenumber of ondisk metadatablocks which may prevent blockindex operation checkpoint fromfinishing. This causes the volumemanager health check to failwhich results in Data Servicerestart.

Data Service mayrestart due to vol-ume managerhealth check failureduring generationdeletion


Not applicableIf the Asynchronous Task Man-ager (ATM) does not respondwithin the expected timeout, theData Service will restart to recov-er from the condition. In rare in-stances, while ATM is collectingblock mapping information, itmay take longer than expectedcausing the timeout.

Data Service mayrestart unexpected-ly due to healthcheck failure


NimbleOS 5.2.1.800 36



Not applicableIn rare instances during offsetcalculation, the File System ser-vice may restart when attempt-ing to read data for replication.The service restarts to recoverfrom the condition.

File system servicemay restart on up-stream array duringreplication orresync of syn-chronous replica-tion.



When flash cache Garbage Col-lection copies forward live dataof a fragmented segment, itcould consume more buffersthan provisioned and cause theData Service to restart to recov-er.

Data Service mayrestart when run-ning out of buffers


Not applicableDue to issues in communicatingwith the partner array duringreplication, there are few caseswhere the operation is not ableto make progress. As a result,the file system may restart tocorrect this condition.

File system restartto recover fromstalled replication


Not applicableWhile committing internaltransactions, Data Service mayhit a rare race condition. To re-cover from this Data Servicemay restart.



Not applicableWhile committing internaltransactions, Data Service mayhit a rare race condition. To re-cover from this Data Servicemight restart



Perform one of the followingoptions to avoid/mitigate the is-sue: 1. Reduce overall workloadon the array. 2. Reduce copy of-fload workload on the array.Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consoleif further assistance is needed.

Copy offload optimizations fordeduplication were added inNimbleOS versions 5.1.x andlater. The changes may lead toincreased latency when copy of-fload workflows are added to thearray. This is due to an increasein update entries requiring pro-cessing.

Arrays may experi-ence increased la-tency during copyoffload workloadoperations


Not applicableIf IO to disk are slow or fail, theData Service may restart to tryto recover the condition.

Data Service restartdue to slow disk IOor disk IO failure


NimbleOS 5.2.1.800 37



Not applicableDue to a race condition, the DataService may restart during agraceful shutdown causing unex-pected Data Services restartmessages to be generated. Thisshould not cause any I/O impactbecause the Data Service is al-ready in the process of shuttingdown.

Data Service canrestart unexpected-ly during shutdownprocess


Not applicableIn rare cases, Data Service mayrestart during snapshot replica-tion due to the failure of check-sum algorithms to detect allnetwork errors.

Data Service mayrestart due to net-work errors


Not applicableDue to a miscommunication be-tween the group leader andbackup group leader arrays in aSynchronous Replication group,a discrepancy occurs while set-ting the state of a volume asupstream or downstream. TheData Service restart will clear thediscrepancy and no additionalaction is necessary.

Data Servicerestarts unexpect-edly on a Syn-chronous Replica-tion Group


Enabling encryption will resolvethe issue

Disabling encryption mightcause NVRAM data to fill upwith data waiting for masterpassphrase, which can causeSynchronous Replicated vol-umes to go out of sync.

Disabling encryp-tion may causeSynchronous Repli-cated volumes toremain out of sync


Not applicableThe Data Service may restartunexpectedly if Fibre Channelsession termination exceedstimeout to complete. The servicerestarts to recover from thecondition.



Review network to determinecause of iSCSI aborts to reduceand/eliminate the aborts.

In Synchronous Replication con-figurations, iSCSI aborts canmark volumes out of sync, caus-ing repeat group transitions andresync for out of sync volumes.This can cause the volume syncto become stuck or increase la-tency.

iSCSI aborts maycause SynchronousReplication vol-umes to go out ofsync


NimbleOS 5.2.1.800 38



The array will continue to shut-down after the Data servicerestart.

Volume manager does not resetinternal callbacks during theshutdown phase causing theData service to restart.

Data service mayrestart during arrayshutdown


None. The Data Service Restartwould resume normal I/O opera-tions.

A rare scenario can result into arace condition between clonecreation and I/O operations onan encrypted volume; during thistime while fetching the encryp-tion keys Data Service mayrestart and resume normal I/Ooperations.

Data Service restartdue to a race condi-tion


Not applicableThe Data Service on the arraycommunicates with the GroupManagement Service to handlearray operations. If the commu-nication between the two ser-vices exceeds timeout, the ser-vice will restart to recover.



Not applicableThe volume manager will decre-ment bin and child volume folderentries during volume deletion.In rare occurrences, anotherthread may decrement the childvolume entry and not the binentry, causing the Data serviceto restart.

Data Service mayrestart on down-stream array due torace condition dur-ing volume deletion


Remove downstream replica us-ing the correct steps orderedbelow. 1. Claim the repli-ca volume. 2. Delete all of thesnapshots for the replica vol-ume. 3. Delete the replica vol-ume.

The service may restart whenremoving the downstream repli-ca using the steps below. 1. Deletion of all snapshots forthe replica volume. 2. Claim thereplica volume 3. Delete thereplica volume

Very rare race be-tween Vol claim(with all snapshotsmarked for dele-tion) and space re-calculation on repli-ca downstream vol-ume


Retry operation after a few min-utes to reassign array to pool.

Assigning an array to a pool im-mediately after unassigning itfrom the same pool will fail withthe following error - Failed toassign arrays to the pool: A ser-vice is not running or is notreachable

Unassigning andreassigning array toa pool within 5 min-utes will fail


NimbleOS 5.2.1.800 39



Not applicableThe Data Service may restartduring array shutdown whenjobs related to the Data Serviceare unable to be gracefullyshutdown. The restart shouldnot have any impact as the arrayis in shutdown process.

Data Service mayrestart during arrayshutdown


Reduce the frequency of volumeclone create and delete opera-tions to minimize the likelihoodof occurrence.

Aggressive volume creation anddeletion may cause one of thetree data structures to hit maxi-mum children it can have. Thislimit can occur on volume cre-ation because volume deletiondestroys the children asyn-chronously. The Data Service willrestart when this limit has beenreached.

Aggressive Volumecreation and dele-tion may result inData Service restart


Not applicableIn rare cases, during volumedeletion processing, an objectmay be released while waitingon a lock in error. This will causethe Data Service to restart torecover from the condition.

Data service mayrestart due to racecondition duringvolume deletion


Not applicableVolume moves transfer datafrom one Nimble array to anoth-er. During this move, if the hostsends I/O to the incorrect array,the I/O needs to be forwardedto the correct array owning thedata. This results in higher thanusual I/O latency. To avoid thisissue in VMware environments,the Nimble Connection Managerfor VMware needs to be installedon all hosts accessing the vol-ume.

Volume move mayresult in latency ifNimble ConnectionManager is not in-stalled


Not applicableThe Group Management servicecommunicates with the VSS ser-vice when creating VSS snap-shots. If this communication isnot able to be completed withinthe expected timeframe, theservice restarts to recover.

Group Managementservice may restartwhen attempting tocreate VSS snap-shot


Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consoleto increase the memory limit forthe service.

In some instances the cimserverservice will exceed its memorylimit and no longer be able tostart.

Cimserver stopswhen memory limitis exceeded.


NimbleOS 5.2.1.800 40



Please Contact HPE NimbleSupport, supporting HPE Alletra6000 and Data Services CloudConsole

In rare incidents, controllers donot power on following powercycle.

Controller does notpower on followinga power cycle.

PlatformAS-100088

Referencing the correspondingExpansion Shelf Quick StartGuide, correct the hardware ca-bling issue.

If a shelf is cabled incorrectlyand added to an array, the rawcapacity calculation could failleading to multiple restarts ofthe Data Service.

Incorrect shelf ca-bling can cause aseries of unexpect-ed Data Servicerestarts

PlatformAS-67242

The controller reboot should re-store SAS HBA to normal state.HPE Nimble Support, supportingHPE Alletra 6000 and Data Ser-vices Cloud Console may contactcustomer to collect additionaldiagnostics if required.

When the SAS HBA detectsfaulty states, to recover, the ar-ray needs to reset the SASHBA’s firmware. The SAS HBAfirmware reset can block diskI/Os significantly longer than ourHigh Availability monitoringtimeouts allow. Instead, a con-troller reboot is triggered imme-diately if this state is detected,resulting in an unexpectedtakeover event.

Unexpected con-troller takeover dueto incorrect state ofthe SAS HBA

PlatformAS-33725

Not applicableUnder certain conditions, dataservice may restart during seg-ment allocation. The servicerestart will clear in-memory dis-crepancy and no additional ac-tion is necessary.

Data Service mayrestart unexpected-ly during to seg-ment allocation

PlatformAS-123153

Not applicableThe Enclosure Managementservice may unexpectedlyrestart due to a race conditionwhich is resulting from drivestatus which has already beenmarked removed by the DataService. The restart of the ser-vice clears the condition.

Enclosure Manage-ment service mayunexpectedlyrestart due to drivestatus race condi-tion

PlatformAS-95294

Not applicableIn certain conditions, I/Ostracked for certain operationsmay total zero, which triggers acheck exception. The reason forthe incorrect value is variabletruncation. The service restartsto recover from the condition.

File System servicemay be restart ontruncated variableread

PlatformAS-114564

Not applicableThe Data Service may restartunexpectedly when it detects aninternal check error. The restartof the service clears the condi-tion.

Data Service mayunexpectedlyrestart

PlatformAS-90850

NimbleOS 5.2.1.800 41



Not applicableThe Data Service may restartdue to a transient resource allo-cation failure. This happenswhen the service cannot com-plete a disk IO due to transientmemory allocation failure. Thisdoes not cause a service outageas Data service continues normal-ly after a restart.

Data Service restartdue to resource allo-cation failure.

PlatformAS-103802

Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consoleto identify the faulty drive forreplacement.

Data Service may become latentand restart when the array hasa faulty drive with a lot of medi-um read and IO timeout errors.

Data Service mayrestart if the arrayhas an faulty drivewith a high numberof medium read er-rors

PlatformAS-110036

The restart of the service willclear the low memory condition,no further action is needed.

Data Service may restart in therare case when a controller islow on memory.

Data Service mayrestart if a con-troller is low onmemory

PlatformAS-99567

Please Contact HPE NimbleSupport, supporting HPE Alletra6000 and Data Services CloudConsole so the problematic drivecan be replaced.

In rare instances due to slow IOto one of the disks, the DataService may hit a timeout andrestart unexpectedly. This issuemay occur during a NimbleOSsoftware update.

Data Servicerestarts due to slowIO to one of thedisks

PlatformAS-90096

Not applicableIn rare circumstances, the arraysdata service may restart unex-pectedly when it encounters arace condition following internaldata striping processing. Thedata service will stabilize follow-ing the restart.

Data Service mayrestart unexpected-ly during internalstriping operation

PlatformAS-113002

Not applicableIn rare circumstances, a kernelfile system defect may manifestitself and cause a controller re-boot. If this occurs on the activecontroller, it will result in an un-expected takeover on the array.

Kernel file systemdefect may causean unexpectedtakeover

PlatformAS-116189

Not applicableIn rare circumstances, a kernelmemory management defectmay manifest itself duringmemory unmapping operationsand initiate a controller reboot.If this occurs on the active con-troller, it will result in an unex-pected takeover.

Kernel memorymanagement de-fect may cause anunexpectedtakeover

PlatformAS-116155

NimbleOS 5.2.1.800 42



Not applicableIn rare circumstances, the kernelmay hit a bug where a stalememory management datastructure entry is referenced,causing a controller reboot torecover from the condition.

Unexpected con-troller reboot dueto a kernel memorymanagement bug

PlatformAS-123316

An array failover or reboot canbe performed to clear incorrectstale information.

When physically attaching newexpansion shelf, if it is connectedto wrong port multiple times,Array Management and DataServices may restart due to incor-rect stale information.

Array Managementand Data Servicesmay restart whennew expansionshelf is connectedto wrong port

PlatformAS-127484


Due to an incorrect responsefrom dual flash carrier (DFC)firmware the Array ManagementService might restart intermit-tently.

Array Managementservice may restartunexpectedly

PlatformAS-117809

Not applicableThe Enclosure Managementservice may unexpectedlyrestart when it detects an inter-nal check error. The servicerestart clears the condition.

Enclosure Manage-ment Service mayrestart unexpected-ly

PlatformAS-105053

Review the array configurationmatrix for the array model: <ahref=https://infos-ight.hpe.com/InfoSight/media/lo-cal/active/34/CSxxx%20Con-fig%20Matrix.pdf>https://infos-ight.hpe.com/InfoSight/media/lo-cal/active/34/CSxxx%20Con-fig%20Matrix.pdf</a> Removeany additional cache from theexpansion shelf that exceeds themax cache limit based on arraymodel.

ES2 and AFS2 expansionshelves contain additional slotsfor upgrading cache capacity ofthe array. Older array modelshave a maximum cache limit thatcan be handled by the array. IfES2 or AFS2 expansion shelvesare added to an array and thecache exceeds the max cachelimit for the array type, the dataservice may restart due to run-ning out of data pages.

Data service mayrestart if maximumcache exceeded forCS215, CS235,CS300, CS500,CS700 arrays

PlatformAS-93296

NimbleOS 5.2.1.800 43



Add the disk from the CLI usingthe disk add command and out-put from disk list: 1. Run disk --list 2. Note the slot number, andshelf location for the disk la-beled foreign. 3. Add the disk:disk --add <slot number> --array <arrayname> --shelf_location <shelf loca-tion> Note: the --force optionmay be required Contact HPENimble Support, supporting HPEAlletra 6000 and Data ServicesCloud Console if the disk doesnot move to resynchronizingstate after completing the com-mands.

Disk will report foreign for diskstate in the GUI Hardware pageor in the output of disk --listfrom the CLI. This typically oc-curs if diagnostic data may nothave been removed after test-ing.

Replacement diskreports foreign fordisk state inGUI/CLI

PlatformAS-99428

No workaround is required. Thearray will recover itself automat-ically by restarting the DataService.

On rare occasions, the arraygroups Data Service may fail toinitialize if a shelf state changeoccurs simultaneously.

Delay with DataService startingduring shelf statechange

PlatformAS-101570

Not applicableIn rare cases, an SSD may reachits endurance limit but continuesto pass Nimble Drive Error Re-covery algorithm. This causes anever-ending process of off-lin-ing and on-lining the drive. Thismay occur for Intel and ToshibaSSDs.

SSD has reached itsendurance limit(wear leveling) butthe disk is notmarked failed.

PlatformAS-91522

Not applicableThe Array Management Serviceprocess may restart after run-ning for a long time due to amemory leak issue. This cancause a short interruption todata service, but it will resumeonce the process has restarted.

Array ManagementService restart

PlatformAS-116083

Not applicableIn a rare scenario, EnclosureManagement service may occa-sionally restart upon a drivefailure in system partitions. Therestart of the service clears thecondition.

Enclosure Manage-ment service occa-sionally restarts up-on a drive failure

PlatformAS-86100

NimbleOS 5.2.1.800 44



Not applicableAn alert may appear with thefollowing text: The power supplyof 0 watts at <PSU unit num-ber> on array <array> isincorrect. The array requires apower supply of 3000 watts. Ifthe system monitor fails to re-ceive wattage information, thealert may appear and resolve afew minutes later.

Array may fail toquery power supplywattage readings

PlatformAS-112817

Not applicableDue to a possible software de-fect, the red disk slot LED mayilluminate even when there areno issues with the correspondingdrive. This is a cosmetic issuethat does not impact functional-ity.

4U Chassis disk slotLEDs illuminate er-roneously

PlatformAS-116339

Not applicableThe File System Service mayrestart during memory allocationif there is not enough memoryavailable. The service restarts torecover from the condition.

File System Servicerestarted unexpect-edly

PlatformAS-116111

Not applicableThe Group Management servicemay restart if an error whileopening interconnect-ips fileoccurs. The service restarts torecover from the condition.


PlatformAS-112701

Not applicableFile System service may restartunexpectedly when a healthcheck exceeds timeout for vol-ume manager on the array. Theservice restarts to recover fromthe condition.

File System Servicemay restart unex-pectedly

PlatformAS-92471

Please contact HPE NimbleSupport, supporting HPE Alletra6000 and Data Services CloudConsole to resolve this issue.This issue is resolved in Nim-bleOS version 5.0.2.0 and later.

An attempt to create new VMsin a VMWare vVol environmentfails after updating to NimbleOS4.x.x.x. This occurs when a sys-tem partition within the array isrunning out of space.

VM creation fails inVMWare VVol envi-ronment after up-date to NimbleOS4.x.x.x

PlatformAS-76174

NimbleOS 5.2.1.800 45




Under rare circumstances, if anarray software update previous-ly failed and reverted, and anarray chassis swap is performedbefore the next software update,the array configuration files maycontain invalid data. This willcause the controllers to rebootand the Array Management ser-vice to restart.

Array may fail toboot following asoftware update ifprevious chassis in-formation is refer-enced

PlatformAS-114390


In a dHCI Brownfield environ-ment, if CHAP authentication isenabled on the hosts, the CHAPaccount will need to be addedto the array via CLI by HPENimble Storage Support beforecompleting dHCI setup.

Brownfield dHCIsetup with CHAPenabled hosts willrequire assistancefrom HPE NimbleStorage Support

dHCIAS-121775

Free up /var disk space bydeleting unwanted log files(/var/log/EMU/mili/mili2d.log)and rebooting the host.

As part of the dHCI updateworkflow which involves the ES-Xi server update, some com-mands need to be run on theserver. Running these com-mands fails with error Error: Ageneral system error occurred:Internal error. if the /var is full.This is a known issue with 6.7builds running with Emulexdriver which has been fixed aspart of ESX 6.7 U2. <ahref=https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-esxi-67u2-release-notes.html#re-solvedis-sues>https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-esxi-67u2-release-notes.html#re-solvedissues</a>

dHCI update work-flow can fail if /varmountpoint on thehost is full.

dHCIAS-103315

The only way to resume the up-date would be to failover to theoriginal group leader array andthen resuming the updatethrough the dHCI vCenter plug-in.

When the dHCI unified updatefeature is used to update thedHCI stack, the update will failif a Group Leader Failover oc-curs during the process.

An in-progess dHCIupdate fails if thegroup leader failsover

dHCIAS-103247

After confirming the array hasreturned to Active/Standby sta-tus, stop the hung task and runthe Nimble Add ProStack servertask again.

If an array failover occurs whileNimble Add ProStack server taskis running from vCenter, theprocess may hang and not com-plete.

Nimble AddProStack servertask may hang ifarray failover oc-curs

dHCIAS-97685

NimbleOS 5.2.1.800 46



Not applicableWhen an ESXi server is added todHCI cluster, the update pagedoes not get updated to includethe newly added ESXi server.This refresh happens every 4hours. After the next refresh, thenew ESXi servers version will beincluded and accounted for onthe update page.

The update pageon the dHCI plugintakes 4 hours to re-fresh

dHCIAS-103769

Modify the browser size orscreen resolution so the EULAis fully displayed, and thecheckbox is activated. If issuespersist, please contact HPENimble Storage Support.

When attempting a dHCI update,due to the browser size orscreen resolution, the EULA istruncated when you attempt toscroll down. And as a result, thecheckbox to agree to the Termsand Conditions is not activated.This will prevent you from mov-ing ahead with update process.

Cannot select EULAcheckbox duringdHCI update

dHCIAS-117035

A valid ESXi license must be as-signed to the server.

When adding a server with anexpired ESXi license to the dHCIcluster, through the vCenterplugin, you may see an errorsaying - Failed to submit a taskto add server.

Addition of a serverwith expired ESXilicense fails

dHCIAS-95054

Admission Control should bedisabled on a dHCI cluster forthe update to proceed.

For ESXi server update, DRS isused to migrate VMs running onthe server. If admission controlis enabled on the dHCI cluster,DRS is not able to migrate VMsoff a server.

dHCI update failswhen AdmissionControl is enabled

dHCIAS-101915

Not applicableThe Data service will wait up to300 seconds for host responseto iSCSI commands. If host doesnot abort commands that takemore than 300 seconds to com-plete, the Data service willrestart.

Data Service mayrestart when hostdoes not issueabort to timed outcommand

SANAS-108946

Not applicableDuring a controller reboot, dueto resource contention betweennew Fibre Channel (FC) connec-tion attempts and shutdown ofthe FC module, the Data Serviceon the array may restart unex-pectedly.

Data Service restartduring shuttingdown FC service onthe standby con-troller

SANAS-100197

NimbleOS 5.2.1.800 47



These issues appear to be trig-gered by events external to thecontroller such as host HBA, ca-ble, SFP, zoning or switch issues.Port congestion may also play arole. Migrating volumes to otherarrays or adding additionalHBAs may alleviate the conges-tion.

The Fibre Channel driver statemachine that handles IO aborts,IO timeouts on the fabric, ses-sion logout, and port ownershipchanges relies heavily on theport owner to properly managestate in a timely manner. Thisgenerally works, but there aresituations where the port ownerdoesnt have enough context todo the job properly. This canlead to more IO timeouts, HBAresets, DSD restarts and occa-sionally kernel crashes. A futureversion of the driver will simplifythis process by handling it entire-ly within the driver.

Fibre Channel IOaborts and hangsmay lead to DSD orkernel crash

SANAS-94229

Not applicableDuring port owner failover be-tween the Data Service and theSCSI Failover/Forward Service,outstanding IOs may timeoutand get aborted. This can leadto state machine issues in be-tween the two services whichmay cause one or both servicesto restart. In rare instances, thismay also cause the controller toreboot. The service restartand/or controller reboots to re-cover from the condition.

Data Service, SCSIFailover/ForwardService, and/orcontroller mayrestart unexpected-ly for Fibre Channelarrays

SANAS-92368

Increase NCMs configuration forminimum number of connectionsper volume from 2 to 4 or high-er.

The NCM (Nimble ConnectionManager) handles the iSCSIconnection counts. Today, theNCM is designed to use a FULLMESH connection mechanism.However, when there are veryfew initiators and target ports,the total connection count willbe minimal. The NCM will defaultto a minimum of 2 connectionsper volume. Increasing the num-ber of connections to 4 or moremay produce the maximumthroughput.

Poor iSCSI perfor-mance seen whenminimum connec-tion count is used

SANAS-111926

NimbleOS 5.2.1.800 48



Identify the offline volumes orstale targets and initiate a hostside cleanup/rescan to avoid re-peated login attempts and fail-ure to the offline or stale targets.

Logins to CHAP authenticationenabled volumes could leak asmall amount of memory by re-peated failed login attempts tooffline volumes or stale targets.Over a period of days andweeks, this leak can result in oneor more of the NimbleOS pro-cesses running out of memory.As a result, NimbleOS servicesmay restart unexpectedly.

NimbleOS servicesmay restart unex-pectedly due tomemory leak in lo-gin path

SANAS-113037

Identify the offline volumes orstale targets and initiate a hostside cleanup/rescan to avoid re-peated login attempts and fail-ure to the offline or stale targets.

Logins to CHAP authenticationenabled volumes could leak asmall amount of memory thesize of CHAP username. Over aperiod of days and weeks, thisleak can result in one or more ofthe NimbleOS processes runningout of memory. As a result, Nim-bleOS services may restart unex-pectedly. The issue is exacerbat-ed by repeated failed login at-tempts to offline volumes orstale targets.

NimbleOS servicesmay restart unex-pectedly due toslow leak withCHAP logins

SANAS-109412

Not applicableWhen the active controller isbeing shutdown, the Data Ser-vice runs into an internal errorcondition that causes the serviceto restart unexpectedly. Sincethe process is already beingshutdown, there is no impact touser data availability.

The Data Servicerestarts unexpect-edly during shut-down

SANAS-98042

Not applicableDuring command processingeach iscsi target nexus (ITN) isassigned to a specific eventqueue (EQ) by the FC driver todeliver IO_COMPLETE eventsfor that ITN. In certain cases, ifthe driver cannot determine theITN or EQ to deliver theIO_COMPLETE events, the DataService may restart to recover.

Data Service mayrestart due to racecondition in FibreChannel transportlayer

SANAS-110717

Not applicableThe Data Service or SCSI HighAvailability Service may restartdue to race condition encoun-tered during process shutdown.

Service may restartdue to race condi-tion

SANAS-89753

NimbleOS 5.2.1.800 49



Not applicableUnder certain conditions, theData Service on the GroupLeader array may restart unex-pectedly while removing mem-ber array. This is due to a racecondition when processing SCSIRTPG (REPORT TARGET PORTGROUPS) commands. The ser-vice should stabilize on its ownshortly following the restart.

Data Service mayrestart unexpected-ly while removingmember array

SANAS-101325

Not ApplicableThis scenario is rare to happenbut presently not handledgracefully leading to a File Sys-tem restart for recovery. The is-sue is triggered when a write re-quest with valid length is re-ceived in SCSI Command Descrip-tor Block but invalid (zero) valuein Data-Out Buffer.

File System servicemay restart whenan invalid write re-quest is received

SANAS-94761


If a software update of a mem-ber array in an array group failsand does not complete, theGroup Data Service may restartdue to unsupported iSCSI re-quest handling.

Group Data Servicemay restart unex-pectedly if groupsoftware updatefails

SANAS-103540

Not applicableInitiator Target Lun (ITL) ob-jects are the connection pathsbetween an initiatior and a tar-get LUN. During a shutdown ofservices on an array, a memoryleak may be detected for theseITL objects. As a result, the DataService and/or SCSIFailover/Forwarding Service mayrestart unexpectedly in order torecover from this condition.

Data Service and/orSCSI Failover/For-warding Servicemay restart due toa memory leak dur-ing shutdown

SANAS-126234

Use an alternative authentica-tion method for SMTP service.

SMTP authentication usingNTLM uses DES encryption,which is considered insecure andis not allowed.

NTLM Authentica-tion with use ofDES is not allowed

SecurityAS-118739


In certain scenarios, off-lining avVol snapshot will not completedue to the existence of a ACLbound with it. The vVol snap-shot cannot be off-lined until thebound ACL has been removed.

Cannot take abound virtual vol-ume snapshot of-fline

System Manage-ment

AS-116512

NimbleOS 5.2.1.800 50



Not applicableGroup Management service mayrestart during internal databaseACL processing if the task ex-ceeds the expected timeout.

Group ManagementService may restartdue to internaldatabase timeout

System Manage-ment

AS-121579


The Group Management servicecommunicates with the DataService to add the ProtocolEndpoint ACL to the Scale OutDatabase for vVols. If the pro-cess does not complete as ex-pected the Group Managementservice will attempt to retry theoperation. Under some circum-stances, instead of retrying thatoperation, this ACL get erro-neously deleted and any PEreads (REST or cli) followingcould result in GMD restart.

Group Managementservice may restartwhile reading proto-col endpoint infor-mation

System Manage-ment

AS-119873

Not applicableA race condition may occur whileupdating records in the ScaleOut database when two threadsattempt to update the samerecord.


System Manage-ment

AS-110367

Not applicableGroup Management service mayrestart due to Volume Manage-ment thread and API thread forsnapshot creation/deletion en-tered race condition. The servicerecovers after restart.

Group ManagementService restart dueto race condition

System Manage-ment

AS-105929

Not applicableGroup Management Service mayrestart unexpectedly as onethread has taken a ReadWritelock which has another writerthread, which is waiting forScale-Out Database (SODB)transaction to be completed.The service restarts due to theSODB transaction exceeding theexpected timeout.

Group ManagementService may restartdue to race condi-tion

System Manage-ment

AS-103766

NimbleOS 5.2.1.800 51



Not applicableAPI calls occur between theEvent service and Group Man-agement service when an alarmis cleared. The calls will lookupthe alarm ID and onset event inthe Scale Out Database (SODB).If the alarm is cleared before theonset event lookup completesand is not found during thesecalls, the service may restartunexpectedly.


System Manage-ment

AS-83604

Run the following command viathe array CLI: alarm --deletealarm_id

Alarms raised by a member ar-ray are visible when issuingalarm --list even after the mem-ber array is removed from thegroup.

Member arrayalarms are still visi-ble after array is re-moved from group

System Manage-ment

AS-106124

Not applicableDue to the multi-threaded na-ture of the Alerts and AlarmsService, it is possible for anevent with a later id to be post-ed prior to an event with an ear-lier id. Therefore it will have anearlier timestamp even thoughits ID is higher. This can causethe IDs to appear out of order.However, the alarms in the listare ordered correctly by times-tamp.

Alarm IDs in alarmlist may appear outof order.

System Manage-ment

AS-105431

After confirming that the config-uration sync has completed thealarms can be deleted or ac-knowledged through the GUI.

Alarms describing a delayedconfiguration synchronizationto an array may not be clearedproperly after an extended net-work outage and remain activein the GUI.

Alarms for Configu-ration Synchroniza-tion may remain af-ter sync has com-pleted

System Manage-ment

AS-99427

The problematic alarm can bemanually deleted through theCLI command: alarm --delete<alarm_id>

Due to a race condition it ispossible for an alarm to remainuncleared on the system evenwhen the alarm condition it isreporting is no longer the case.This can happen if the onsetalert and recovery alert weregenerated at close to the sametime.

Alarm may notclear as expected

System Manage-ment

AS-94398

NimbleOS 5.2.1.800 52



Not applicableIn rare instances, a postgrescommand issued by the Eventservice to delete alarms may faildue to an old reference. Theservice will restart to recoverfrom the condition.

Event Service mayrestart due to faileddatabase command

System Manage-ment

AS-98996

Clear the alarm manually fromthe Command Line Interface: 1.Use alarm --list command to de-termine the alarm ID. 2. Usealarm --delete <ID> com-mand to delete the alarm.

An alarm onset should be setprior to discovery request aspart of alarm processing. Insome cases the discovery re-quest will appear first resultingin the alarm not being cleared.

Configuration syn-chronization recov-ery alarm is notcleared

System Manage-ment

AS-87604

Not applicableAlerts and Alarms processingservice may restart unexpected-ly when certain operation sur-pass the designated healthcheck timeout. The process willstabilize following the restart.

Alerts and Alarmsprocessing servicemay restart unex-pectedly

System Manage-ment

AS-50821

Not applicableThe Eventd process may infre-quently crash, due to a bug in anexternal library used by the DNSresolver. The restart will not im-pact data connections to the ar-ray and the process will recoverafter the restart.

Eventd processmay restart inter-mittently

System Manage-ment

AS-46024

Not applicableThe eventd process may restartunexpectedly in systems withlarger configurations due to ex-ceeding memory limit. The ser-vice will recover after the restart.

Eventd processmay restart due toexceeding memorylimit

System Manage-ment

AS-99679

Not applicableThe Event service may restartunexpectedly due to a memoryaccess issue. The restart will notimpact data connections to thearray and the process will recov-er after the restart.

The Event servicemay restart unex-pectedly

System Manage-ment

AS-85608

Not applicableThe Eventd process may restartunexpectedly due to a raredeadlock condition between itsthreads. The restart of the pro-cess will clear the condition.

Eventd processmay restart unex-pectedly

System Manage-ment

AS-103567

NimbleOS 5.2.1.800 53



The recipient can create an in-bound transport rule to appenda disclaimer to the messagesfrom the problematic sender.The disclaimer will append theexpected CR-LF combination tothe message so that it can bedelivered. (This disclaimer mayconsist of a single charactersuch as a period or a dash.)

When the array sends an alertvia SMTP, the message may berejected by the server indicating550 5.6.11 SMTPSEND.BareLine-feedsAreIllegal. This can occurif the receiving server does notsupport BDAT command forSMTP chunking

Alerts sent viaSMTP may be re-jected

System Manage-ment

AS-72902

Not applicableThe array will reuse the sessionto the SMTP server each time itattempts to send an email. If thissession is no longer active, it willreport a failure to send the alert,then open a new session, andsuccessfully send the alert if theSMTP server configuration isvalid.

Event Service mayreport SMTP failureto send alert whenalert is sent success-fully

System Manage-ment

AS-40238

Not applicableEvent Service may restart whenmemory limit is reached. Theservice will restart to recoverfrom the condition.

Event Servicerestarts due tomemory

System Manage-ment

AS-84502

Rename the folders to beunique. If the Event Manage-ment service continues to restartor remains unavailable, contactHPE Nimble Storage Support.

When an alert is raised on oneof two folders having the samename but are located in twoseparate pools, the Event Man-agement service may restart re-peatedly.

Event Managementservice restartwhen two folders indifferent poolshave the samename

System Manage-ment

AS-101273

The alarm can be cleared fromthe CLI using the steps below.List the alarms with the follow-ing command to obtain thealarm ID: alarm --list Delete thealarm using the alarm ID ob-tained from the list. alarm --delete <alarmID>

Alarm notifying about delay insnapshot attribute synchroniza-tion is not cleared automaticallyafter successful synchronization.If the alarm is not manuallydeleted it will keep remindingthe customer even after success-ful synchronization.

Snapshot AttributeSynchronizationDelay Alarm is notcleared

System Manage-ment

AS-86837


The Array Management Servicethreads may become deadlockedwhile checking NIC status. Whenthis occurs the service will stopresponding.

Array managementservice may stoprunning due todeadlock

System Manage-ment

AS-116513

NimbleOS 5.2.1.800 54



Not applicableInternal database maintenanceprocessing may exceed the ex-pected health check timeout.When this occurs, the ArrayManagement service may restartto recover from the condition.

Array Managementservice may restartunexpectedly

System Manage-ment

AS-110436

The ID in the auditlog can bespecified in a GET curl call to thealarms REST API to retrievemore identifying information forthe alarm.

The ID field for alarms in audit-log does not match the ID foralarms listed in the CLI amongother places. The ID in the audit-log is actually a value generatedby the REST API to track differ-ent alarm objects. The valuedoes serve as a unique identifierfor the alarm but is not easy tocross reference with the alarmsdisplay.

ID field for alarmsin auditlog does notmatch ID in CLIoutput.

System Manage-ment

AS-120433

Not applicableEvery night, when inactive keysfor deleted encrypted volumeare deleted by the array, it cre-ates an audit log entry with rootas the user performing the ac-tion.

Purge inactive en-crypted keys ap-pears in audit log.

System Manage-ment

AS-96966

Not applicableIf a user tries to create a newuser account, but the user does-nt have the privilege to do so,the user creation will fail. Howev-er, an audit log entry is not cre-ated.

No Audit Log entryis created if userdoes not have theprivilege to createuser

System Manage-ment

AS-71090

Not applicableThe Group Management Serviceon the array may restart unex-pectedly when a SOAP timeoutis encountered after trying tocreate a volume. This occurs dueto a race condition where thearray attempts to delete thevolume after the creation at-tempt fails.

Group ManagementService restarts fol-lowing volume cre-ation timeout

System Manage-ment

AS-104640

Not applicableWhen an array has been re-moved from an array group,stale FC configuration entriesmay not be cleared causing theGroup Management service torestart to recover from the con-dition.

Group Managementservice may restartafter array removalfrom group

System Manage-ment

AS-100106

NimbleOS 5.2.1.800 55



Not applicableThis issue is seen when theGroup Leader role migrationcompletes, and the previousleader is left unconfigured. Inthis scenario, the Group Leadermay treat the previous leader asa member array and initiate theBackup Group Leader (BGL)setup. If a customer attempts toremove the array during thisBGL setup, the removal fails.

Array removal willfail if Backup GroupLeader setup is inprogress

System Manage-ment

AS-115935

Perform the Autosupport valida-tion process again.

When autosupport configurationvalidation is performed, internalprocess tracking may abortcausing the Group ManagementService to restart.

Group Managementservice may restartwhen performingAutosupport Valida-tion

System Manage-ment

AS-104185

Not applicableConnection to the Scale-OutDatabase can not be reinitializedwhen a transaction is inprogress. The Group Manage-ment Service will restart to re-store connection.

Group Managementservice may restartwhen connectionsto Scale-OutDatabase exceedsthreshold value

System Manage-ment

AS-109127

Not applicableThe Group Management Servicemay restart unexpectedly dueto receive buffer exhaustion onthe management network inter-face. No visible impact has beenreported because of this issue.

Unexpected GroupManagement Ser-vice restart due toreceive buffer ex-haustion

System Manage-ment

AS-98885

Not applicableDuring key value metadata han-dling, the process adds an entryin one table of the Scale-OutDatabase (SODB) and the newentry may reference to an al-ready deleted primary key inanother table in SODB. Thiscauses foreign key violations inPostgres and eventually leadsthe Group Management serviceto restart.

Group Managementservice may restartdue to key valuemetadata insertion

System Manage-ment

AS-118982

NimbleOS 5.2.1.800 56



Not applicableDuring key value metadata han-dling, the process may deleteprimary keys in one table of theScale-Out Database (SODB) andthen try to reference the deletedprimary key as foreign key inanother table in SODB. Thiscauses foreign key violations inPostgres and eventually leadsthe Group Management serviceto restart.

Group Managementservice may restartdue to snapshotkey value metadatahandling

System Manage-ment

AS-105454

Not applicableDuring key value metadata han-dling, the process may deleteprimary keys in one table of theScale-Out Database (SODB) andthen try to reference the deletedprimary key as foreign key inanother table in SODB. Thiscauses foreign key violations inPostgres and eventually leadsthe Group Management serviceto restart.

Group Managementservice may restartdue to key valuemetadata handling

System Manage-ment

AS-108146

Not applicableUnder certain circumstances, theArray Management Service maydetect operations that are stillrunning during a planned shut-down. This may result in an un-expected service restart prior tothe array completing shutdown.

During a plannedshutdown, the Ar-ray ManagementService may restart

System Manage-ment

AS-115493

Not applicableIn some cases, when the BackupGroup Leader is transitioning toa member array role, the ArrayManagement Service mayrestart unexpectedly. This oc-curs if the array experiences atimeout when the Array Manage-ment Service is attempting tostop the Backup Group Leaderprocesses.

Array ManagementService restartswhen attempting tostop the BackupGroup Leader pro-cesses

System Manage-ment

AS-125686

Not applicableIf there is a network connectivityissue between the witness andGroup Leader array, the groupstatus CLI output will update theFailover Mode from Automaticto Manual until the connectionis reestablished. It also displaysthe Witness Status as N/A asopposed to Unreachable.

group --status CLIoutput shows incor-rect Failover Modeduring networkconnectivity issues

System Manage-ment

AS-99704

NimbleOS 5.2.1.800 57



None.Due to a race condition, the Ar-ray Management service mayrestart when the array is in theprocess of shutting down allservices as part of the arrayshutdown. An alert and a pro-cess core could be generated asa result but there is no impactto functionality. Once the arraypowers back on, all services startup normally.

Array ManagementService may restartduring array shut-down

System Manage-ment

AS-106539

Not applicableDuring software update of arraygroup with Peer Persistenceconfiguration, AutomaticSwitchover (ASO) is disabled. Ifan array goes down (both con-trollers down) during the soft-ware update process, due to apower failure or other unexpect-ed event, hosts could lose accessto data until the failed array re-covers, or a manual switchoverof the affected volumes is done.

Peer PersistenceAutomaticSwitchover (ASO)is disabled duringsoftware update

System Manage-ment

AS-94649

No workaround is needed. Theservice recovers on its own.

In Automatic Switchover environ-ments, in rare instances, networkisolation of the Group Leaderand Backup Group Leader maycause the service that handlesthe automatic switchovers torestart unexpectedly.

Network isolationof the Group Lead-er and BackupGroup Leader arraymay lead to Auto-matic Switchoverservice restarts

System Manage-ment

AS-94683

Not applicableWhen performing group --check_takeover command onBackup Group Leader array in asynchronous replication group,the command may return thefollowing error: ERROR: The re-quest could not be understoodby the server

Check takeovercommand may re-turn generic errorwhen AutomaticSwitchover (ASO)is enabled

System Manage-ment

AS-123309

Not applicableThe Finite State Machine (FSM)handles different states for theAutomatic Switchover Service(ASD). If FSM is in process ofstartup and is requested toshutdown again, the heartbeatto ASD will not be available. Thiswill cause ASD to restart to re-cover from the missing heart-beat.

AutomaticSwitchover Servicemay restart unex-pectedly

System Manage-ment

AS-125613

NimbleOS 5.2.1.800 58



Not applicableThe array group CLI commandwith limits option (group --list-limits) displays numeric internalidentifiers as part of the informa-tion listed for the volume infor-mation. These numeric identi-fiers are used by the array onlyand can be ignored.

Group limits com-mand lists internalidentifiers

System Manage-ment

AS-101342


The Event service may restartunexpectedly if an invalid net-work interface entry is found inthe Scale Out Database (SODB).

Event Service mayrestart due to in-valid database en-try

System Manage-ment

AS-103275

Not applicableThe Array Management servicemay restart unexpectedly dueto an a memory allocation failurewhen attempting to synchronizeconfiguration with a member ar-ray. The restart of the ArrayManagement service clears thesituation.

Array Managementservice restarts dueto memory alloca-tion issue

System Manage-ment

AS-104812

If array is queried for statistics,then its frequency should be re-duced. This is specially relevantfor the setup where script orsome monitoring tool does statsquery frequently in a loop.

There is a memory leak in thestat (statistics) component ofthe Array Management service.Magnitude of the memory leakis proportional to the frequencyof stats query.

Array Managementservice restart dueto memory alloca-tion exception

System Manage-ment

AS-98297

Please contact HPE NimbleSupport.

Following a power outage, it ispossible that the Backup GroupLeader is not assigned to thegroup. This may occur if theSODB database does not startdue to an SSH key issue.

Backup GroupLeader is not as-signed due to pow-er outage

System Manage-ment

AS-99702

Not applicableThe Group Management servicemay restart while collectingstatistics from member array.This can occur when the requestfrom the group leader to mem-ber array exceeds timeout,causing the service to restart torecover.

Group Managementservice may restartwhile collectingmember arraystatistics

System Manage-ment

AS-93469

Manually delete the alarm fromthe command line interface. 1.Login to CLI 2. Find the alarm IDusing: alarm --list 3. Delete thealarm using the ID alarm --delete<ID>

If an IP address is migrated fromone interface port to another,the alarm that was generatedindicating the original interfacedown may not be cleared.

Alarm not clearedafter IP addressmoves to differentinterface

System Manage-ment

AS-92515

NimbleOS 5.2.1.800 59



The only workaround is to im-prove the quality of the networkconnection so that the loss ofTCP packets is minimized.

Service threads within the groupmanagement API handler arerestricted to run for no morethan 5 minutes. In a very lossynetwork, TCP throughput canbe throttled to almost nothingdue to retransmission timeoutback-off. In one observed case,where the API response wasabout 190 KB, the connectionmanaged to send only about 90KB before the 5 minute timeoutoccurred and caused the GroupManagement service restart.

A lossy networkcauses Group Man-agement servicerestart

System Manage-ment

AS-73595

The workaround would be todelete the alarm using the alarm--delete CLI.

When alarm for number of nicsin array net config reaching 120is triggered, it doesnt getcleared even when the numberof nics goes down.

Max limit of 120nics in netconfigalarm does not getcleared

System Manage-ment

AS-87749

Not applicableDue to an issue in POCO library,in certain cases, causing theGroup Management service torestart. The service restarts torecover from the condition.

During shutdownthe Group Manage-ment service mayunexpectedlyrestart

System Manage-ment

AS-112702

Not applicableArray Management Service mayrestart unexpectedly when try-ing to start or stop the GroupManagement Service. The sys-tem recovers automatically afterthe Array Management Serviceis restarted

Array ManagementService restartedwhile starting/stop-pingGroupManage-ment Services

System Manage-ment

AS-74556

Not applicableThere is a --force switch avail-able when deleting a perfor-mance policy via the HPE Nim-ble Storage Array CLI. This --force switch does not work andwill fail with the following: ER-ROR: Failed to delete perfor-mance policy. Resource busy.The --force command is notsupported since the specifiedperformance policy should notbe removed without first check-ing its volume or folder associa-tions.

Force deletion ofuser defined perfor-mance policyshould not be sup-ported

System Manage-ment

AS-74242

NimbleOS 5.2.1.800 60



Not applicableVolume move operations copyboth data and associated meta-data from the source array todestination array. For groupswith multi-array pools, copyingthe metadata can take a signifi-cant amount of time, and theestimate calculation may be inac-curate initially. These estimateswill auto-correct themselves byusing feedback mechanisms.

Volume Migrationestimates may beinaccurate whenmultiple volumesare being migrated

System Manage-ment

AS-101832

Not applicableBackup Group Leader (BGL)may be promoted and/or SystemManagement Service mayrestart. This occurs when thereis high load on either the GroupLeader array or the BGL array,or when there is networkthroughput issues between thearrays. The service restarts torecover from the condition.

Array managementservice may restartunder high load ornetwork connectivi-ty issues betweengroup arrays

System Manage-ment

AS-117683

Not applicableThe Array Management servicemay restart on a group config-ured for high availability duringvarious scenarios when GroupLeader to Backup Group Leadercommunication is high or degrad-ed. Scenarios which may causean array management servicerestart include degraded net-work connectivity between GLand BGL arrays.

Array managementservice may restartduring periods ofdegraded networkconnectivity be-tween high avail-able Group Leaderand Backup GroupLeader arrays.

System Manage-ment

AS-116625

Not applicableArray Management service mayrestart on a group configuredfor high availability during vari-ous scenarios such as whenGroup Leader to Backup GroupLeader communication is at highload or degraded. Scenarioswhich may cause an array man-agement service restart includedegraded or broken networkconnectivity between GL andBGL arrays.

Array managementservice may restarton Group Leader orBackup GroupLeader arrays

System Manage-ment

AS-105041

The service will stabilize on itsown following the restarting.

The Array Management servicemay restart unexpectedly whenthe array is under high workload.

Array ManagementService restarts un-expectedly underhigh load

System Manage-ment

AS-101420

NimbleOS 5.2.1.800 61



Not applicableGroup Management service mayrestart due to health checktimeout exceeded. The servicewill recover after the restart.

Group ManagementService may restartunexpectedly

System Manage-ment

AS-97697

Not applicableIf communication betweenGroup Management and Post-gres services does not completewithin expected timeout, theGroup Management service mayrestart. The restart will resumethe communication and try thetransaction again.

Group Managementservice may restartdue to communica-tion timeout exceed-ed

System Manage-ment

AS-97327

Not applicableArray Management services maybe unavailable for a short timedue to restart during automaticfailover.

Array Managementprocess may restartduring automaticfailover

System Manage-ment

AS-94835

Disable automatic failover andremove witness from configura-tion.

After Group migration is per-formed, backup group leaderbecomes group leader. If witnessis disconnected for more thanfive minutes, the new leader willnot remove quorum.

After group leadermigration, quorumis not removed af-ter witness discon-nection

System Manage-ment

AS-98434

If possible, issue the volcoll --handover command while nosnapshot collection(s) are beingcreated or replicated.

During handover internal checks,the Group Management Servicecan prematurely change theownership of the volcoll beinghanded over to the downstreamand all the volumes in the volcollbefore the volumes in the volcollare offlined on the upstream.This causes the service torestart due to the ownershipmismatch: the upstream stillconsiders itself the owner of thevolcoll and all the associatedvolumes since the volumes inthe volcoll have not been offlineon the upstream yet.

Group ManagementService may restartif volcoll handoverto downstreamhappens premature-ly

System Manage-ment

AS-117418

Update the downstream arrayto NimbleOS 3.4.x or later.

Encrypted and deduped vol-umes cannot be replicated from3.4.x and later to previous 3.xreleases because they do notsupport encryption and dedupeat the same time. Note that thisdoes not affect replication from3.4.x and later to 2.3.x where thevolumes will not be deduped onthe downstream running 2.3.x.

Replication ofdedupe and en-crypted volumeswill stall between3.4 and previous3.x releases

System Manage-ment

AS-57574

NimbleOS 5.2.1.800 62



Not applicableThe Group Management servicemay restart due to a race condi-tion between threads when up-dating replication records inScale Out Database (SODB). Theservice restarts to clear the con-dition.

Group ManagementService may restartdue to race condi-tion

System Manage-ment

AS-49720

Please reissue the command. Ifthe operation was already per-formed by the earlier command,an appropriate message will bereturned.

Under system busy conditions,when an excessive amount ofoperations are being issued inparallel or too many internal re-tries are occurring to performtasks, you may receive a Nomessage received error after is-suing a CLI command.

Error No messagereceived after issu-ing CLI commandto disassociate vol-ume from collection

System Manage-ment

AS-98378


Arrays with volumes that havelarge branch structures maycause internal command process-ing timeout to be exceeded. Thiswill cause the Group Manage-ment service to restart due tohealth check failure.

Group managementservice may restartunexpectedly

System Manage-ment

AS-98155

Not applicableInternal workflow processing re-lated to the replication partnerobject may cause the process todeadlock, resulting in Groupmanagement service restart.


System Manage-ment

AS-105064

Not applicableWhen the system has a highnumber of objects, the GroupManagement service may restartwhile running REST query or CLIcommand.

Group Managementservice may restartdue to high memo-ry usage

System Manage-ment

AS-96241

Not applicableNimbleOS uses a defined statemachine for the replicationworkflow. At the end of the exe-cution of each step defined inthe state machine, it moves tothe next step. If it leads to anyunexpected step throughout theworkflow then it will lead to as-sertion failure which results inGroup Management servicerestart.

Group managementservice may restartdue to assertionfailure

System Manage-ment

AS-96143

NimbleOS 5.2.1.800 63



Not applicableThe group --list_limits CLI com-mand does not list the Syn-chronous Replication volumecount Limit. Synchronous Repli-cation on 5.1.0.0 and later canprotect up to 128 volumes.

Synchronous Repli-cation VolumeCount Limit

System Manage-ment

AS-89124

Not applicableGroup management service mayrestart due to the race conditionbetween branch creation for thereplica volume and replica vol-ume deletion. This race condi-tion will cause an assertion fail-ure and lead to Group Manage-ment service restart. This mayimpact management workflows.

Group managementservice may restartdue to assertionfailure

System Manage-ment

AS-113950

Not applicableWhile syncing the configchanges on the downstream ar-ray, group management servicemay restart due to a race condi-tion between protection policydeletion and adding the volumeto the protection policy.

Group managementservice may restartdue to race condi-tion

System Manage-ment

AS-106490

Not applicableIn the data protection environ-ments, replication partners needto sync their configuration. Incertain situations, the partnerswill attempt to sync all volumecollections as opposed to justthe volume collections that wererecently modified. This may leadto unexpected delays is replica-tion partner sync.

Unexpected delaysin configurationsync between repli-cation partners

System Manage-ment

AS-115200

Not applicableIn rare occurrences, the GroupManagement may restart whena race occurs between threadsthat are disassociating a volumefrom a volume collection andmodifying a schedule for thesame collection. The servicerestarts to recover.


System Manage-ment

AS-81209

NimbleOS 5.2.1.800 64



Not applicableA deadlock can occur betweentwo management processeswhen updating the replicationbandwidth throttle. The Manage-ment Process will restart andclear the condition. There is nodisruption to data services andreplication will continue automat-ically without intervention.

Group managementprocess my restartdue to a deadlockwhen replicating

System Manage-ment

AS-59470

Not applicableIf a user performs multiple vol-ume collection handovers be-tween two arrays during a shorttime span, this may cause a situ-ation where both upstream anddownstream array may claimvolume collection ownership.This is due to a race conditionin the workflow.

Both upstream anddownstream mayclaim the volumecollection owner-ship when exces-sive handovers areperformed

System Manage-ment

AS-99520


Snapshots may display in GUI asreplicated snapshots when theyare not involved in replication.

Snapshots displayas replicated in GUIon array with noreplication config-ured

System Manage-ment

AS-73432

Not applicableIf two on-premise array groupsconfigured with a Nimble CloudVolumes (NCV) partner aremerged, the Cloud Volumereplication will stop working.

Cloud Volumesreplication does notsupport groupmerge

System Manage-ment

AS-88308

Not applicableIn the case where the down-stream array is reaching itssnapshot rate limit and the userperforms the volume collectionhandover, the handover will beabort if the limit is surpassed. Analert will be raised but the alertmessage may be missing thereason for aborting handover.

Alert for abortedhandover does notspecify reason

System Manage-ment

AS-98650

vol --list can be used to deter-mine pool/folder attributes ofthese volumes.

For the volcoll --info output forsync replication volume collec-tions, the Associated volumes:and Associated pinned volumes:fields lack pool/folder qualifica-tion for the associated volumes.

volcoll --info outputlacks pool/folderqualifications forassociated volumes

System Manage-ment

AS-90286

NimbleOS 5.2.1.800 65



Update the downstream poolsDeduplication setting to matchthe upstream pool.

If the Default Deduplication set-ting differs for upstream anddownstream pools, the configu-ration of deduplication volumesfor replication might fail with thefollowing error Deduplicationnot allowed since no applicationcategory is assigned to the per-formance policy

Configuration ofdeduplication vol-umes for sync repli-cation might fail

System Manage-ment

AS-90649

Please reissue the command. Ifthe operation was already per-formed by the earlier command,an appropriate message will bereturned.

Under system busy conditions,when an excessive amount ofoperations are being issued inparallel or too many internal re-tries are occurring to performtasks, you may receive a Nomessage received error after is-suing a CLI command.

Error No messagereceived after issu-ing CLI commandto associate volumeto volume collec-tion

System Manage-ment

AS-90633

Not applicableThe Group Management servicemay restart due to a race condi-tion between replication com-mands and group managementservice restart. The servicerestarts to recover from thecondition.

Group managementservice may restartdue to race condi-tion

System Manage-ment

AS-108119

Retry the command for thefailed volumes.

Due to a rare race condition, theGroup Management Service mayrestart unexpectedly during abulk volume update operation.

Group ManagementService restartsduring bulk volumeupdate

System Manage-ment

AS-95610

Not applicableDuring shutdown of the groupmanagement process, the SSLlibrary logging of the debugmessages can cause a race con-dition, resulting in a segmenta-tion violation error. The servicerestarts to recover from thecondition.

Group Managementservice may restartduring processshutdown due torace condition

System Manage-ment

AS-114179

Confirm all Domain Controllersin the Active Directory environ-ment that is integrated with thearray are reachable.

In environments with an arrayintegrated with Active Directory,the create/validate session codeholds a lock on an underlyingsessions table and then goes toActive Directory to collect moreinformation about the user. Ifthis operation takes too long,the Group Management servicemay timeout to free this lock.

Group Managementservice may restartwhile collecting us-er information fromActive Directory

System Manage-ment

AS-108868

NimbleOS 5.2.1.800 66



Not applicableDuring a planned array shut-down, the Group Managementservice may restart due a racecondition. There is no data ormanagement interruption sincethe service is already shuttingdown.

Group Managementservice restartsduring the arrayshutdown

System Manage-ment

AS-108765

Not applicableIn environments with largeamount of Fibre Channel ses-sions, the Group Managementservice may restart if the RESTAPI processing session informa-tion does not respond within theexpected timeout of 300 sec-onds. The service restarts to re-cover from the condition.

Group Managementservice may restartdue to health checktimeout

System Manage-ment

AS-125919

Avoid using the secondarymanagement IP to access thearray. Contact HPE Nimble Sup-port, supporting HPE Alletra6000 and Data Services CloudConsole if this has occurred.

If the secondary management IPin an array group is accessedusing a browser, this could causeexcessive logging which can leadto system partition filling up onthe backup group leader array.This may cause the backupgroup leader array to becomeunavailable.

Accessing sec-ondary manage-ment IP for BackupGroup Leader arraymay cause exces-sive log messages

System Manage-ment

AS-118301

Please review the network andsee if there is a consistent pack-et loss and fix any networkglitches. If you need any assis-tance, please contact HPE Nim-ble Support, supporting HPE Al-letra 6000 and Data ServicesCloud Console

If the network response to aREST request takes more than5 minutes, a thread performingthe REST request times out andas a result Group ManagementService restarts. The servicestabilizes itself and as long asthe network is serving the re-quests faster. A single instanceof the Group Management ser-vice restart should not cause anydisruptions.

Group ManagementService restarts dueto packet loss innetwork

System Manage-ment

AS-91638

Ensure setup completes success-fully before attempting to discov-er volumes.

If a user runs --resetup followedby setup on an array that waspreviously configured and setupdoes not complete successfully,data IPs may be unconfigureduntil the setup actually com-pletes successfully. As a result,GDD may restart unexpectedlyif any attempts are made to dis-cover the volumes on those un-configured data IPs.

Discovering vol-umes after arrayresetup may causeGroup Data servicerestart

System Manage-ment

AS-66182

NimbleOS 5.2.1.800 67



Not applicableDue to a race condition withsnapshot retention on a down-stream replication partner, theGroup Management service maycrash unexpectedly. The ProcessManagement service automati-cally restarts the Group Manage-ment service after the crash andit should resolve the race condi-tion.

Group Managementservice may restartdue to race condi-tion with snapshotretention on adownstream replica-tion partner

System Manage-ment

AS-117078

Not applicableIf communication betweenGroup Management and Post-gres services does not completewithin expected timeout, theGroup Management service mayrestart. The restart will resumethe communication and try thetransaction again.

Group Managementservice may restartdue to communica-tion timeout exceed-ed

System Manage-ment

AS-97899

Not applicableThe Group Management per-forms queries on the informationstored in the Scale Out Database(SODB) for the array. If thequery does not complete withinthe expected timeframe, theservice may restart to recoverfrom the issue.

Group Managementservice may restartdue to internaldatabase timeout

System Manage-ment

AS-95868

Not applicableIf clones are created using anunmanaged snapshot, then thisunmanaged snapshot will not bedeleted even if cleanup is en-abled.

Unmanaged snap-shots remain aftercleanup is enabled

System Manage-ment

AS-93113

Not applicableDue to a race condition, theGroup Management Service ona downstream group may restartwhile updating volume collec-tions from the upstream group.

Group ManagementService may restartdue to a race condi-tion

System Manage-ment

AS-105291

Not applicableGroup Management service mayrestart due to memory exhaus-tion in configurations that ap-proach 10,000 volumes and300,000 snapshots.

Group Managementservice may restartdue to memory ex-haustion

System Manage-ment

AS-94517

The TTL can be updated on thesnapshots which have a negativevalue to a current value. Thesnapshot may also be removedif it has been confirmed it is nolonger needed.

NimbleOS protects the lastreplicated collection, in somecases, the TTL expiry date onthose snapshots can becomenegative when the snapshotsexist beyond TTL.

Time to Live (TTL)expiry date on lastreplicated snap-shots can be nega-tive

System Manage-ment

AS-105944

NimbleOS 5.2.1.800 68



Contact HPE Nimble StorageSupport to identify ACLs inforced-delete/create-retry state,verify these ACLs are associatedwith snapshots that no longerexist, and delete these ACLsfrom the internal NimbleOSdatabase.

In certain scenarios, a volumedeletion will not complete dueto the existence of a stale ACLassociated with it. This stale ACLis associated with a snapshot ofthe volume that was previouslydeleted. These volumes will notshow up in the CLI/GUI as theyare in a hidden state.

Deletion of a vol-ume is not complet-ed due to the pres-ence of stale ACLsassociated with it

System Manage-ment

AS-105432


Arrays with SNMP get enabledmay encounter full /var partitionon the array if the SNMP logsfail to rotate. The log file thenbecomes large enough to fill thepartition. This may lead to soft-ware update precheck failures.

System partitionmay become full forarrays with SNMPenabled

System Manage-ment

AS-107656

Not applicableDuring planned service stop,such as array shutdown, theGroup Management service mayrestart due to a race condition.There is no data or managementinterruption since the service isalready shutting down.

Group Managementservice may restartduring array shut-down

System Manage-ment

AS-104965

Not applicableWhen internal threads encountera locking conflict, the GroupManagement service may restartto recover from the condition.

Group Managementservice may restartunexpectedly dueto a conflict with in-ternal threads

System Manage-ment

AS-62192

Not applicableAfter a controller restarts due toover temperature condition,System Management processesmay restart and take longer tostart up due to slow systemperformance.

System Processesor Systems maystart slowly afterover temperaturerestart

System Manage-ment

AS-115819

Not applicableAn Array Management servicemay be restart if unable to com-municate with internal databaseservices. The Array Managementservice restarts to recover fromthe condition.

Array managementservice may restartrestart unexpected-ly

System Manage-ment

AS-111431

Not applicableIf internal database processingfor array statistics exceeds theexpected timeout, the GroupManagement service will restartdue to health check failure torecover.


System Manage-ment

AS-98504

NimbleOS 5.2.1.800 69



Please contact HPE NimbleSupport, supporting HPE Alletra6000 and Data Services CloudConsole

When the array is powered onand off excessively, services mayfail to start on the array.

Services may notstart on the arrayafter it is poweredon and off severaltimes

System Manage-ment

AS-101392

Not applicableThe Group Management servicemay restart when service commu-nication for internal databaseprocessing is terminated. Theservice restarts to restore con-nections between the services.


System Manage-ment

AS-105453

Scheduling snapshots to occurat different times instead of allat once may help alleviate thisissue.

Group Management service mayrestart on the array when thereis a high amount of snapshotactivity being performed. Theservice restart will recover fromthe condition and the snapshotoperations will resume.

Group Managementservice may restartunexpectedly whenperforming highsnapshot activity

System Manage-ment

AS-105804

Not applicableArray Management service mayrestart on a group configuredfor high availability during vari-ous scenarios when GroupLeader to Backup Group Leadercommunication is high or degrad-ed. Scenarios which may causean array management servicerestart include degraded or lostnetwork connectivity betweenGL and BGL arrays.

Array managementservice may restartduring periods ofdegraded networkconnectivity be-tween Group Lead-er and BackupGroup Leader ar-rays.

System Manage-ment

AS-108378

Not applicableIn rare instances, the GroupManagement Service mayrestart unexpectedly when theGroup Leader and Member arrayhave lost connectivity due tonetwork outage. The servicerestart recovers GUI and CLI ac-cess, data services are not im-pacted by the restart.

Group ManagementService may restartunexpectedly dueto network connec-tivity

System Manage-ment

AS-103982

Not applicableThe Array Management Servicerestarts unexpectedly followingautomatic Group Leader Failover(AFO). The restart is non-disrup-tive.

Array ManagementService restarts un-expectedly follow-ing automaticGroup LeaderFailover

System Manage-ment

AS-99615

NimbleOS 5.2.1.800 70



Not applicableIn rare instances, the ProcessManagement service restart mayoccur during software update.The system recovers after therestart of the service.

Process Manage-ment service mayrestart during soft-ware upgrade

System Manage-ment

AS-95132

Not applicableThe Group Management servicemay restart while synchronizinginformation between the GroupLeader and Backup GroupLeader arrays. This may be morecommon with high system load.The service restarts to recoverfrom the condition.

Group managementservice may restartunder high load

System Manage-ment

AS-91627

Not applicableThe Group Management servicewill make updates to the internalScale Out Database (SODB) forthe array. If the update is unsuc-cessful, the service may restartto successfully apply the update.

Group Managementservice may restartduring internaldatabase update

System Manage-ment

AS-108740

Not applicableThe Group Management Servicemay restart unexpectedly whenthe array is under heavy load,has many snapshots scheduled,has performed a group merge inthe past, and has recently per-formed an automatic GroupLeader Failover.

Group ManagementService restarts un-expectedly follow-ing automaticGroup LeaderFailover

System Manage-ment

AS-100382

No workaround available. Toavoid encountering this issue,reduce IO load when performingsoftware update. Software Up-date will succeed after one ormore System Management ser-vice restarts.

The underlying scale-outdatabase competes with CASLand other system processes forIOPS. During software update, amigration script runs against thedatabase. Under heavy file sys-tem load, the migration stepsmay not complete within theexpected amount of time. As aresult, the migration may time-out leading to a restart of theGroup Data Service. After therestart, the migration shouldcomplete as normal without anyuser impact or intervention.

Group Data Servicemay restart whenthe array is underheavy load duringsoftware update

System Manage-ment

AS-110374

NimbleOS 5.2.1.800 71



Not applicableDuring the periodic updateswhere the Group ManagmentService refreshes the space us-age data for the group, if thereis a communication failure be-tween the group leader arrayand the backup group leader ar-ray, the service will restart. Theservice will stabilize once commu-nication is restored between thegroup leader and backup groupleader.

Group ManagementService may restartduring space up-date operations

System Manage-ment

AS-120145

Confirm the usage level for vol-ume or pool indicated by thealarm is below threshold. Onceconfirmed usage level is belowthreshold, clear the alarm fromthe Command Line Interface(CLI): 1. List the alarms to findthe alert ID alarm --list 2. Deletethe alarm that is no longer validalarm --delete

Alarms are generated on the ar-ray when volume or pools ex-ceed thresholds. In some in-stances the recovery event toclear the alarm when the condi-tion is cleared does not clear thealarm.

Alarm not clearedafter volume orpool drops belowwarning threshold

System Manage-ment

AS-77045

The alarms can be deleted man-ually either in the GUI or on theCLI.

After the update to 5.1.x.x orlater, the Snapshot limit warningalarm is no longer used. Thispresents a situation where stalealarms are present on the arrayand they will not be cleared evenif the space situation is rectified.The alarm follows the followingformat: WARNING Mon DDYYYY HH:MM:SS Acknowledged- Volume <volume name>snapshot space usage is overthe configured warning limit.

Snapshot limitwarning alarmspersist after updateto 5.1.x.x or later

System Manage-ment

AS-98694

Not applicableA Group Management servicemay restart or become unavail-able for brief periods of time.Data Service is not affected. Theservice restarts to recover fromthe condition.

Group Managementservice may restartor become brieflyunavailable.

System Manage-ment

AS-120170

Not applicableInfrequently the Group Manage-ment service may encounter anerror while shutting down. If theservice is being restarted it maytake a few more seconds tostart. GUI and CLI will be unavail-able for a few seconds.

Group Managementservice may be-come unavailablewhen shut down

System Manage-ment

AS-110123

NimbleOS 5.2.1.800 72



Not applicableHigh concurrent REST volumereads with statistics may causeGroup Management service torestart. Current REST requestswill fail, GUI and CLI will be un-available. The system will recov-er after the restart of the service.

Group managementservice may restartwhen there aremany REST re-quests for volumestatistics values

System Manage-ment

AS-108432

Not applicableAn Array Management servicemay restart or become unavail-able for brief periods of time.This may be more common withSNMP activity. This may be seenwhen reading statistics via SN-MP. Data Service is not affected.Array Management service willbe restored automatically.

Array Managementservice may restartor become brieflyunavailable

System Manage-ment

AS-120279

If ICMP is disabled for the arraynetwork, please contact HPENimble Storage Support.

The array will attempt to pingthe server update.nimblestor-age.com during the updateprecheck process. If ICMP is dis-abled or the ping is unsuccessful,the update precheck will fail.

Software updateprecheck may fail ifICMP is dis-abled/blocked forthe array interfaces

System Manage-ment

AS-115896

The ASO checkbox is enabledby default, however ASO is notenabled until a witness has beenconfigured. In the GUI, navigateto Administration > Availabil-ity. If witness is configured andthe ASO check box is checked,disable ASO by unchecking thebox and clicking save. Performthe array software update again.If the update continues to failwith generic messaging, contactHPE Nimble Support.

Software updates to 5.1.4.200are not allowed when AutomaticSwitchover (ASO) is configured.If a software update to 5.1.4.200fails for this reason, a genericsoftware update failure messageis returned in the GUI. The causeof the failure would need to bedetermined by looking at thesystem configuration and deter-mining if ASO is configured.

Arrays with Auto-matic Switchoverenabled fail soft-ware update withgeneric message

System Manage-ment

AS-106848

Running the software --re-sume_update command from theconsole will clear this condition.

Under rare conditions, a soft-ware update may report an erroreven though the actual updatehas completed successfully. Thisoccurs when software updatetakes longer than 4 hours.

Timeouts duringsoftware update

System Manage-ment

AS-40516

Not applicableGroup management service mayrestart during software updatedue to race condition involvingunlocking the download lock file.

Group managementservice may restartduring software up-date

System Manage-ment

AS-72559

NimbleOS 5.2.1.800 73



Ignore the error message andretry the management operation

While the volume ownership isbeing changed, any manage-ment operation involving thatvolume could fail. This is expect-ed behavior. Depending on theprogress of that handover oper-ation, management operation isfailing with different errors.

Volume manage-ment operationsmay fail with unre-lated error mes-sages while han-dover is in-progress

System Manage-ment

AS-91962

Not applicableGroup Management Servicemight restart unexpectedly whilecreating Peer Persistent snap-shot during Daylight SavingTime adjustment window.

Group ManagementService may restartunexpectedly dur-ing creation of PeerPersistent Snap-shot

System Manage-ment

AS-92209

Not applicableThe Group Management Serviceon the array may restart whenthe winbind component at-tempts to communicate with anActive Directory Domain Con-troller that is unreachable.

Group ManagementService restarts un-expectedly whenattempting to com-municate with Ac-tive Directory

System Manage-ment

AS-121255

As a local admin on the array,restart the Active Directory inte-gration service by disabling andenabling the integration. Via theGUI: Navigate to Administration-> Security -> MicrosoftActive Directory Click DisableClick Enable Via the CLI: user-auth --disable <domain>userauth --enable <do-main>

The expected configuration isntloaded when the Active Directo-ry integration service starts,leading to failed logins or incon-sistent Active Directory connec-tivity in certain configurations.

Active Directoryconnection failures

System Manage-ment

AS-111166

Not applicableActive Directory Authenticationcauses the arrays Group manage-ment service to wait for a re-sponse from the Active Directo-ry. If this response is delayed,the Group Management Servicemay restart unexpectedly.

Active Directoryauthentication insome cases, maylead to a GroupManagement ser-vice restart

System Manage-ment

AS-104512


In rare instances, if the array isunder heavy load and GroupManagement service restartswhile new user is being created,the service can fail to start.

Group Managementservice may crashwhile trying to cre-ate users while ar-ray is under heavyload

System Manage-ment

AS-105035

NimbleOS 5.2.1.800 74



Add the computer account cre-dentials for the array by addingit to the RODCs password repli-cation policy (PRP) under theAllowed RODC Password Repli-cation Group group.

Array authentication to an Ac-tive Directory Read-Only Do-main Controller (RODC) may fail,resulting in failures to authenti-cate Active Directory users.

Active Directoryauthentication mayfail with Read-OnlyDomain Controller(RODC)

System Manage-ment

AS-107491

Not applicableIn rare instances, the GroupManagement Service mayrestart while a user is beingdeleted. The service restarts torecover from the condition.


System Manage-ment

AS-114105

Confirm all Domain Controllersin the Active Directory environ-ment that is integrated with thearray are reachable.

In environments with an arrayintegrated with Active Directory,the create/validate session codeholds a lock on an underlyingsessions table and then goes toActive Directory to collect moreinformation about the user. Ifthis operation takes too long,the Group Management servicemay timeout to free this lock.

Group Man-agement servicemay restart whilecollecting user infor-mation from ActiveDirectory

System Manage-ment

AS-113885

Not applicableThe Group Management servicemay restart when internaldatabase processing exceedsthe expected timeout value. Theservice restarts to recover fromthe condition.

Group Managementservice may restartdue to health checktimeout

System Manage-ment

AS-109805

Not applicableThe Group Management servicemay restart when user lookup isin process and the winbinddservice does not respond withinthe expected timeframe. Theservice restarts to recover fromthe condition.

Group Managementservice may restartdue to healthchecktimeout

System Manage-ment

AS-115002

Not applicableThe Group Management Servicemay restart unexpectedly if ittakes longer than 300 secondsto authenticate an Active Direc-tory (AD) user.

Group ManagementService restartswhile authenticat-ing AD users

System Manage-ment

AS-71137

Not applicableIf collection of user group infor-mation from Active Directorytakes longer than expected, theGroup Management servicetimeout may be exceeded. Theservice will restart to resume in-formation collection.

Group Managementservice may restartwhile collecting us-er information fromActive Directory

System Manage-ment

AS-69084

NimbleOS 5.2.1.800 75



Not applicableCommand to join AD may failbecause of latency in gettingback a response from AD server.This may cause a health checkfailure for Group Managementcausing the restart of the serviceto recover.

Command to joinActive Directory(AD) may fail caus-ing Group Manage-ment serviceRestart

System Manage-ment

AS-66437

Not applicableWhen the HPE Nimble Storagearray is configured to use ActiveDirectory integration, the arrayis joined to one specific domain,as a domain member. Undernormal circumstances, users intrusted domains will also be ableto authenticate to the array. Ifone or more trusted domains arejoined to the forest using an MITKerberos type trust relationship,users and groups in any trusteddomain (e.g. not the domain thearray is joined to) will be unableto authenticate to the array.

HPE Nimble Stor-age array compati-bility issues withMIT Kerberos trusttypes

System Manage-ment

AS-95212

Resolve any connectivity issuesbetween the array and ActiveDirectory Domain Controller(s)to avoid unexpected servicerestarts.

If there are connectivity issuesbetween an array with ActiveDirectory integration enabledand an Active Directory DomainController, then the responsesmay take more than 300 sec-onds and timeout. As a result,the Group Management servicemay restart.

Active Directoryconnectivity issuemay result in GroupManagement ser-vice restart.

System Manage-ment

AS-65654

Not applicableWhen joining Active Directoryvia GUI or userauth command inCLI, the command is expectedto complete within 300 seconds.If this timeout is exceeded dur-ing the process, the Group Man-agement service will restart torecover.

Group Managementservice may restartwhen joining arrayto Active Directory

System Manage-ment

AS-74830

Not applicableAD communication may be slowleading to a delay in responsesthat may cause Group Manage-ment service to crash. Leavingthe AD domain requires interac-tion with AD wherein this delaycan cause a restart.

Latency in commu-nicating with Activedirectory duringmay cause GroupManagement ser-vice to restart

System Manage-ment

AS-78946

NimbleOS 5.2.1.800 76



Not applicableThe Group Management servicemay restart when an assertionis encountered during a volumeupdate. The service will restartto recover from the conditionand complete the volume up-date.

Group Managementservice may restartduring volume up-date

System Manage-ment

AS-122355

The Group Management Servicewill eventually restart itself

Enabling and disabling thededupe setting on volumes andconcurrently deleting volumescan cause the Group Manage-ment Services to become tem-porarily unavailable on the array.

Group ManagementService is temporar-ily unavailable afterdeleting volumes

System Manage-ment

AS-101535

Delete unused clones or volumesto bring down the volume count.

Following operations will failupon reaching the volume limit:-adding Synchronous replicationschedule to a volume collection -associating a volume toa volume collection with Syn-chronous Replication enabled -editing a Volume Collectionschedule to add SynchronousReplication partner

Enabling syn-chronous replica-tion fails uponreaching volumelimit

System Manage-ment

AS-102893

Please contact HPE NimbleSupport, supporting HPE Alletra6000 and Data Services CloudConsole to update the compres-sion algorithms of these oldervolumes.

After updating to NimbleOS 4.x,volumes initially created whenan array was running NimbleOS1.3.x and earlier may experiencea negative impact on perfor-mance. This is caused by soft-ware changes made to the com-pression algorithm used bythese older volumes.

Performance issuesfor volumes createdwhen array wasrunning NimbleOSversions 1.3.x andearlier

System Manage-ment

AS-84276

Once the array is able to deter-mine its deduplication capability,all newly created volumes willhave dedupe enabled, if speci-fied. In order to enable dedupeon the previously created vol-umes, you may run the followingcommand via the HPE NimbleStorage Array CLI: vol --edit<vol_name> --dedupe_en-abled yes

After a CSx000 array is installed,it takes one minute for the arrayto determine its deduplicationcapability. If a volume is createdprior to this, it will not havededupe enabled even if the ar-ray is dedupe capable.

Unable to creatededupe enabledvolumes on a newinstall

System Manage-ment

AS-86545


In the case of a large number ofFC sessions, a volume read RESTrequest can lead to high memoryusage which can cause groupmanagement service restart.

Group managementservice might getrestarted due tohigh memory usage

System Manage-ment

AS-127173

NimbleOS 5.2.1.800 77



Not applicableCurrently, there is no CLI sup-port for changing the witnessport. The nimble-witnessd.ser-vice file needs to be editedmanually.

No CLI support forchanging the Wit-ness Port

System Manage-ment

AS-92157

Not applicableA member array might not belisted under the Add Array toGroup option within the GUI ifthe member is configured witha different protocol (iSCSI vs Fi-bre Channel). Also when thereare multiple arrays in the subnet,arrays which cant be discoveredwithin the stipulated time maynot be listed in Add Array toGroup.

Member arraymight not be dis-played under AddArray to Group op-tion

System Manage-ment

AS-100067

Not applicableIn rare instances, the Group Dataservice may restart duringstartup due to a misconfigura-tion in NimbleOS pertaining toa file descriptor limit.

Group Data servicemay restart onstartup

System Manage-ment

AS-107015

Not applicableIn rare circumstances, followingan Automatic Failover (AFO) arace condition may cause theArray Management Service torestart or an unexpected con-troller takeover.

Array ManagementService restarts orTakeover occursunexpectedly fol-lowing automaticGroup LeaderFailover

System Manage-ment

AS-99431

Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consoleto recover the array from thiscondition.

In rare circumstances, a powerfailure can result in inconsistentconfiguration information ondisk, preventing the array fromfully restarting and serving data.No user data nor configurationsettings are lost.

Power failure of ar-ray can lead to anerror condition thatprevents the arrayfrom fully restart-ing and serving da-ta

System Manage-ment

AS-116337

If the Unknown error is encoun-tered, select OK. When the Dis-card Changes dialog appears,select Cancel. The Date and timewill be saved, refresh the pageto confirm the changes.

After updating the date andtime on the Date/Timezone Ad-ministration page of the GUImay report an Unknown Error.When OK is clicked, another dia-log may appear asking to Dis-card Changes.

Date/Timezonepage of GUI mayreport errors onsave

System Manage-ment

AS-117508

NimbleOS 5.2.1.800 78



Contact HPE Nimble Support,supporting HPE Alletra 6000and Data Services Cloud Consoleto clear the configuration.

If invalid settings are configuredin the SMTP configuration forthe array, the GUI will not allowthe configuration to be removedand saved without any configu-ration.

SMTP Configura-tion cannot becleared from theGUI

System Manage-ment

AS-104383

Not applicableWithin the HPE Nimble Storagearray GUI, custom SSL certificateimport is only supported onGoogle Chrome version 71 orlater.

Custom SSL certifi-cate import notsupported on olderversions of GoogleChrome

System Manage-ment

AS-99343

To keep the same reminder fre-quency and reset the next re-minder time based on the cur-rent time, change the reminderfrequency to a different value,save it, and change it back andsave it, or use CLI to make thechange.

When updating an alarm fromthe Events > Alarms page inGUI, selecting an alarm andclicking CHANGE REMINDERbutton, without changing thereminder frequency time, andclicking SAVE button, does notchange next reminder time. Thisbehavior is different from CLI.Setting alarm reminder frequen-cy to the same value from CLIresets the next reminder timebased on the current time.

Setting alarm re-minder frequencyto the same valuefrom GUI does notchange next re-minder time

System Manage-ment

AS-98177

When the controller is back up,all the information is displayedcorrectly on hardware page.

When a controller is down, theuser may see incorrect represen-tation of physical ports withinthe Hardware Page of the arrayGUI. This is due to the lack ofinformation from the missingcontroller.

Incorrect informa-tion on hardwarepage displayedwhen controller isdown

System Manage-ment

AS-87701

NimbleOS 5.2.1.800 79



The group merge and interfaceassignment can be completedusing the Command Line Inter-face (CLI) or REST API as aworkaround. From CLI on thetarget array: array --add<sourcearrayname>

During add array, when you tryto configure the subnets on inter-faces for the incoming array, ifthe subnet has VLANs it shouldnot mark the port as used unlessboth tagged and untagged as-signment is done for the inter-face. The interface should bereusable for multiple VLANs.This works from the CLI &REST API but just not from GUIIf the subnets for interfaces onthe target (destination) arrayhave VLANs configured, the GUIwill not display them as availablefor assignment on the source(incoming) array during arrayadd. If VLANs are in use, the in-terface should be reusable formultiple VLANs.

Unable to add arrayto group using GUIwith VLAN assign-ments

System Manage-ment

AS-87505

Use Google Chrome or MozillaFirefox browser.

The Array GUI incorrectly re-turns an error when a valid valuefor the folder overdraft limit hasbeen entered. This happens onlyin Internet Explorer and Mi-crosoft Edge browsers.

GUI Error when en-tering a valid folderoverdraft limit val-ue

System Manage-ment

AS-102299

To work around this issue: 1.Log into the destination arrayto resolve the conflicts. 2. At-tempt the group merge again.

Currently within the HPE NimbleStorage Array GUI, when per-forming a group merge, if thereis a large amount of groupmerge conflicts (1000 or more),the GUI is unable to process andresolve all of them.

Group Merge viaGUI unable to pro-cess large amountof conflicts

System Manage-ment

AS-77372

Not applicableDuring group merge, the GUImight show Successful messageeven though the group mergebackend processing fails.

GUI may show Suc-cessful messagewhen group mergefails

System Manage-ment

AS-87886

Please contact HPE NimbleStorage Support.

When adding a Fibre Channel(FC) array to an existing group,you are unable to change themgmt subnet to Mgmt + Data.You will see the following error:Data subnet Management<SUBNET> must allowgroup traffic

Unable to assigngroup traffic to Fi-bre Channel arraywhen joining arraygroup

System Manage-ment

AS-76896

NimbleOS 5.2.1.800 80



Not applicablePool merge is not allowed ifSynchronous Replication is en-abled and pool merge is not al-lowed when a witness config-ured. If an array group has awitness configured for Automat-ic Switchover and has Syn-chronous Replication configured,when a user tries to perform apool merge, the following erroris generated: “pool merge is notallowed when witness is config-ured”. In this case, if the user re-moves the witness and then re-attempts the pool merge, thefollowing error is then generat-ed: “Pool merge is not allowedwhen involved in sync replica-tion”. This error should super-sede the previous error.

Incorrect orderingof pool merge errormessages whenSynchronous Repli-cation and Witnessare configured

System Manage-ment

AS-95591

Edit the pool name and assign /un-assign the array a in separatesteps.

When attempting to edit a stor-age pool and assign an array atthe same time, you receive thefollowing error: Cannot updatearray list and name or descrip-tion simultaneously.

Unable to edit astorage pool andassign an array atthe same time

System Manage-ment

AS-94575

Fix the name conflict on thedownstream array.

When associating multiple vol-umes to a volume collectionfrom, the volume association forall volumes can fail due to aname conflict for one of the vol-umes on the downstream.

Volume collectionassociation for avolume can fail dueto a name conflicton the downstreamarray

System Manage-ment

AS-104099

Run the following command viaCLI: snap --list --all --unmanaged

The Array GUI does not specifywhich snapshots are unmanagedand no longer belong to a vol-ume collection.

Array GUI does notspecify which snap-shots are unman-aged

System Manage-ment

AS-93157

After a new custom certificatehas been imported or existingcertificate is deleted, pleaseclose the browser where the ac-tion was performed and reopena new one to guarantee a newconnection request to the Nim-bleOS web interface.

After changing a certificate, theGUI may present an error suchas follows: The web service isvery slow or unreachable...

Browser becomesunstable upon cer-tificate change

System Manage-ment

AS-99024

NimbleOS 5.2.1.800 81



The values should report correct-ly within 24 hours after the up-date has completed.

The volume performance num-bers displayed in the GUI underManage > Data Storage >Volumes > Performance Tabmay display invalid values tem-porarily after an array softwareupdate.

Volume perfor-mance numbersmay report invalidvalues after soft-ware update

System Manage-ment

AS-92634

Not applicableIf the array Data Service is en-countering memory exhaustionwhile the ConfigDB encountersmemory allocation errors, theservice will restart to recoverfrom the condition.

Data Service mayrestart due to Con-figDB allocation er-rors in low memoryscenarios


Not applicableWhen the Group Leader at-tempts to complete the BackupGroup Leader promotion, if thereis not a healthy data path, theBackup Group Leader promotionfails. Despite, the network error,the Backup Group Leader promo-tion goes into a loop and ulti-mately leads to an unexpectedrestart of the Array Manage-ment Service.

Array ManagementService restartswhen Group Leadercannot reach Back-up Group Leader


There is no workaround. Toavoid encountering this issue,reduce IO load when performingsoftware update.

The underlying scale-outdatabase competes with CASLand other system processes forIOPS. During software update, amigration script runs against thedatabase. Under heavy file sys-tem load, the migration stepsmay not complete within theexpected amount of time. As aresult, the migration may time-out leading to a restart of theGroup Data Service. After therestart, the migration shouldeventually complete as normalwithout any user impact or inter-vention.

Group Data Servicemay restart whenthe array is underheavy load duringsoftware update


Not applicableThe File System service mayrestart during array shutdown ifit exceeds expected time toshutdown. The service restartsto clear the condition.

File System servicemay restart duringarray shutdown


NimbleOS 5.2.1.800 82