module 4 install trouble shooting rev_1.1
TRANSCRIPT
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
1/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 1
2009 EMC Corporation. All rights reserved.
V-Max Series Installation TroubleshootingV-Max Series Installation Troubleshooting
V-Max Series Installation Troubleshooting
Welcome to Installation Troubleshooting module.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
2/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 2
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 2
Module Objectives
Upon successful completion of this module, students
should be able to: Identify common installation failures
Resolve issues to ensure completion of installation and power-up ofmachine
Document customer power connections
Document installation issues.
After successfully completing this module, you should be able to:
Identify common installation failures
Resolve issues to ensure completion of installation and power-up of machine
Document customer power connections Document installation issues.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
3/37
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
4/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 4
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 4
Cabling: Loop Issues
Prior to cabling verification, DA Link verification is
performed.
Prior to cabling verification, a DA Link Verification is performed. If the links are not active, the
script can not continue. Link errors must be fixed before the script can continue and the cable
verification can begin.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
5/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 5
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 5
Cabling: Back End Cable Verification
Backend Cabling Verification script
Included in Configure and Install New Symmetrix script
Steps through a light sequence for valid connections from DA Portsto DAEs cables may be moved
Separate Backend Cabling Verification script available for datacenter relocations, moves, etc. cables must NOT be moved
The Backend Cabling Verification script (available as a standalone script and also included in the
Configure and Install New Symmetrix script) will check for Backend cabling errors, and an error
report will detail the errors. Running the Backend Cabling Verification script as a standalone is
available for data center relocations, machine moves, etc. Do not attempt to move or disconnect
cables during this light test when live data is involved; contact PSE Lab for assistance. The
standalone script can be found in SymmWin under Procedures Procedure Wizard System
Initialization Tools Backend Cabling Verification.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
6/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 6
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 6
Cabling: Error Log
If backend cabling errors, script halts + display Cabling.logwith errors
Errors mustbe fixed before script can continue
If errors occur, the script will halt and display a log file listing the errors. All errors must be fixed
before the script can continue. Do no disconnect or move cables during the test when live data is
involved; please contact the PSE lab for assistance for any issues concerning backend cabling with
live data.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
7/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 7
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 7
Cabling: V-Max Power Connections
Installation Manual Worksheet
Document customers PDU and circuit breaker numbers for every bay inthe system
Use this info to populate the PDU label
* See Installation Manual for Power Configuration Worksheets
Each bay in the V-Max has two power zones, A & B. Each power zone needs to be connected to
different PDUs at the customer site. Not having proper power connections leaves the Symmetrix in a
vulnerable state, and will cause the machine to vault and go to an offline state and remain that way
until sufficient power can be restored to the unit. The Power Connection Worksheet for the V-Max
System is shown above, and can be found in the Installation Manual. Complete this Worksheet to
document the customers power setup for each installed V-Max.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
8/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 8
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 8
Cabling: V-Max SE Power Connections
Installation Manual Worksheet
Document customers PDU andPDU Circuit Breaker numbers forevery bay in the system
* See Installation Manual for Power Configuration Worksheets
Each bay in the V-Max has two power zones, A & B. Each power zone needs to be connected to
different PDUs at the customer site. Not having proper power connections leaves the Symmetrix in a
vulnerable state, and will cause the machine to vault and go to an offline state and remain that way
until sufficient power can be restored to the unit. The Power Connection Worksheet for the V-Max
Model SE System is shown above, and can be found in the Installation Manual. Complete this
Worksheet to document the customers power setup for each installed V-Max.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
9/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 9
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 9
Cabling: Storage Bay Power Connections
Visually inspect internalconnections to verify properconnections. Zone A cables are Gray
Zone B cables are Black
System Bay PDU Zone A to A SPSs
PDU Zone B to B SPSs
SPS Zones A & B to MIBE
SPS Zones A & B to V-Max Engines
SPS Zones A & B to UPS
UPS to KVM, Server and Modem if
used Storage Bay
PDU Zone A to A SPSs
PDU Zone B to B SPSs
SPS A to DAE A PSs
SPS B to DAE B PSs
Storage Bay Rear View
Power Connections
Visually inspect all internal power connections for both Zones. Internal (PDU to SPS; SPS to PS,
etc.) Power Zone A cables are Gray, Power Zone B cables are Black. Zone A components should be
powered by the Zone A SPSs; Zone B components should be powered by the Zone B SPSs. The
MIBE, UPS and V-Max Engines are powered by both Zones for redundancy.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
10/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 10
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 10
Cabling: Power Cable Verification
Power Cabling Verification Script
Included in Configure and Install New Symmetrix script Steps through a sequence of Powering on and off Zone-specific PDP
Switches for all existing bays to verify proper power distribution
Separate Power Cabling Verification script available for relocation, moves,etc.
Within the Configure and Install New Symmetrix script, there is a power check sequence which must
be run for all new installs to verify internal power connections are correct and valid. There is also a
standalone Power Cabling Verification script available for data center relocations, machine moves,
etc. Should an error condition exist, the script will halt, and the issue must be resolved before
continuing.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
11/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 11
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 11
Cabling: SPS Monitoring System Bay
System Bay V-Max Engine 4
Management Module to SPS
Management Module to UPS
All other System Bay V-MaxEngines
Management Module to SPS
SymmWin Environmental Screen
Run Health Check to verify properconnection to all FRUs for
reporting purposes.System Bay: Engine 4 Rear View
Environmental cabling must be verified for monitoring SPSs and UPS. Visual inspection of cabling
in addition to using the SymmWin Environmentals Health Check must be done before the system can
be brought online to hosts.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
12/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 12
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 12
Cabling: SPS Monitoring Storage Bay
Storage Bay
LCC to SPS cables SymmWin Environmental Screen
Run Health Check to verify properconnection to all FRUs forreporting purposes.
DAE
16
DAE
15
DAE
14
DAE
13
DAE
4
DAE
3
DAE
2
DAE
1
DAE
8
DAE
7
DAE
6
DAE
5
DAE
12
DAE
11
DAE
10
DAE
9
SPS
2B
SPS
2A
SPS
1B
SPS
1A
SPS
3B
SPS
3A
SPS
4B
SPS
4A
LCC-
B
LCC-A
LCC-
B
LCC-A
LCC-
B
LCC-A
LCC-B
LCC-A
D C B A
0
1
0
1
Storage Bay: Rear View
Environmental cabling must be verified for SPSs. Visual inspection of cabling in addition to using
the SymmWin Environmentals Health Check must be done before the system can be brought online
to hosts.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
13/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 13
2009 EMC Corporation. All rights reserved.
DMX-3&4 Series Installation TroubleshootingDMX-3&4 Series Installation Troubleshooting
DMX-3&4 Series Installation Troubleshooting
Welcome to Installation Troubleshooting module.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
14/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 14
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 14
DMX-3 & 4 Cable Diagnostics
Upon completion of this lesson, you will be able to:
Run the diagnostic software
Follow the diagnostic sequence
Verify the physical location using the light sequence
Use the Cable Verification Report
The objectives for this lesson are shown here. Please take a moment to review them.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
15/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 15
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 15
Location of Cable Diagnostic Script
CableVerification
Script
The Cable Diagnostic script is under the Procedure Wizard, DAE Maintenance Utilities. Cable
Diagnostics are also included in Config and Install New Symmetrix script. The cable verification script
can be used on both DMX-3 & 4 24-slot and 6-slot models.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
16/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 16
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 16
Cabling Verification Tool
The Cabling Verification Tool checks the connections between the DAs and DAEs and can be used at
any time. Configuration source is selected during the procedure.
Note: Cables should never be disconnected on a live System.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
17/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 17
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 17
.Scan Step
The scan step issues commands to the Link CPU directly and looks for actual connections to DAEs.
Look at the configuration just for verification of # of DAEs per Port.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
18/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 18
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 18
Cabling Light Sequence
Confirm Single Step: Y/N
The Light Sequence is the only way we have to verify physical location.
Turns columns then rows in each bay.
Columns are turned on by lower directors.
Rows are turned on by the Dual Initiator directors.
Single Step or not
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
19/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 19
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 19
Bay Sequence
Light Sequence order for existing bays is:
Side A 2A, 1A Side B 1B, 2B
Side A Side B
The light sequence completes Side A (to the left of the system from the front) and continues to the B
side (left of the system from the front). In the case of a 10 storage bay system the light sequence order
would be 5A-1A then 1B-5B.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
20/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 20
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 20
Light Sequence Bay Progression
Column
For each bay starting with 1A, the columns connected correctly display a light.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
21/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 21
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 21
Light Sequence Bay Progression - Row
Row
Starting on the bottom and working toward the top, you have the same choices to repeat or to continue
to the next row.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
22/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 22
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 22
Light Sequence Bay Progression - Next Bay
Column
Bay 2A
In this example, a fully configured DMX-3 & 4 2500 is shown. The script runs first on the A side (Bay
1A), as this is the first Mohawk cabinet installed. It then goes to the next bay on the A side if
connected (Bay 2A), if not, it would jump over to the first Mohawk on the B side (Bay 1B).
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
23/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 23
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 23
Accessing Cabling.Log file
Cable verification log only appears if there is an issue. To view the report, in Symwin:
Under Tools, choose View logfiles and select cabling.log in the log browser.Log files can also be found at 0:\EMC\\logs\symmwin.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
24/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 24
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 24
Cable Verification Report
In the header of the Cable Verification Report, system information is displayed. If errors are present, a
list displays.
This report also shows the set of DAEs connected to paired director ports.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
25/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 25
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 25
DMX-3 & 4 6-Slot Cable Verification
Within SymmWin Environmentals tab, Cable check
for DMX950 (Caracal)
Cable verification for the DMX-3 & 4 950 is run through the Environmentals screen in SymmWin.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
26/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 26
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 26
Lesson 5: Error Example
Upon completion of this lesson, you will be able to
identify, understand and respond to errors during thecable diagnostics.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
27/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 27
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 27
Error Example - Gray or No LED
DAE 4LED off
In this slide, SymmWin indicates that there is no logical connection to DAE 4.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
28/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 28
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 28
Error Example - Error Indication
Error Indications
This screen indicates an error with DA 1d, port 0. (See next slide for additional info)
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
29/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 29
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 29
Error Indications (continued)
Error Indications
After scrolling to the bottom of the report, we can see that Director 16d port 0 has a valid connection,
but Director 1d port 0 does not (shown with an X in the primary column and Disconnected! in the
display).
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
30/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 30
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 30
DA Loop, Cabling and Drive Check steps
If there is a loop issue the script will loop from step 30
back to 2D until the loop issues are fixed. Once the loops are active and alive then the cabling
check will start.
The Config and install cabling check is exactly the sameas the Online cabling check script.
The script will not continue until the CE agrees that thelight sequence was correct.
The next check just before the VTOC,Verify_Drives_Are_Ok checks to see if we have any bad(NR) or missing drives before we start to VTOC.
These are the steps for DA Loop, cabling and drive checks in the Config and Install New Symmetrix
script.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
31/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 31
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 31
Drive Check Step (Failed)
After fixing the cable problem, the script re-IMLs to initialize the loops again (all DAs FF again). This
time, the loops are all good and the script moves on to verify that all the configured disk drives are
ready and inserted into the correct DAE disk slots.
In this example, we removed Disks C0,C1,C2,C3 to simulate multiple disk failures.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
32/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 32
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 32
Verify Drives are OK Step
To aid the CE in locating any bad or missing drives, the step Verify_Drives_Are_Ok turns on the bad
or missing drives hot replace (amber) LED. If the drive was missing from the DAE drive slot, the hot
replace LED will turn on when a drive is inserted. This is an indication that the CE has found the
correct DAE drive slot. Once the drives have been replaced, the script will IML again and re-run thestep.
If after the IML the drives are all good, the script now moves on and VTOC all drives. If any drives
are still not ready on the loop, the script shows the user what drives are bad and it loops through the
drive check until the issue has been resolved.
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
33/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Module 4 - Installation Troubleshooting - 33
2009 EMC Corporation. All rights reserved. Module 4: Installation Troubleshooting - 33
Drive Enclosure LCC: Rear View DMX-4 shown
PRIan
dEXP
ports
LCC-A
/LCC-B
4Gb/s
Loop
ID
Enc
los
ure
ID
RJ-1
1
Link Control Card A, referred to as LCC-A, connects to the odd Director of a V-Max Engine, while
LCC-B connects to the even Director of that same V-Max Engine.
The Drive Enclosure has a RJ-11 type connector for a cable that is connected to the SPS modules for
monitoring purposes. That way, the LCC can communicate with the SPS through the cable thatconnects them.
Commands and read status are send from the SPS modules that supplies power for the Drive
Enclosure , using the RJ-11 RX/TX differential data lines, to the director.
Per 4 Drive Enclosures there is 1 SPS tray. That means a total of 8 LCC boards and 2 SPS units. To
monitor the SPS modules, only 2 LCCs are needed.
Six grey-colored RJ-11 cables are therefore missing and those LCCs do not monitor any SPS, yet
depend on the two LCC ports with the grey cables, one for SPS/Zone A and one for SPS/Zone B,
to do all the communication. Both the LCC primary (PRI) ports that connect directly to the back-end
(DA) ports, and the expansion (EXP) ports, that daisy chain Drive Enclosures, are High Speed SerialData Connectors (HSSDC).
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
34/37
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
35/37
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
36/37
-
8/6/2019 Module 4 Install Trouble Shooting Rev_1.1
37/37
Copyright 2009 EMC Corporation. Do not Copy - All Rights Reserved.
2009 EMC Corporation. All rights reserved. Module 4: Installati on Troubleshooting - 37
Closing Slide