procedure to follow during vm hung
DESCRIPTION
VM troubleshooting step during hungTRANSCRIPT
![Page 1: Procedure to Follow During VM Hung](https://reader035.vdocument.in/reader035/viewer/2022082200/55cf8ee3550346703b96b0cb/html5/thumbnails/1.jpg)
Procedure: VM Hung-Check out the output of 'ps -s | grep < VM's name >' multiple times.
-It will display a list of worlds. Check whether for a single world, the state field (3 and 4th field from
left) shows "WAIT" and "SCSI" respectively consistently in all the outputs of the above command
1. vmdumper -l to get the world id for the VM
2. vmdumper < wid-for-a-thread > backtrace < -- log backtrace in vmkernel.log
3. vmdumper < wid-for-a-vm > samples_on
4. vmdumper < wid-for-a-vm > unsync < -- check if vmmcores-unsync-< N >.gz is generated
5. vmdumper < wid-for-a-vm > sync < -- check if vmmcores-< N >.gz is generated
6. vmdumper < wid-for-a-vm > vmx < -- check if vmx-zdump.* is generated
7. vmdumper < wid-for-a-vm > vmx_force < -- check if vmx-zdump.* is generated
8. vmdumper < wid-for-a-vm > suspend_vm < -- check if .vmss is generated
9. vmdumper < wid-for-a-vm > samples_off
10. If step 8 fails, will try to collect the suspended state from the vSphere Client or using the vm-
support command
(see Suspending a virtual machine on ESX/ESXi to collect diagnostic information (2005831) ESXi 5.x
section )
11. Capture host logs with performance snapshots in batch mode using the cmd:
esxtop -a -b -d 5 -n 120 > esxtop.csv
and
vm-support -p
12. Capture Guest OS logs
13. Capture VMRC/Remotemks logs (C:\Users\< USER >\AppData\Local\VMware\vpx)
14. Migrate all the virtual machine from affected host, except virtual machine which is hung.
15.
a. Enable the VMFS file core dumping:
- The VMFS file dump has to be turned on and the core dump has to be redirected to a file. KB article
that explains the process:
http://kb.vmware.com/kb/2077516
b. Then run this command to get the live dump:
localcli --plugin-dir /usr/lib/vmware/esxcli/int/ debug livedump perform
Monsanto Company Confidential Page 1 of 2 Printed: 4/21/2023
![Page 2: Procedure to Follow During VM Hung](https://reader035.vdocument.in/reader035/viewer/2022082200/55cf8ee3550346703b96b0cb/html5/thumbnails/2.jpg)
16. Crash the host using NMI and collect the dump / ESX host log bundle.
See Using hardware NMI facilities to troubleshoot unresponsive hosts
(http://kb.vmware.com/kb/1014767) Can be done from the HP iLO
Monsanto Company Confidential Page 2 of 2 Printed: 4/21/2023