ESXI Crash need help to understand log and support about nexentastor on virtual machine

Question

If i understand right, the following core dump means that the cpu4 has crashed the Host

if i read the next line it seem that at the time the CPU 4 was assigned to the NexentaStore Vm ... SO if im right i can say that NexentaStor Vm crash my esxi

Am i right ?

Does that core dump can provide me some more informations ?

2012-11-14T03:48:01.046Z cpu4:6089)0x41221f25ba08:[0x41803007abff]PanicvPanicInt@vmkernel#nover+0x56 stack: 0x3000000008, 0x41221f25ba
2012-11-14T03:48:01.046Z cpu4:6089)0x41221f25bae8:[0x41803007b4a7]Panic@vmkernel#nover+0xae stack: 0x2e067c00000010, 0x0, 0x1f25bb38,
2012-11-14T03:48:01.047Z cpu4:6089)0x41221f25bc18:[0x4180300a7823]TLBDoInvalidate@vmkernel#nover+0x45a stack: 0xca, 0x0, 0x0, 0x0, 0x0
2012-11-14T03:48:01.047Z cpu4:6089)0x41221f25bc68:[0x418030489e17]UserMem_CartelFlush@<None>#<None>+0xce stack: 0xcaa0b, 0x0, 0x0, 0x4
2012-11-14T03:48:01.047Z cpu4:6089)0x41221f25bd78:[0x41803048ab91]UserMemUnmapStateCleanup@<None>#<None>+0x58 stack: 0x0, 0x41221f25bd
2012-11-14T03:48:01.047Z cpu4:6089)0x41221f25be58:[0x41803048b97d]UserMemUnmap@<None>#<None>+0x104 stack: 0x41221f267000, 0x41221f25bf
2012-11-14T03:48:01.048Z cpu4:6089)0x41221f25be98:[0x41803048bf20]UserMem_Unmap@<None>#<None>+0xe3 stack: 0x426, 0x0, 0x41221f25bef8,
2012-11-14T03:48:01.048Z cpu4:6089)0x41221f25beb8:[0x4180304a5985]UW64VMKSyscallUnpackReleasePhysMemMap@<None>#<None>+0x18 stack: 0x10
2012-11-14T03:48:01.048Z cpu4:6089)0x41221f25bef8:[0x418030476791]User_LinuxSyscallHandler@<None>#<None>+0x17c stack: 0x41803004cc70,
2012-11-14T03:48:01.048Z cpu4:6089)0x41221f25bf18:[0x4180300a82be]User_LinuxSyscallHandler@vmkernel#nover+0x19 stack: 0x3ffe63bed80, 0
2012-11-14T03:48:01.049Z cpu4:6089)0x41221f25bf28:[0x418030110064]gate_entry@vmkernel#nover+0x63 stack: 0x10b, 0x0, 0x0, 0x426, 0xcf76
2012-11-14T03:48:01.049Z cpu4:6089)VMware ESXi 5.1.0 [Releasebuild-799733 x86_64]
PCPU 1 locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU(s): 1).
2012-11-14T03:48:01.050Z cpu4:6089)cr0=0x80010031 cr2=0xcaa0b750 cr3=0x197d7b000 cr4=0x42768
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:0 world:6111 name:"vmm0:Windows_2012_-_SQL" (V)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:1 world:6032 name:"vmm0:Windows_2012_-_AD" (V)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:2 world:6098 name:"vmm0:Windows_2012_-_App" (V)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:3 world:4099 name:"idle3" (IS)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:4 world:6089 name:"vmx-vcpu-0:NexentaStor" (U)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:5 world:6134 name:"vmm0:Ubuntu_-_NGINX" (V)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:6 world:4102 name:"idle6" (IS)
2012-11-14T03:48:01.050Z cpu4:6089)pcpu:7 world:4103 name:"idle7" (IS)
2012-11-14T03:48:01.050Z cpu4:6089)@BlueScreen: PCPU 1 locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU(s): 1).

EDIT

it has been working good since 2.5 days but crash with an error

2012-11-16T16:15:35.233Z cpu6:4102)World: 8381: PRDA 0x418041800000 ss 0x0 ds 0x4018 es 0x4018 fs 0x4018 gs 0x4018
2012-11-16T16:15:35.233Z cpu6:4102)World: 8383: TR 0x4020 GDT 0x4122001a1000 (0x402f) IDT 0x41800b112000 (0xfff)
2012-11-16T16:15:35.233Z cpu6:4102)World: 8384: CR0 0x80010031 CR3 0x125f24000 CR4 0x42768
2012-11-16T16:15:35.238Z cpu6:4102)Backtrace for current CPU #6, worldID=4102, ebp=0x41220019bc10
2012-11-16T16:15:35.239Z cpu6:4102)0x41220019bc10:[0x41800b052105]IRQ_DoInterrupt@vmkernel#nover+0x5c stack: 0x0, 0x418041800180, 0x0,
2012-11-16T16:15:35.239Z cpu6:4102)0x41220019bc50:[0x41800b04bd92]IDT_IntrHandler@vmkernel#nover+0x139 stack: 0x41220019bd68, 0x41800b
2012-11-16T16:15:35.239Z cpu6:4102)0x41220019bc60:[0x41800b110064]gate_entry@vmkernel#nover+0x63 stack: 0x4018, 0x4018, 0x0, 0x0, 0x0
2012-11-16T16:15:35.240Z cpu6:4102)0x41220019bd68:[0x41800b2dbd6f]Power_HaltPCPU@vmkernel#nover+0x276 stack: 0x41220019be68, 0x4122001
2012-11-16T16:15:35.240Z cpu6:4102)0x41220019be68:[0x41800b1bd114]CpuSchedIdleLoopInt@vmkernel#nover+0x873 stack: 0x41220019be98, 0x41
2012-11-16T16:15:35.240Z cpu6:4102)0x41220019be78:[0x41800b1c66ae]CpuSched_IdleLoop@vmkernel#nover+0x15 stack: 0x6, 0x6, 0x41220019bfe
2012-11-16T16:15:35.241Z cpu6:4102)0x41220019be98:[0x41800b04f6ce]Init_SlaveIdle@vmkernel#nover+0x49 stack: 0x0, 0x0, 0x0, 0x0, 0x0
2012-11-16T16:15:35.241Z cpu6:4102)0x41220019bfe8:[0x41800b2e1f86]SMPSlaveIdle@vmkernel#nover+0x31d stack: 0x0, 0x0, 0x0, 0x0, 0x0
2012-11-16T16:15:35.241Z cpu6:4102)VMware ESXi 5.1.0 [Releasebuild-799733 x86_64]
#PF Exception 14 in world 4102:idle6 IP 0x41800b052105 addr 0x417fd1837b01
2012-11-16T16:15:35.242Z cpu6:4102)cr0=0x8001003d cr2=0x417fd1837b01 cr3=0xcdff6000 cr4=0x216c
2012-11-16T16:15:35.242Z cpu6:4102)frame=0x41220019bae0 ip=0x41800b052105 err=0 rflags=0x10006
2012-11-16T16:15:35.242Z cpu6:4102)rax=0x66f1400 rbx=0x41220019bc50 rcx=0x41800b2dbd6f
2012-11-16T16:15:35.242Z cpu6:4102)rdx=0x417fcb146700 rbp=0x41220019bc10 rsi=0x41220019bc70
2012-11-16T16:15:35.242Z cpu6:4102)rdi=0x19bc50 r8=0x4100018d29b0 r9=0x4ca88b
2012-11-16T16:15:35.242Z cpu6:4102)r10=0xdf r11=0x1 r12=0x4122001a7000
2012-11-16T16:15:35.242Z cpu6:4102)r13=0x19bc50 r14=0x41220019bc70 r15=0x1
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:0 world:6211 name:"vmm1:Windows_2012_-_SQL" (V)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:1 world:4109 name:"directMapUnmap" (S)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:2 world:6255 name:"vmm0:NexentaStor" (V)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:3 world:6194 name:"vmm0:Windows_2012_-_App" (V)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:4 world:6207 name:"vmm0:Windows_2012_-_SQL" (V)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:5 world:5855 name:"vmm0:Windows_2012_-_AD" (V)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:6 world:4102 name:"idle6" (IS)
2012-11-16T16:15:35.242Z cpu6:4102)pcpu:7 world:4103 name:"idle7" (IS)
2012-11-16T16:15:35.242Z cpu6:4102)@BlueScreen: #PF Exception 14 in world 4102:idle6 IP 0x41800b052105 addr 0x417fd1837b01
2012-11-16T16:15:35.242Z cpu6:4102)Code start: 0x41800b000000 VMK uptime: 2:11:18:25.729
2012-11-16T16:15:35.242Z cpu6:4102)0x41220019bc10:[0x41800b052105]IRQ_DoInterrupt@vmkernel#nover+0x5c stack: 0x0
2012-11-16T16:15:35.243Z cpu6:4102)0x41220019bc50:[0x41800b04bd92]IDT_IntrHandler@vmkernel#nover+0x139 stack: 0x41220019bd68
2012-11-16T16:15:35.243Z cpu6:4102)0x41220019bc60:[0x41800b110064]gate_entry@vmkernel#nover+0x63 stack: 0x4018
2012-11-16T16:15:35.244Z cpu6:4102)0x41220019bd68:[0x41800b2dbd6f]Power_HaltPCPU@vmkernel#nover+0x276 stack: 0x41220019be68
2012-11-16T16:15:35.244Z cpu6:4102)0x41220019be68:[0x41800b1bd114]CpuSchedIdleLoopInt@vmkernel#nover+0x873 stack: 0x41220019be98
2012-11-16T16:15:35.244Z cpu6:4102)0x41220019be78:[0x41800b1c66ae]CpuSched_IdleLoop@vmkernel#nover+0x15 stack: 0x6
2012-11-16T16:15:35.245Z cpu6:4102)0x41220019be98:[0x41800b04f6ce]Init_SlaveIdle@vmkernel#nover+0x49 stack: 0x0
2012-11-16T16:15:35.245Z cpu6:4102)0x41220019bfe8:[0x41800b2e1f86]SMPSlaveIdle@vmkernel#nover+0x31d stack: 0x0
2012-11-16T16:15:35.247Z cpu6:4102)base fs=0x0 gs=0x418041800000 Kgs=0x0
2012-11-16T16:15:35.247Z cpu6:4102)vmkernel             0x0 .data 0x0 .bss 0x0

Is that related to previous one or just another host instability

Solution

VMware article on debugging "Failed to ack TLB" purple screens: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1020214

It sounds like hardware, you may need a VMware tech to investigate.

Comments

[Bgnt44]

i have ESXI 5.1 so it seem that Windows 2012 is supported

[Bgnt44]

btw i use it with \Bios mode not EFI, don't know if that could be an issue

[Rqomey]

yea I heard it works, but I meant I don't think it is "officially supported" I'll cheek though. I'm fairly sure it is meant to use EFI boot however (this is a check bo in the settings)

[Rqomey]

win2k12 is supported on 5.1 I was wrong there. Just had a hunch but am probably wrong, do you have any power management options turned on in the bios at all?

[Bgnt44]

most issue fixed after disabling power management in bios ... but i haven't be able to have a stable esxi, some crash with datastore, hdd fails randomly ... i think that esxi really require entreprise hardware..

[Rqomey]

Im not sure if enterprise hardware is required (although it is a good idea if you can get it). You HDD fail and probably your datastore have nothing to do with esx, it sounds like you have storage problems. Check /var/log/vmkernel.log for storage errors. @Tims answer is probably correct given your last comment

[Tim]

Most "consumer" grade disks are practically garbage. You will want SSD or Enterprise storage. ESXi does not handle "blips" with disks well like a desktop OS. Most desktops ignore the blip and move on. ESXi likes to halt and ham it up.