HP Blade ILO not responding in chassis ILO

Question

I have just started at a new company and I am inspecting their current server config. The HP 480c blades in a c7000 chassis aren't responding to ILO, although the chassis ILO is working fine. I have a feeling the last sysadmin configured the blades ILO as static IPs and it is not responding correctly. The servers are sitting in a datacenter and I'm hoping to be able to fix this remotely.

Is there a way that I can change the ILO static IPs for the blades remotely? If not and I do have to go onsite, how do I change the IP addresses of the ILO for the blades? (Sorry I'm not very familiar with HP servers)

thanks for you help!

Solution 1

Yep. Console onto the server OS itself and use the hponcfg app to dump and edit the iLo configuration.

Best bet is to dump the current config to a file, edit, then re-up.

Solution 2

The chassis OA has the ability to over-ride existing iLO configurations and talk to the blades no matter what the previous admin did. You should be able to apply iLO configuration values to the iLO via the OA. If the OA can't talk to the iLOs on the blades, you have a problem.

One of my customers has a couple of C3000 chassis. With an older firmware, the iLO on the blades would become non-responsive both to the chassis and to the rest of the network (the iLOs were IP-enabled for IP-KVM purposes). This was repeatable, and all blade iLOs would go unresponsive within about an hour of each other, 30 to 90 days after being started.

The only way to get everything talking together again was to power-cycle the whole chassis -- shutdown all blades, then pull the power inputs. Removing and reseating individual modules didn't work. I will admit we didn't try the downloadable iLO configuration application; a couple of these blades run ESX, which makes the tool rather academic anyways...

In our case, HP denied there was a known problem even though both of our chassis exhibited the issue.

Depending on when this chassis was shipped, you may be in this boat. Look for, or call HP and ask for, a firmware update and apply that. Note that you have to update the firmware for all components -- OA, VC (if you have it), as well as individual iLOs, and blade BIOSs. The whole chassis will be inoperable during this update, and components are updated serially so it can take several hours to run. One co-worker told me that he'd been warned by HP that there is a specific order you have to run the updates in, otherwise you risk bricking components; however an HP service agent denied that. In the end we managed to get HP to deal with it as a warranty issue, we raised enough of a fuss that they had someone come in and do it for us.

Solution 3

In my case i had changed some VLAN and IP settings, using a static IP on one blade. something messed up and i was unable to ilo to the blade. rebooting it didnt help. even thought rebooting the OA tray helpted to clean up some mess in the gui, i still was unable to ilo to the blade (even thought i was in the same vlan and subnet). Using EAIP to reset the IP didn't work since i was using a static ip on the blade ILO which has always higher priority then the EAIP. However here's what helped me:

SSH to the OA and type: RESET SERVER [bay number]

Solution 4

You can also do the re-addressing it via the onboard admin GUI in case you are not particularly familiar with HP blade infrastructure.

Also be aware that at times both blades and virtual connect modules if you have them) do seem to stop communicating or somehow get out of sync with the onboard admin, and you may not be able to use them in that case. This typically does not effect the whole chassis, just one blade/VC module at a time.

When this happens, we've found that we often need to remove and reseat the blade/VC module. Occasionally the the VC a power reset/failover will fix it.

The act of physically removing and reseating a blade/VC module seems to reset something in the ILO/OA/Chassis/VC that is not reset with a simple powercycle.

The fact that this happens is also sucks and it pretty much the only bad thing I have to say about HP blade technology, which I otherwise think is pretty darn good.

Solution 5

I also had an issue were the OA wouldn't communicate with the onboard ILO's.

It was telling us that we should check the default gateway which was just fine.

Resetting the OA using the virtual button solved the issue for us. No reseating of the blades was neccesary

View more solutions

Comments

[Admin]

Its also worth mentioning that these servers are running production websites, so a reboot is not really an option right now. Some of the docs I've been reading seem to state you need to configure ILO on boot.

[Admin]

iLO is not part of the main computer. Reboot the computer does not reboot iLO; similarly rebooting iLO does not interfere with the main computer itself. The easiest way to configure iLO is by entering the iLO RBSU, but this option is only available during the boot process. Chris Thorpe has the answer for a running server below.

[Philip]

+1, Correct. Quick link, go here: h20000.www2.hp.com/bizsupport/TechSupport/… Select your OS, select the "Software - Lights-Out Management" section. Find the "Online Configuration Utility" there.

[Scott Parrott]

I ran into a similar problem with a C7000 Blade Enclosure. From the OA, I went into Enclosure Bay IP Addressing and sure enough, the newly installed blade wasn't setup with a static IP address. Once assigned, problems were solved.