Best practices for power cabling racks in DC

Question

With network cables it is easy. I use velcro-ties, coloured cables, 0U vertical cable organizers from APC on side of the rack -- easy and neat.

However with power cables it was always giving me headaches.

With some of the racks I am using Vertical PDUs from APC (0U) so it's kind of neat as am getting 0.3m/0.5m cables and connecting them to servers which are close to the PDU sockets. It's kind of neat, but it could be better.

With horizontal PDUs it is really bad -- no matter how hard I try there is always few cables that are too long and are hanging loose below the APC.

Eg. http://www.flickr.com/photos/92011471@N00/25695150/ -- this will cause the cables on the bottom to be different length and its going to get quite messy...

What are best practices for power cabling? How do you guys do this so your cables once put in once are neat and you don't have to worry about them for years? Do you also cut them to match right length? Do you use some special organizers? Or maybe you know some tricks like hiding them between the side of the server and rack?

I went through many many posts on SF however most of the are oriented around network cabling and very few are touching slightly on the power problem.

Of course I am talking about having network cables on one side and power cables on the other side.

Any suggestions, recommendations, suggestions much appreciated.

Solution 1

Horizontal PDUs are a mess -- In my experience there's nothing that can be done to make these neat: You can bundle your cables neatly down the side of your rack, but when you get to the PDU they fan out into a rat's nest.

For vertical ("Zero-U") PDUs you can acquire custom-length power cables (they're available from various suppliers, usually in the same assortment of lengths you can get ethernet cables in - 1' 3' 5' 7' 14' etc.). The cables can then be bundled and tucked into the side of the rack the same way you would with ethernet cables - I use velcro ties to hold mine to the side of the rack so they don't trap server exhaust...

Some other tips:

• Bundle your cables either by branch (if you have A/B power) or by machine.
  (If you have A/B power you can also use different color power cords to distinguish them - e.g. Black/White)
• Label your cables (Either on the plug or on the PDU) so you know what you're unplugging
• Remember to leave designated power points for a crash cart (keyboard/monitor) if necessary.
  (If you do not have an in-rack keyboard/mouse it IS necessary - a KVM or IP-KVM is not always an adequate substitute)
• Don't run your AC power cables on the same side of the rack as your ethernet cables
  (You already seem to be aware of this)

Solution 2

Buy short power cables. In the US, I buy 1-foot and 2-foot cables and run to the vertical Zero-U PDUs located on the left and right sides of the rear of the cabinet, since the power supplies are usually situation on the left or right side of the server chassis (never center).

I don't bother with server cable management brackets since they restrict airflow and have become clumsier over the past few years. I buy a roll of raw velcro to cut into strips as necessary to secure the power cables. If I need to pull a system out, I remove the power cable and slide the system out, leaving the power cable hanging in place.

Solution 3

I think you are spot on for the zero-U PDUs. If you have 3-phase power available, I would install one of these on each side of the rack: http://www.apc.com/products/resource/include/techspec_index.cfm?base_sku=AP7892

(208V output is also available but not with 42 connections).

Then connect each server to the appropriate numbered outlet with short power cables. In fact, you can shorten the .5m cables even further by folding them over and using zip ties. Then connect the left power supply to the left PDU, and right to right. For single power supply servers, just alternate between left and right.

With this method, no power cables are ran down the sides of the rack and you don't interfere with your Ethernet cabling.

I would avoid any horizontal PDUs in the rack because they just get in the way of the rest of your equipment.

Solution 4

I used to be a Linux Admin in one of the bigger hospitals here in Amsterdam, they relied on massive ammounts of 1u/4p xen boxes which in normal situations is a nightmare when you dealing with proper cable managment. For the first time ever, I fell in love with HP swing-arm based rackmounts, The arms provide the managment while keeping your flexibility (and unblocked cooling!) in the back.

There is one thing though, they had (hospital/goverment = lots of ISO reqs.) special power cables. I had never seen them before, to bad I forgot the material they where coated it. The good news, they flex as good as CAT6, they have about the same size as CAT6 & they should be able to withstand 90C of heat. The bad news: Appearntly they come by 20,- EUR @ 1,5m. Doable when doing half a rack, but kinda costly when you have a full rack of redundant 1u's ;)

Looking at that picture of yours I can do nothing but salute you, nice job.

Solution 5

Use short power cables and Zero U PDUs.

View more solutions

Comments

[voretaq7]

For whatever it's worth, if your cabling job is as neet as that photo implies you're in the 99th percentile of nice cabling jobs. Even my rack isn't THAT neat (because of different classes of server with power supplies in different spots). You're doing great!

[bart613]

Thanks, it's close to this one ;-) I will take a photo over upcoming days and will post it here ;-)

[voretaq7]

I discovered not long ago that I need to re-cable MY main production racks -- They look great, except the ethernet trunks obstruct the removal of the top-of-rack switches. Getting a failed switch out & the replacement in without knocking the whole rack offline was an interesting experience...

[bart613]

Do you suggest basicaly running cables horizontaly towards PSU on the other side of the rack or placing both PDUs on one side (I have done this before)?

[Philip]

+1 Label cords. We have a cheap label printer and just put a number on both ends of any cable that goes in; no particular significance to the number, just the same number at both ends.

[bart613]

Yeah I was worried you gonna say this -- i.e. "there's nothing that can be done to make these". Seems like the "rat nest" underneath is going to be the only sane option here...

[dmourati]

I've been looking into the label issue for cat5e cables and found vendors willing to prelabel the cables uniquely. I wonder whether anyone makes power cords with the same feature.

[voretaq7]

One thing that helps with horizontal PDUs is to mount them in the middle of the rack and use fit-to-length cords. If your PDUs have outlets on the side that's facing "into" the rack leave 1U of space between them (This eats up more rack space, but you can run cables between that gap and have things be a bit neater). The real answer is "use Zero-U PDUs" though.

[voretaq7]

@dmourati label printers and cable labels (similar to amazon.com/DYMO-Self-Laminating-Vinyl-Self-laminating/dp/…) are relatively cheap and let you buy bulk cable and label it yourself. Very handy to have.

[voretaq7]

+1 about not wasting time and effort with "cable management" arms -- I've never seen a case where they were really helpful. In my experience they just trap heat and make servicing machines a hassle.

[dmourati]

@ voretaq7 yea, you definitely need a label printer. I've used DYMO in the past but never with proper cable labels, always flags. Still trying to figure out which one I want: panduit, idexpert, brother, dymo, etc.

[Hyppy]

I'd suggest these as well. You print them from an inkjet/laser, and they're far more durable than anything else we tried: panduit.com/Products/ProductOverviews/ProductSearch/…

[voretaq7]

@Hyppy The Panduit labels are great, especially if you have a whole lot of cables to label. You can write on them with a quick-drying pen too in a pinch too, though it smudges sometimes. The thermal-transfer labels are good if you've got a lot of 1 or 2 cable jobs...

[John Gardeniers]

I no longer label cables. I maintain a database of the endpoints of the cables instead. Far easier to maintain and much easier to use, as it completely eliminates the need to search for the other end of a given cable.

[voretaq7]

@John my experience is that those databases eventually get out of sync with reality (one memorable incident resulted in the unplugging of a core switch!) - particularly with power I find it helpful to have labels on both ends of a cable saying what it's for (eg. SW001-PSUA , SW001-PSUB , Database01-PSUA , etc...)

[Kyle Gibbons]

One thing to keep in mind if you go with a 3PH option is that you want to balance the load as best you can across the phases to avoid having power on the neutral leg and it heating up. The one @voretaq7 linked in his comment is nice because it has the Amp meters right on it for you.

[voretaq7]

@bobert5064 Nicer still because you can poll the amp loads via SNMP -- If you know which phases each branch connects to & your UPS reports phase loads via SNMP as well you can do capacity planning from the comfort of your desk.

[Kyle Gibbons]

@voretaq7 that's awesome. Hadn't thought of that!

[Stefan Lasiewski]

And keep in mind that a "2-foot power cable" will vary between 18-24 inches.

[ewwhite]

@StefanLasiewski Really?

[Stefan Lasiewski]

It depends on the vendor, but yes. I've ordered several hundred power cables in the last couple of years, and we're about to do another batch for the new datacenter. I've wasted a ridiculous amount of time dealing with cables which were too short.