Server room survival kit?

Question

Trips to the server room can mean extended periods away from the comforts of home, or at least your desk. Especially if it is an off-site hosting facility.

What should you take with you, apart from a warm sweater for places with good air-conditioning?

Solution 1

Things that I always carry on my person, so would be present:

• cell phone
• iPod
• pen/notepad
• thumb drive
• multitool

Things that I keep in my laptop bag so I don't have to think about it:

• "carb bars" (I don't know what these are, but they last forever. My wife made me start carrying them after I had to sleep in a data center during a blizard.)
• quarters for snack/pop machines
• a baggy of splenda (nothing worse than being stuck with people who only drink their coffee black)
• notepad
• Post-it notes
• recovery disks (live CDs)
• USB/serial/RS-232 cables and adapters (the 5-in-1 cable kit specifically, though I;ve tweaked it to have things like T1 loopbacks)
• penlight
• electrical tape
• CO scissors (the kind that central office guys always carry around that you can cut and strip wires with)
• screwdriver (the kind with 6 ends)
• a small hand mirror for looking behind/around things

Things I keep in my toolbag - not guaranteed to have with me always, but I usually know if I'll need it:

• a second 5-in-1 kit and some more cables
• crimpers with RJ45 and RJ11 ends
• labels (like, mailing labels - very sticky and handy for rapid labeling until a professional job can be done with a label maker
• screwdrivers/plyers/end cutters/small socket set - basic tools
• a huge screwdriver that can either be used as a crow bar or to reach the mounting screws on devices that stick out of the rack (like mid-mounting a 40-inch server in a 2-post telco rack)
• velcro wraps and wax string (never used wax string? Try it, it's awesome)
• a collection of writing utensils including sharpies and wax pencils to write on racks
• a collection of screw driver heads - flat, phillips, hex, torx(sp?), and some other specialty ones
• spare heavy-duty power extension cords and a three-plug expander
• a decent digital mutilmeter
• duct tape

I think I have more, but that's the basics. Everything on that list addresses a specific need I've had in my career. The laptop bag is heavy but well worth the bulk in saved trouble. The tool bag I'm rather proud of, it's not big (it's one of those "big mouth" bags that opens like a doctor's black bag), maybe 18 inches long and 12 wide. I spent a great deal of time customizing the contents to maximize the value for the volume. For instance, I threw away the bulky plastic container the socket kit came in; I built a much smaller organizer for it. Same with the screw heads - I built a cloth with elastic on it that the heads slide into. It's also modular - all the screw drivers are in a large pencil case, so I can find them easily and, if I know I will only need them, I can just grab them out of my car and carry them into the DC instead of the whole tool bag.

Solution 2

• A workspace with enough room to work comfortably on a broken 19" server, with screen, keyboard, mouse. Separate from the racks.
• An old PC. Optimally with controllers and slots to fit every piece of hardware you may have to analyze. Mine speaks SCSI wide & narrow, IDE, SATA, PCI, USB, Firewire 400. Keep a small stash of old computers, if you can. They will come and ask if you can rescue the data from this 5.25" disk one day.
• A notebook on the side. The rescue PC has no internet connection, to make sure it cant be infected.
• A big enough disk to put data on that you may have to rescue.
• Room for spare parts and cables. Room for a museum of old stuff you might need for old systems.
• Cart. In case you have heavy servers, a lift of some sort.
• A selection of tools you know you will need. You can keep this small, if you have a complete set elsewhere.
• Telephone with outside access, if your mobile doesnt allow that.
• Pen and Paper.
• Spare parts for your most important servers. If you have several identical machines, keep one spare. It is the organ donor. It may be used for testing new setups, but be prepared to rip it apart.
• A few switches, network converters, cables of all kinds.

Generally make the server room your fortress of solitude, where you can retreat when the brown stuff hits the rotating thing. Nothing like coming out smiling after one hour of hacking, and the broken server is back up, with all data.

Solution 3

I'd add

A magnifying glass** so you can read the ridiculously teeny-tiny writing you get on some equipment, and a mini-maglite so you can use it.

** yes, I am seriously old

Solution 4

Aside from tools I would highly recomend a small first aid kit, and some nonmessy snack foods that keep in storage well. Being able to put a bandage on a paper cut or other small nick on the spot is nice insted on having to hunt down someone from security just for a small bandage. The snacks are good for when it has been two hours too long and you are still more or less stuck in the computer room.

Solution 5

First, let's assume you have the essentials, including a cell phone and something that tells time (the cell phone will do).

Let's remember, if you have to go to the server room, it means something doesn't work. Otherwise, you could fix it from your desk. So, you should at least have:

• A USB stick with your favorite utilities
• A blank USB stick to move files on
• A flashlight
• A dustmask (if you have allergies)
• A screwdriver set
• A tray to put screws in after you remove them
• Pen and paper to take notes with
• An OS image CD, if it doesn't fit on the stick.

After that, it depends on how thick you want your bag to be and your budget. You could do well with:

• A roll-up mini USB-keyboard, if the server room doesn't have one
• A laptop to test things "from the outside" and access the internet if your phone doesn't do that already
• A few extra cables, just in case

View more solutions

Comments

[Oskar Duveborn]

Things is, a decent server room temperature is around 23 degree Celsius - which should be comfortable enough... but many configurations have inadequate cooling which means they're cooling the room to freezing just to get a decent temperature inside the racks instead :/

[jj33]

In my experience, anything left in a data center that won't actively trip an alarm when removed, will be gone within a year. Some stuff will be accidentally borrowed and some maliciously stolen, but gone none the less.

[Bill Weiss]

+1 :( However, in this case, our cage is locked tight, and I don't think anyone's going to do some ninja stuff to steal my tools.

[Jeshizaemon]

I know I get cold at 23C (('_')) brr +1 to jackets.

[Oskar Duveborn]

Though a good cooling solution seams to mean it's freezing on one side of a rack and way too hot on the other side - an even worse situation ^^

[JamesBarnett]

@jj33 What kinda datacenter's are you colo'ed in ;)? If you have a locking cabinet/cage I don't think anything is going anywhere.

[jj33]

@JamesBarnett - It wasn't a colo, it was our corporate data center for a midwestern CLEC/ISP. That meant nothing was locked up. I was really thinking about shared tools though. When we started there was a toolbox with basic tools - screwdrivers, strippers, label maker, etc. All got "lost" slowly but surely.

[Chris Jester-Young]

To that, I'd add OS repair CDs, screwdriver kits in case of hardware malfunction, etc. :-)

[Richard Gadsden]

Don't take drinks into the server room; sooner or later you'll spill all over a server and regret it.

[Mark S. Rasmussen]

A dustmask... In a server room?

[Adam Gordon Bell]

don't use drink containers with low structural integrity like cups or open tumblers. something more secure like well-sealed water bottles or backpacks with straws.

[lc.]

Well let's say hopefully not, but I've been in a situation or two where I wish I had one. It's useful if you a) need to go feeding cables, or b) are unfortunate enough to work for someone who doesn't keep their server room clean. The latter, of course, may very well be the reason things broke in the first place.

[Adam Gordon Bell]

He must cough or sneeze a lot.

[jj33]

If you're in a big shop, they are likely to have policies related to this kind of thing. For instance, I know that a lot of COs have things like maximum number of people allowed in room at once, maximum length of time you can be in the room without coming out (which I think can be an OSHA working-conditions issue with the cold and noise), minimum check-in times (like, go tell the site manager you're still alive, etc)

[lc.]

Yeah, feeding cable through the walls always got my allergies going.

[Adam Gordon Bell]

If you have ever witnessed entire structures burnt down because of a single cigarette butt, you will know smoking is highly prohibited in data centres.

[Chris Jester-Young]

lol The data centres I've been to have (somewhat) easy access to the outside. Of course, inside is no-smoking! :-P

[BIBD]

jj33 - True. Though if you are a small shop, you probably haven't thought about this stuff.

[Oskar Duveborn]

+1 for velcro wraps or whatever cabling-straps you use

[Scott]

Dustmask can also come in handy when you're doing a new server room in a building where construction isn't finished yet.

[Deb]

Kudos on the ear protection. When we put in our first blade-rack the noise in the DC went up. Plus it had an /evil/ harmonic. If I'm going to be in there for more than 5 minutes, I wear my shooting muffs.

[ceejayoz]

Some have server rooms that aren't conveniently located near additional secured storage space. I'd rather have equipment safely stored in a cabinet in the server room itself than trekking across a building because I forgot something I'd never use outside the room...

[Bill Weiss]

Ok... why? I'm intrigued.

[Bill Weiss]

Unfortunately, my servers are in a colo facility an hour from the office. Some things I just don't want to haul back and forth. Plus, I leave for there from different places: the office, my house, bars (when I'm really unlucky), etc. Otherwise, I'd be with you.

[Bill Weiss]

Absolutely agreed.

[Helvick]

Amen to point #1. As someone who has to regularly work for a couple of days at a stretch in customer server rooms the one thing that I find that is most frequently overlooked is a decent work surface. Doesn't have to be a full desk but as you say, big enough to open up a broken 19" server and take a screen, keyboard and mouse.

[Posipiet]

lets add some velcro ties, too

[Posipiet]

You may want to check out this system: It combines into a real proper screwdriver, locked tight. Available large and small. wiha.com/england/Onlineshop/Reversible-blade-systems/…

[quack quixote]

better than just keeping your Dell DVDs, keep a latest-drivers archive on a handy network share. don't delete old driver versions, since you never know what new drivers might break (or not work with the ancient OS you're installing). having a DVD is handy, but not as handy as having any drivers you need ready to drop onto a thumbdrive, optical disc, or whatever.

[quack quixote]

for sealing yourself in, of course. in case of zombie attack or world-ending virus. someone's got to keep the servers running, right?

[Bill Weiss]

I've read that zip ties aren't a good idea for Ethernet cables. A reference, though not a great one: lanshack.com/cat5e-tutorial.aspx . I don't know how accurate that is, but I've heard similar. However, velcro ties, I whole-heartedly agree.

[Massimo]

It will get useful when some (l)user comes screaming in because he wants his data immediately.

[Massimo]

Carpets can also be quite useful, you know.

[MrGigu]

Floppy Disk! The number of times I've been saved by that long-obsolete technology...

[James]

Urban myth. As long as you don't tighten the ties so tight that they cut into the cable you'll be fine...

[Bill Weiss]

Fair enough. (15 character max)

[ceejayoz]

Agreed with @James. There's no physical reason an inert plastic tie would be inherently bad.

[Bill Weiss]

I think the concern is that it's easy to over-tighten those plastic ties. I didn't think that the plastic would set up an EM field or something :)

[Oskar Duveborn]

I always use velcro ties for long-term binding as well, so much easier when the time comes to un-bind a run for whatever reason ^^

[quack quixote]

but carpets leak. sure, they're opaque, but you need that first layer to be waterproof or you'll end up with physical evidence all over the place.

[brijesh shukla]

Have you tried the "releasable" cable ties? They have a little tab you can pinch and it releases the tie. They work really well, and for short term bundling, try velcro ties.

[Massimo]

Not to speak about installing pre-Windows 2008 systems when a controller driver is needed...

[Fernando]

@Oskar - one reason to use the zip ties is so that others don't unbind the run just because they came up with a reason. It creates a barrier to change - both for good and for ill. All in all, 6/half dozen, IMHO.

[pboin]

For when the ceiling leaks (water), or someone decides they have to drill holes in the walls (dust tent), or when you have to rig up some emergency cooling.

[Massimo]

+1. Nothing worse than having critical info for fixing a server... on the same failed server.

[Bill Weiss]

But, I don't want to make cables :( You're right though.

[Bill Weiss]

That's a neat looking tool.

[Bill Weiss]

The colo offering tables and chairs would be so nice... doubt it's going to happen, unless I want to cart the machines out to the break room.

[quack quixote]

@pboin: that's actually a really good reason. gonna add it into your answer so it's more obvious and detached from our collective sarcasm.

[Massimo]

But you will need to.

[Chris B]

OMG I want one so badly :D But appear that it's not in sale anymore :( Can't find it...

[Fernando]

In Stock: amazon.com/Coleman-5-4V-Flashcell-Cordless-Screwdriver/dp/…

[Bill Weiss]

Oh, I get it. Great idea!

[C. Ross]

Forget papercut! Some of the worst cuts I've ever had have been from moving and cataloging old hardware, those steel cases can cut like a knife under the right(wrong?) circumstances. I used a paper towel and duck tape at the time, but a first aid kit would have been much appreciated.

[Bill Weiss]

I'm completely with you on that one.

[Philip]

A network boot image (or WinPE CD) for booting a borked computer and retrieving data without having to change any physical hardware.

[Michael Stum]

If you have space in your rack, I wouldn't be surprised if some company offers 19" lockable drawers...

[Joe]

Most datacenters (think: private cages) will not allow you to run your own cable to their meet-me punch-down block; Instead, I recommend having a few spares of appropriate length and color pre-made.

[Joe H.]

For when it's in awkward places, I've been known to use a digital camera. (and then zoom in on the display, if necessary) It also keeps me from needing to copy down serial numbers while wedging my head into racks.

[Joe H.]

Cell phone cameras work okay for cabling ... but most can't handle the macro issues of shoving it between servers to snap a picture of a serial number.

[Oskar Duveborn]

+1 ESD precaution and that the effects can be very subtle

[Oskar Duveborn]

+1 for the trolley alone (and a bonus point for the normal printer ;)

[Doug]

Ah, a kindred soul that uses Wax String.. +1

[MrGigu]

What the hell is Knob Creek? Where I live that sounds like some sort of homo-erotic horror movie (I'm thinking... a pornographic version of Wolf Creek)

[Developer]

+1 for the camera tips mate !

[Developer]

how about PS/2 mouse & Keyboard ?

[Developer]

LOL, some comedy movies for stress buster.

[JamesBarnett]

Jacket is an often overlooked idea.

[JamesBarnett]

Why if it's not a Colo we have those things in our Colos. Crash carts are all the more important in a Giant Server Room.

[JamesBarnett]

Serial-to-USB and cisco null modem cable good call.

[JamesBarnett]

What colo's have you been in that have a phone in the suite?

[JamesBarnett]

Food != Raised Floor If you are in the Datacenter Suite with the equipment say no to food.

[Joe H.]

@James : because the couple of colos I've been to didn't have places to store & secure them. I guess some might be different, looking at the various answers here. And I'd think a lift would be something that the colo should buy, not each individual renting space.