Hardware requirements for a Virtual Server

Question

We have decided to go with a virtualization solution for a few of our development servers. I have an idea of what the hardware specs would be like if we bought separate physical servers, but I have no idea how to consolidate that information into the specification for a generalized virtual server.

I know intuitively that the specs are not additive - I shouldn't just add up all the RAM requirements from each machine to get the RAM required for the virtual server. I can't really treat them as parallel systems either because no matter how good the virtualization software is, it can't abstract away two servers trying to peg the CPU at the same time.

So my question is - is there a standard method to estimating the hardware requirements for a virtualized system given hardware requirement estimations for the underlying virtual machines? Is there a +C constant for VMWare/MS Virtual Server overhead (and if so, what is C?)?

P.S. I promise to move this over to serverfault once it goes into beta (Promise kept)

Solution 1

Yes add 25% additional resources to manage the VM. So if I need 4 servers that are equal to single core 2 ghz machines with 2 gigs of ram I will need 10 ghz processing power plus 10 gigs of ram. This will allow all systems to redline and still be ok.

In the real world this will never happen though, all your servers will not always be running all the time. You can get a feel for usage by profiling your current servers and determine their exact requirements and then adding an additional 25% in resources.

Check out this software for profiling utilization http://confluence.atlassian.com/display/JIRA/Profiling+Memory+and+CPU+usage+with+YourKit

Solution 2

The requirements are in fact additive. You should add up the memory requirements for each VM, and the disk requirements, and have at least one processor core per VM. Then add on whatever you need for the host system. VMs can share a CPU, to some extent, if you have really low performance requirements, but they cannot share disk space or memory.

Solution 3

Answers above are far too high, second (1 core per VM) is closer. You can either 1) plan ahead and probably over-purchase 2) add just-in-time. Do you have some reason that you must know well ahead (yearly budget? your chosen host platform doesn't cluster hosts, so you can't add later?)

Unless you have an incredible simple usage profile, it will be hard to predict before and you'll over purchase. The answer above (+25%) would be several times more than you need for an modern server virtualization software (VMware, Zen, etc) that manages resources smartly. It's accurate only for desktop products like VPC. I chose to rough it out on a napkin and profile my first environment (set of machines) on the host. I'm happy.

Examples of things that will confound your estimation

• Disk space, Some systems (Lab Manager) use only the difference in space from the base template. 10 deployed machines with 10 GB drives using about 10 GB (template) + 200MB.
• Disk space: You'll then find you don't like the deltas in specific scenarios.
• CPU / Memory: This is dev shop - so you'll have erratic load. Smart hosts don't reserve memory and CPU.
• CPU / Memory: But then you'll want to do perf testing, and want to reserve CPU cycles (not all hosts can do that)
• We all virtualize for different reasons. Many of the guests in our environment don't have much work. We want them there to see how something behaves with a cluster of 3 servers of type X. Or, we have a bundle of weird client desktops waiting around, being used one at time by a tester. They rarely consume many host resources.

So, if you are using something like that doesn't do delta disks, disk space might be somewhat calculable. If lab manager (delta disk), disk space is really hard to predict.

Memory and processor usage: You'll have to profile or over-purchase heavily. I have many more guest CPUs than host CPUS, and don't have perf problems - but that's because of the choppy usage in our QA environments.

Comments

[cdonner]

You could have a database server and a build server share the same set of CPUs. In my experience, you'll run out of disk space (particulary on SCSI systems that tend to be smaller) or memory first.

[mark]

We're using linux-vserver.org in our development environment for that exact reason. Bottleneck is disc I/O. Memory is cheap nowadays, stick into it as much as you can.

[ShallowHeart]

This is very helpful, but what if this is a new build and none of the servers already exist?

[Al Katawazi]

Ah, well if you don't want to take any risks find out performance by slapping your VM's on amazons elastic computing cloud aws.amazon.com/ec2. They will bill you, when you have an idea of what resources are being used, buy a server to meet those needs.