Balancing RAM when using multiple CPUs

Question

We have HP DL360 G7 server with one cpu and 16G 12G ram. We plan to add another cpu. So, we need also ram for second cpu. Is there any negative performance impact if we add different size of ram to second cpu? for example 20G ?

Current ram configuration:

Solution 1

The HP ProLiant DL360 G7 server (and other Nehalem-and-newer CPU systems) have a set of memory DIMM population guidelines. Can you share what's currently populated and what your final RAM amount/goal is?

• This is documented primarily in the Quickspecs for the system, but I'll try to give some specific guidelines.
• HP also has an interactive Memory Configuration Tool to help step you through the process and your options.
• Here's a technical deep-dive on the Nehalem/Westmere CPU architecture that explains the memory side of things.

At present, you have 9 available DIMM slots out of the 18 slots on the server. You can only use half of them because the server only has one CPU installed. Installing the additional CPU opens the other 9 slots up for use.

• Performance is maximized if you balance across each CPU's DIMM banks. E.g. results are best if an equal amount of RAM is assigned to both CPUs.

The other critical rules are:

• Do not mix Unbuffered memory (UDIMMs) with Registered memory (RDIMMs)
• Do not install DIMMs if the corresponding processor is not installed
• To maximize performance, balance the total memory capacity between all installed processors
• Populate DIMMs from heaviest load (quad-rank) to lightest load (single-rank) within a channel

There are also memory channel population tips that affect bus speed. E.g. using 3 to 6 DIMMs per CPU is going to be faster than running with all 18 slots populated. RAM can be seen by both CPUs, but you have to popular on both sides if you have two CPUs.

Reply back with your setup, and we can help optimize...

Solution 2

The system you are refering to does not differentiate between RAM modules for one or the other processor. All RAM is shared between the two processors. In principle there is no impact for adding memory of different size. The only thing you need to keep in mind is that you respect the channels (dual channel= 2 DIMMS, Triple channel is 3 dimms). Also make sure your RAS-CAS latencies and timings are always the same.

EDIT

As stated by David Schwartz:

If you have different amounts of RAM on each CPU, then one direction of the QPI will be more saturated than the other, which will have a negative impact on performance.

Solution 3

I'm afraid that that's not how it works. There is no such thing as per-cpu memory (at least on such servers, supercomputers is a different story). So you don't need to add extra memory to use an extra CPU. Of course you can do it if you find that you're memory constrained as well as cpu constrained.

Comments

[ewwhite]

Looks like 12GB RAM.

[Guntis]

Updated question.

[ewwhite]

There are per-CPU memory banks and definitely performance implications tied to RAM location relative to those banks.

[Dennis Kaarsemaker]

I stand corrected, thanks for your detailed answer!

[Guntis]

yes, my fault about ram amount.

[David Schwartz]

If you have different amounts of RAM on each CPU, then one direction of the QPI will be more saturated than the other, which will have a negative impact on performance.

[Dan Is Fiddling By Firelight]

@DavidSchwartz are server chips somehow different in regards to memory amounts on each channel? I recall regularly seeing imbalanced memory amounts on consumer LGA1366 systems offered in retail several years ago; and at the time was told the only penalty would be for the portion of the total not spread across all 3 channels. ex installing 4-2-2gb would give 6gb at 3 channel performance and 2gb at single channel.

[David Schwartz]

In the Xeon-5500 benchmarks I've seen, three imbalanced channels (though they used 2GB/2GB/4GB-2GB/2GB/4GB) was no better than two balanced channels (4GB/4GB/0-4GB/4GB/0). Three balanced channels was 23% faster in synthetic benchmarks and 6% faster in real-world benchmarks.