How to build a supercomputer by linking common PCs on a network?

Question

Fellows,

I have several computers, some new, some old ( I collect them, since my first one, which has 2Kbytes of RAM, from 1979 ). The collection is getting huge, but the purpose of this question is related to the fact that I always loved the power of supercomputers, or at least, the power of the big machines ).

I ever thought about the idea of joinning machines in order to get a more powerful one. I run a LAN ( local area network ) 1Gbit apeed, where there are 4 machines intel i7 2600k running at 4.8Ghz watercooler, each one with 16Mb RAM, SSD and common hard drives, for a total of 30Tb of space ( total on the LAN ). Having read articles and watched many videos about virtualization, I question for the possibility of installing bare metal ( Type 1 ) hypervisors on each machine, then, creating a virtual machine which spread across the physical machines, so I could install an operational system like windows on top, to run softwares that need much resources, like CPU, RAM, Hard disk, etc.

I imagine that there must be a way that a virtual machine "thinks" that it´s installed on a single machine, but indeed, it´s spread along several nodes ( like a cluster ). For the virtual machine, it sees the system as only one big machine, but indeed, there´s shared CPU, shared RAM and shared hard drives.

Using this way, we could install an OP and run for instance, Adobe After Effects, or Adobe Premiere, which needs an outstanding parallel processing ( or cpu power ) in order to make previews in real time, or to run complex software which could benefit from multiple processors. I know many people would suggest purchasing a big multi-cpu, multi-core xeon machine for parallel processing, but it´s not the case...I like to think that with the current technology, there must be a way to join PCs and get more computational power.

I see people joinning Raspberry pi and making "supercomputers" at youtube, with kind of 1 teraflop, so why can´t we do it with our own machines, which has LAN, ram, disks...isn´t it the same thing, we only need the software and how to do it, no ? Is it possible ? How to do it ?

Thanks

Solution 1

Unfortunately, for now, the "supercomputer" you want to run has not been developed yet in terms of hardware and software.

The existing hypervisors for virtualization such as Hyper-V, VMware ESXi, XenServer allow running Virtual Machines on a single host or on a cluster. Hypervisor takes hardware CPU, RAM and storage and "converts" it to virtual resources to run VMs. For storage, it can be configured in shared volumes being mirrored between hosts using network connectivity to data transmission (alike iSCSI SAN, VMware VSAN, StarWind VSAN etc.), but still each VM utilize only local compute CPU and RAM.

From my experience, there is no software or hardware that could combine CPU, RAM, and storage since there is no bus type that could serve the communication between "supercomputer's" hosts.

Solution 2

This actually can be done. There used to be a Linux distro called ClusterKnoppix that was based on an open source fork of a still available propietary, but free AFAICS, Linux package called Mosix.

From the admin guide:

MOSIX is a package that provides load-balancing by migrating processes within clusters and multi-cluster private clouds. MOSIX is intended primarily for distributed, concurrent computing, such as used for intensive computing.

What it does:

Provides a single-system image.

– Users can login on any node and do not need to know where their programs run.

– No need to modify or link applications with special libraries.

Automatic resource discovery and workload distribution.

But

All nodes must have the x86 64 (64-bit) architecture. All cores of the same node must have the same speed.

So it might not fit your specific requirements.

Solution 3

The applications are limited to those that make efficient use of your given resources. You can't run After Effects on your "supercomputer" unless software exists that knows how to split up the workload among all all of your slaves.

First, you don't have the required software. Even if you acquire the needed virtualization software (at whatever cost, or if the company is even willing to sell it to you!), there are minimum requirements for the cluster, which usually include being of nearly identical specifications. The closest thing I could find is VMware ESXi.

Second, there are massive penalties from the computers communicating with each other, to the point which any performance gains essentially cancel out. Sharing RAM over the network is too slow to be viable, and sharing a drive over iSCSI might not turn out to be as reliable as you expected.

Third, the power efficiency of these ordinary desktops are so low put together, you would find yourself unable to pay for this "supercomputer" within a month or so.

You want to make money with your current resources, but this is not the way to do it. Just sell your computers to buy higher-performance ones - even your antiques will sell well.

Comments

[Daniel B]

"I imagine that there must be a way that a virtual machine "thinks" that it´s installed on a single machine" - Yeah, that's definitely just your imagination. There isn't. Also, how about some paragraphs?

[Dihelson Mendonça]

@Daniel B, perhaps you didn´t understand. When I said that the virtual machine "thinks" it´s installed on a single machine, was not assuming that a "machine" could think. Indeed, virtual machines ALWAYS see the host as a physical machine already. What I am trying to say is that there shall be a way that the virtual machine sees the system as a unique machine. Thanks for the tip on the paragraphs.

[txtechhelp]

1 teraflop means 1 trillion floating point instructions per minute. Those kind of metrics can be misleading if you do not understand what they mean, and they can also be misleading if they do not have context. That 1 "teraflop" could simply mean they were able to run SuperPI on the cluster and achieve those metrics specifically with that software only.

[txtechhelp]

Recently I saw one workstation capable of 20 teraflops .. My 5 year old Intel core i7 clocked at 2.4GHz is capable of approximately 153 MEGAFLOPS .. just for reference.

[Daniel B]

@DihelsonMendonça No, I understood perfectly. You want to somehow take all resources from those PCs and combine them into a virtual computer that “would run Crysis”. However, this is not possible. Clusters work with specialized software that is cluster-aware.

[Dihelson Mendonça]

Hello, @txtechhelp , just to show you the link for the 20Tflop workstation: extremetech.com/extreme/…

[txtechhelp]

@DihelsonMendonça .. I've seen that one before, it's an impressive build, but the horsepower (e.g. the FLOPS) are mostly from the GPU's .. Specifically the CUDA cores in those cards. GPU's are massively parallel, which lend themselves well to "super computers", but are specialized and expensive. A lot of amazing work has been done to try and allow "normal" applications to run on GPU's .. but it will probably stay "sciencey" in nature since the average person only ever does web browsing and document editing (which has been done well on CPU's for 20+ years).

[Dihelson Mendonça]

Do you think that those supercomputers made by dozens of raspberry pi linked together through lan runs payware software ? Why can´t I get it and install too ? Will it be no compatible ? For what I´ve seen, there´s some Linux distribution that runs on these clusters and at least one Ubuntu is made specially for it. Of course you are right in many considerations, like the RAM and power management. What about connecting through a 10Mbit fiber ? How are supercomputer nodes connected ?

[oldmud0]

@DihelsonMendonça Dozens of Raspberry Pis do not really create a supercomputer, since the processing power is not really exceptional. What you are looking for is a Beowulf cluster, and I suggest you keep researching to see if what you want is viable or not.

[txtechhelp]

@DihelsonMendonça "super computer" nodes are usually directly linked to each other through fiber channels on the order of 10000's of Mbit/s .. not 10/100/1000 . What you're trying to achieve is completely possible and has been done since computers could be linked together, you want something like a Beowulf cluster .. you're subject to the law of diminishing returns however. A cluster of 10 Pi's clocked at 1Ghz with 512MB RAM each will not equal a "super computer" clocked at 10GHz with 5GB of RAM ...

[txtechhelp]

@oldmud0 .. OpenMPI is one of the more popular open source versions of clustering software

[Dihelson Mendonça]

Thank you for the insights, oldmund and @txtechhelp that´s really adds light to the question. I´l research more about Beowulf cluster. I´ve seen some projects using raspberry pi, and of course, they are not "supercomputers" compared to a TITAN, but considering processing, even our smartphones could be considered supercomputers when compared to those one of the 80s. Recently I saw one workstation capable of 20 teraflops. But for my Adobe After Effects I consulted a specialist at Adobe, and he said that a machine capable of making real time preview would cost at least U$ 120.000,00

[txtechhelp]

@DihelsonMendonça .. for real time previews, yes. Graphics processing is much different than "normal" processing. You might want to look at something like an NVidia CUDA cluster. But $120k for a cluster capable of that kind of processing sounds about right with current technology. You should note that a super computer is something designed for parallel processing, not stream processing (which most user applications are designed around), so actually a super computer from the 80's is still way more powerful than today's "super computer" phones.