How to set up "poor man's" Linux desktop cluster?

Question

There are several small Linux/Android devices on the market with 1 GHz ARM processor, 512 MByte RAM and cca. 4 Gbyte storage for $35-$40. If I buy 5 pcs, it's only around $200.

Let's say, we have 5-10 pcs of them, and we've solved basic things, e.g. they run some not too rare kind of Linux distribution (Debian, say), we have root access, and they are connected on local network, with IP. So, everything is ready for make a cluster (I think).

We have a desktop computer or notebook (cca. 2 GHz x86 processor, 1-2 GByte of RAM, several GBytes of HDD/SSD space), which also runs a standard Linux distro. It has keyboard, mouse, monitor, we want to use it as the "frontend" of the cluster.

So, the question is: what's the easiest way to utilize the horsepower of a couple of small computers with a desktop computer?

Desktop programs are not designed to work in that environment, so I don't want to set up a real cluster, the project is about just to push some jobs from the desktop/netbook computer and share it as optimally as possible amongst the small ones.

Say, I should start the browser on a small computer, using X11, it will not eat up my desktop machine's memory. If I have many pages open, I can start another one on another small machine.

I can also start my text editor on a small computer, I need just share the folder I'm working to it. It has a small problem: I'm using Geany, and I start the compiler by pressing F8, so the small machine should launch the compiler on the desktop machine. I don't mind, if I have to write a small script for it.

What's your advice, would it fly?

Solution 1

An approach that works fairly well for me...

• Connect one of those obsolete monitors you have lying around "just in case" to each of the small computers (RaspberryPi, etc.).

• Run a tiny, fast, RAM-based O/S like Puppy Linux (see how it works) on every computer.

• Setup passwordless (pre-shared password distribution) SSH between all computers.

• Install KVM software like Synergy on every computer, running the "server" on the computer with the keyboard and mouse. The others will be "clients". Synergy can also optionally be run through the SSH for better security.

• Use fuse SSHFS (preferred) or NFS to mount storage devices on boot.

• You may also want to setup booting across the network (PXE, etc.).

Now you have your own multi-monitor console!

With the dramatic increase in the size of your visible desktop, 2.5-7GB RAM and 6-11 "cores", you can run multiple browsers (I've found Chromium to be the easiest on resources) and thus be able to see many pages at once as you look up the syntax of commands and do other research while programming or writing.

Solution 2

I'm a little confused what exactly you want your cluster to do. It sounds a bit like all you want is starting programs and running them on different boxes. If that's the case, SSH/X-forwarding/NFS should do you ok.

IMHO this doesn't really have anything to do with clustering, its simply remote controlling different machines. I really have to ask one thing though. Why all this trouble mate? For $400 dollars you can build a beast of a PC these days, with plenty of memory for running desktop applications :)

Solution 3

It seems you are looking for something like the Beowulf-cluster.

This is a cluster developed for parallel computing on a number of "boxes".

Comments

[JustDanyul]

try reading the "what can't a beowulf cluster do" part of the link you posted. from reading his post, it doesnt seem to me like beowulf is what he is looking for :)