Create cluster in XP (supercomputer)

[quickoldcar]

Hi all,

For a long time I always wanted to combine my good and some of my useless computers together to make a sort of supercomputer. I recently played around and did tons and tons of reading and accomplished my goal. It's easier to do this with linux than with xp. There are Live CD's for linux that you simply just boot off of on all your computers and have an instant cluster. But I prefer to stick with XP and I'm used to it. So with that said I use a live cd version of linux called Knoppix.

I'm not an expert at this nor do I claim to know a lot about this subject.
What I am going to do for everyone is try to tell the easiest way to accomplish this task and mention a few things about it.

I run XP Professional on all my computers. I am using vmware Program (The free vmplayer is supposed to work also) . All my computers are networked together in a home filesharing environment. Meaning they are all on the same router. And it will do wireless usb's as well.

First item you'll need to get will be the latest copy of Clusterknoppix. Download the .iso image and save it in a folder somewhere on your computer. I use this for a number of reasons as I'll explain. The cluster in the title tells me there is a program called Openmosix already installed into the linux os. One less item for you to do. I seem to have less driver issues with the Knoppix versions of linux.

Get this: http://clusterknoppix.sw.be/download.htm

Some openmosix reading material: http://openmosix.sourceforge.net/

Vmware reading and information: http://www.vmware.com/

A good article describing clusterknoppix and openmosix: http://www-128.ibm.com/developerwork...clustknop.html

Here is the order to make your cluster:

1. Have 2 or more running computers with XP operating systems.

2. Install Vmware program on all computers.

3. Start vmware , it will create a new virtual system inside xp.
(clusterknoppix will run inside this)

4. We now need to create a new virtual machine. Click "file", then "new", then "new vitual machine". To make this easy just keep clicking next untill you are done. Ignore what settings you choose.

5. Now we "edit virtual machine settings"

6. Highlight this: CD-ROM (IDE 1:0)
Checkmark "Use ISO image".........now browse for the .iso of your clusterknoppix and use that.

7. There are 2 tabs...Hardware and Options. You can go to options and rename your server if you like but isn't necessary.
The default settings for memory usage seem to work well and it's different on any machine you try it on. I suggest you keep default to try first.

8. Click "OK" and look towards the left side under commands. "Start the virtual machine". You should now see a Knoppix screen. Click inside the virtual window and hit enter to speedy things along. Knoppix should find all your drivers and be ready to go.

To get your "mouse is stuck inside virtual machine window" out you hit the shortcut which is Ctrl+Alt. You now have 1 machine set up and must do it on any others the same way I described above.

The nice thing about all this is that the openmosix program is already running and will autodetect the other computers. Wireless sometimes takes longer so be patient.
If you would like to see the openmosix program in action you click the yellow K at the lower left of the taskbar. Then scroll up to openmosix in the start menu.


Hopefully everything works out for you and you can get some good use from my somewhat of a tutorial. I wanted to say that this does a load-balancing network cluster and will share cpu power as well as alloted physical memory. helps me for doing some of my huge 3dmax renders I do. Otherwise I can not do them at all.

I'm sure there is many people out there wanting to be able to Run XP and also Linux at the same time. Well here you go.

If you just need to cluster a few computers together you can just burn a few clusterknoppix cd's. Then set the computers to boot from cdrom.

Clusterknoppix is a livecd and doesn't even require there to be a harddrive in the computer.


Here's my cluster: http://s57.photobucket.com/albums/g2...ckscluster.jpg
and also
http://s57.photobucket.com/albums/g2...ster4comps.jpg

I use dual monitors and thats why the screen is so wide.

[rob-antwerpen]

hi,

I was reading you article and it seems very simple
Were this the only things you did to make it work?

How much faster was your cluster?
Here in school i have the possibility to cluster 40 dualcore pc's 2,7 GHz with 2 GB RAM each
so i guess it should be pretty fast

cheers,
Rob
t.linders@ha.be

[quickoldcar]

I never did do any speed tests, but was a very big difference.

I used vmware, virtualpc2007 is free now, or can run the live cd's directly each computer.

What makes this great is that your cpu should never hit 100%, it will take from the "pool", and you should not run out of memory as easily.

40 dual cores and 80 gig physical memory, that would be pretty good I'd say.

You will still have to deduct any usage from the main os and clusterknoppix, but it didn't matter to me much when you have access to the "pool"

[bobthebob]

what all is shared? like if i do this with 2 computers each with a high end graphics card, will those also be shared and it be like i have 2 highend graphics cards plugged in or will the only graphics card in the main computer work?

[rob-antwerpen]

Hi again
Which software from VMware did you use; there are soo many!!!
I tried VMPlayer but that didn't work

thx

[rob-antwerpen]

hi

i installed vmware desktop and made a virtual machine that started up
the knoppix iso as descirbed (so far no problem :-))

But i use Solidworks and i don't seem to find it back.
Do i have to re-install SolidWorks in order to let it run?

Can i only use software that can run on knoppix?



thx,
Rob

[GroundZero3]

So my understanding is you are just running clustering Linux distribution in a vm application. How does this exactly help improve the performance of XP?

[J1mmy]

Interesting...

Do you have any comparison charts as to how well it increases performance? Do they do a similar like this to manage large traffic websites?

[ShyguyXPC]

hi

i installed vmware desktop and made a virtual machine that started up
the knoppix iso as descirbed (so far no problem :-))

But i use Solidworks and i don't seem to find it back.
Do i have to re-install SolidWorks in order to let it run?

Can i only use software that can run on knoppix?



thx,
Rob

If your using Solidworks, doesn't it have a Network rendering feature or Render node aspect of it? allowing you to use multiple PC's on the network to assist in rendering? Most decent mid level to high end 3D CAD/CGI programs usually have something like that.

[rob-antwerpen]

If your using Solidworks, doesn't it have a Network rendering feature or Render node aspect of it? allowing you to use multiple PC's on the network to assist in rendering? Most decent mid level to high end 3D CAD/CGI programs usually have something like that.

i don't know, but the main reason i want to use a cluster is for
fea calculations. In our present configuration some calculations tend
to take about 6-8 hours. Hopefully clustering will bring down the time


Rob

[rob-antwerpen]

Hi all,

For a long time I always wanted to combine my good and some of my useless computers together to make a sort of supercomputer. I recently played around and did tons and tons of reading and accomplished my goal. It's easier to do this with linux than with xp. There are Live CD's for linux that you simply just boot off of on all your computers and have an instant cluster. But I prefer to stick with XP and I'm used to it. So with that said I use a live cd version of linux called Knoppix.

I'm not an expert at this nor do I claim to know a lot about this subject.
What I am going to do for everyone is try to tell the easiest way to accomplish this task and mention a few things about it.

I run XP Professional on all my computers. I am using vmware Program (The free vmplayer is supposed to work also) . All my computers are networked together in a home filesharing environment. Meaning they are all on the same router. And it will do wireless usb's as well.

First item you'll need to get will be the latest copy of Clusterknoppix. Download the .iso image and save it in a folder somewhere on your computer. I use this for a number of reasons as I'll explain. The cluster in the title tells me there is a program called Openmosix already installed into the linux os. One less item for you to do. I seem to have less driver issues with the Knoppix versions of linux.

Get this: :: ClusterKnoppix - Get it while it's hot!

Some openmosix reading material: openMosix, an Open Source Linux Cluster Project

Vmware reading and information: VMware: Virtualization via Hypervisor, Virtual Machine & Server Consolidation - VMware

A good article describing clusterknoppix and openmosix: Craft a load-balancing cluster with ClusterKnoppix

Here is the order to make your cluster:

1. Have 2 or more running computers with XP operating systems.

2. Install Vmware program on all computers.

3. Start vmware , it will create a new virtual system inside xp.
(clusterknoppix will run inside this)

4. We now need to create a new virtual machine. Click "file", then "new", then "new vitual machine". To make this easy just keep clicking next untill you are done. Ignore what settings you choose.

5. Now we "edit virtual machine settings"

6. Highlight this: CD-ROM (IDE 1:0)
Checkmark "Use ISO image".........now browse for the .iso of your clusterknoppix and use that.

7. There are 2 tabs...Hardware and Options. You can go to options and rename your server if you like but isn't necessary.
The default settings for memory usage seem to work well and it's different on any machine you try it on. I suggest you keep default to try first.

8. Click "OK" and look towards the left side under commands. "Start the virtual machine". You should now see a Knoppix screen. Click inside the virtual window and hit enter to speedy things along. Knoppix should find all your drivers and be ready to go.

To get your "mouse is stuck inside virtual machine window" out you hit the shortcut which is Ctrl+Alt. You now have 1 machine set up and must do it on any others the same way I described above.

The nice thing about all this is that the openmosix program is already running and will autodetect the other computers. Wireless sometimes takes longer so be patient.
If you would like to see the openmosix program in action you click the yellow K at the lower left of the taskbar. Then scroll up to openmosix in the start menu.


Hopefully everything works out for you and you can get some good use from my somewhat of a tutorial. I wanted to say that this does a load-balancing network cluster and will share cpu power as well as alloted physical memory. helps me for doing some of my huge 3dmax renders I do. Otherwise I can not do them at all.

I'm sure there is many people out there wanting to be able to Run XP and also Linux at the same time. Well here you go.

If you just need to cluster a few computers together you can just burn a few clusterknoppix cd's. Then set the computers to boot from cdrom.

Clusterknoppix is a livecd and doesn't even require there to be a harddrive in the computer.


Here's my cluster: quickscluster.jpg - Image - Photobucket - Video and Image Hosting
and also
quickcluster4comps.jpg - Image - Photobucket - Video and Image Hosting

I use dual monitors and thats why the screen is so wide.

hi
another question
how can i see inside clusterknoppix which computers are connected?


thx,
rob

[ShyguyXPC]

ahhh, I see, when you mentioned solid works thought maybe was some sort of complex/hi-res rendering that needed to be done or something.

[hecavigo]

I don't quite understand if you are using the 3dmax directly in your xp or inside one of your vms. could you specify if there is a way you found to link the vm clusters to the host OS (in this case XP) so the applications running in XP get the benefit of extra CPUs/Memory?
btw Thanks for the Thread and the links associated, excellent info.

[spektyr]

I've been looking to see if there is a way to cluster XP systems and found this thread. I haven't tried it yet but I take it that this procedure creates a virtual Knoppix cluster running within XP but XP programs have no direct access to it and therefore cannot benefit from it. I may try booting the systems to clusterKnoppix and test it with Qosmic and Electric Sheep just to get a performance benchmark for my network, which consists of PIII and P4 systems.

It will be nice when more applications become CUDA compatible so that the new quad-Tesla systems can be used. Talk about the Ferrari of rendering machines! ;-)

Spek

"I'm a PC and I run Linux. B-p"

[wampthing2]

i don't know, but the main reason i want to use a cluster is for
fea calculations. In our present configuration some calculations tend
to take about 6-8 hours. Hopefully clustering will bring down the time


Rob

Soldiworks (and its FEA simulators) do not support hyperthreading, so a cluster would not be beneficial if that is all you are looking to use it for. The best thing for Solidworks is to get as much memory as you can and the fastest CPU you can. Then set the priority of Solidworks.exe as high as your task manager will let you. This brought my calculation times down from 12 hours to 1 hour in most cases. I doubled my ram from 4 GB to 8 GB and rearranged my task manager priorities so that Solidworks was the primary focus of the CPU.

[Taxmancometh]

So how do you really share the processor of more than one XP computer? Looked everywhere and all I could find was networking cluster type stuff and Linux. Nothing about sharing processors. You would think it would be an easy process. Maybe M$ doesn't want to go down that path.

[waesh]

I like the idea. recently I have bought a PC with core i3 3.3 GHz, 4GB DDR-3 Ram. I have a old one with celeron D 2.53 GHz and 256 DDR-1 RAM. Is it possible to cluster them?

I want to use Win 7 in i3 CPU...

[GroundZero3]

No, desktop operating systems for windows arent designed to create clusters

[Mwesten]

it sounds to me like all you did was create a couple os's inside a virtualbox.. thats not really creating a cluster, i do that all the time, throw 2 or 3 os's in a virtual world and share the folders. creating a cluster is more like having 2 or more computers and having them all execute parts of a job to get it to finish in double the speed. and creating it inside a virtual world doesnt effect your computer at all...... because its virtual.

[Interrupt]

You can't create a cluster like that... especially not for Windows (in fact, I think GZ is right and that you can't do that in XP to begin with). Correct me if I'm wrong but there are types of projects you can do for distributed computing, there's also render nets to consider, but as far as general computing and sharing CPU resources that isn't possible using a standard Windows desktop edition.

Server 2008: You can create clusters in Windows Server 2008 but they aren't done the way you described.

Edit: I didn't realize this thread was nearly a month old... I figure the OP ditched the idea by now?

Proxmox Virtualization Cluster: https://www.youtube.com/watch?v=4J5v...eature=related (different from what you're describing, OP... I've never even attempted this so I can't be responsible for any damage this may cause!)