Approaching the problem of distributed computing using the .Net framework
I am interested in programming a project that distributes a certain computation across large files across multiple computers. The need for distributed computing arises from the unstable and unstable software that I use to do the actual computation - so it might fall on some computers, but others will surely do the job. Ideas I have included so far: -Using multiple servers, each running a task from the master server whenever possible -Using VMwares -Using a load balancing cluster
Which is more suitable for the job? Any other ideas I should be aware of?
Also, if you can recommend any reliable C # distributed computing framework, it will be helpful.
a source to share
Have you looked at Hadoop MapReduce ? It is an open source implementation of Google MapReduce. Although this is Java and not C #, it looks like it works great for your scenario; the core server automatically handles load balancing and fault tolerance in a distributed environment.
a source to share
I would take a look at Appity CloudIQ Platform . It integrates multiple machines into a single computing environment identified by a unified address. Your client simply submits jobs to a unified address, and the environment distributes jobs to individual machines. It also monitors the execution of tasks and can automatically restart failed tasks. So if your application is prone to crash, this structure might be perfect. Rather than submitting the same job to multiple computers (and wasting CPU) to cover in case of failure, just submit it once and let the structure handler restart jobs that actually fail. I would find it ideal for your reliability.
a source to share