Big Computing http://vzite.com A new take popular parallel distributed computing Tue, 06 Jul 2010 22:18:18 +0000 http://wordpress.org/?v=2.2.1 en Big Computing http://vzite.com/2010/07/02/hello-world/ http://vzite.com/2010/07/02/hello-world/#comments Fri, 02 Jul 2010 04:21:51 +0000 admin As an old hand at distributed computing (for my PhD I worked on compilers for large scale distributed memory parallel machines) I have been watching with interest the new found popular interest in parallel computing typified by Map-Reduce and Hadoop. After reading the code, experimenting, reading papers and attending talks I have become convinced of a few things:

  • map-reduce is a poor fit for many many applications
  • Hadoop is deeply batch oriented, like the timeshare systems of the days of yore

These systems are still very popular because at the core they possess two important features, they are:

  • composed from redundant, share-nothing, fallible nodes
  • programmed as a network of message passing share-nothing actors

The first feature makes them relatively easy to build out, administer and maintain.  The latter makes them relatively easy to program and, more importantly to recover and continue the computation in the face of node failures.

This blog and the project it will be covering will examine the good and bad aspects of modern Big Computing (Big Data + Big Computers + Big Programs) and propose some solutions.

Stay Tuned!

]]> http://vzite.com/2010/07/02/hello-world/feed/