Thursday, February 07, 2013

Looking at Old HPC IDC Predictions (2010)

It is  more interesting what was predicted by top guru analysts for 2010, now that we are in 2013

Idc Hpc Web Conf Predictions 2010 Final 

IDC predicted Infiniband will be king, on the "evolutionary" supercomputers of 2010 (meaning old technology, stalled thinking) as described on slide 9
  • Most of the biggest, baddest supercomputers are architectural clusters or x86 MPPs, with bulked-up interconnects and support for MPI or PGAS languages
  • Revolutionary change may still be on the horizon, because evolutionary change is not addressing fundamental issues, such as how to program machines that will soon have 1M or more cores
This blog covered the first bullet in the article about Higgs Boson calculations at CERN. Large Hadron Collider people assembled a grid of 150 data centers in 40 countries using opportunistic cycles and resources. delivering the power of High Throughput Computing. There is no supercomputer yet built  using the "evolutionary" thinking (like TOP500 thinking) to deliver this power, never mind the prohibitive costs.

The second bullet, on how to program 1 million cores or more, is dealt in a few blog entries here, like the Interview with David Ungar  He discovered the  genial  "heresy" of not communicating between nodes, an Aha!  similar to the non-Euclidean geometry. We must decide what precision we need, because there are  not absolute single answers in nature. And the precision we need is directly proportional with the cost to produce such calculation.

This is why I am impressed by IDC predictions

Wednesday, February 06, 2013

How Bosco simplifies your life on Amazon EC2

You will find a practical, no non-nonsense, step by step directions  to submit your tens of thousands of jobs to Amazon, at the lowest spot price you possibly can, using OSG's Bosco, the quintessential High Throughput Computing tool.

Quote from Derek Weitzel blog, for easy reference:

Pros of using Bosco to submit Amazon EC2 Jobs:

  • Simple management of Amazon instance from your workstation.
  • Specify spot price right inside of the job description.
  • Ability to bootstrap the instance easily with user data scripts.
  • Ability to use HTCondor policies in order to manage the instances, such as periodic remove statement above.


  • The EC2 universe is only available on Linux builds of Bosco   You cannot manage EC2 instances on the Mac version of Bosco.
  • Amazon EC2 has hundreds and hundreds of features, Bosco only allows you to use the simple submit EC2 instances and spot pricing.  You will not be able to use the vast majority when you are using Bosco to manage your instances.  But if all you need is to run some processing, Bosco is great!

To download Bosco, click below

 let us know how it works for you.

Blog Archive

About Me

My photo

AI and ML for Conversational Economy