Why HPC TOP500 never made any money and never will in its' present shape

In my August 10, 2012 post Tristan Kromer's Customer Development and User Experience  we talked about how any product and service in a startup must have clear descriptions of the people who want them. They do not have just lukewarm desires, they must really "Badly", "Intensely" desire them. We start by making this four quadrants, with the demographics, behavior and goals. This is the same figure 4, from the reference post.

This is a starting point of how the customers look like using our best guess and intuition. The next  step, we "go out of the building". This is a key tenet  of the lean company followers and I am one of them

In High Performance Computing, and whose top events are at  Super Computing SC'12  and ISC'12, have published the TOP500 list of supercomputer sites. In June 12, 2012, Sequoia - BlueGene/Q, Power BQC 16C 1.60 GHz, Custom installed at DOE is ranked #1, and it has - hang on your seats -  1,572,864 cores. There is nothing like it that in the world with over 1.6 millions cores. The #2 rank Riken built by Fujitsu has 705,024 cores, less than a half

I saw a video with top analysts and experts that took place at ISC'12 a few months ago. Jeff Hyman of Tabor Communication asks Peter Ffoulkes, 451 Analyst, Chris Wilard co-founder of Intersect360 Research and Earl Joseph, the IDC Lead specialist in HPC questions on the topic  "TOP500 -- 20 Years Later".

Who is the audience of TOP500 contest?  I treated this as a "going out of the building" exercise. So I tried to hear the experts  with an expanded consciousness, in plain words, to read them beyond the words they say.

The video's text is captioned, so you can follow the content and see the image. It is fascinating to capture the body language, not only the voice. These are the bullets I jotted (you may jot down something else), in a hand written piece of paper with drop of spritz (white wine and soda)  included.

Figure 5: Who needs TOP500?

I stopped and I read what I just discovered, without any rocket science,  using humbly the lean company teachings. TOP500 is not designed to start new companies. It's clear intention is to attract more and more money from mainly governments and corporations intertwined  in very large projects. What's more, after 20 years people forgot what the benchmark really means in practice and build dinosaur machines blindly, why? Because they can exist. There is no doubt TOP500 leads to technological progress, but it did not generate a startup culture. National pride, teams recognition are the primary goals

The whole model is a classic case history for Bob Dorf's brilliant Why Too Many Startups  Suck
Here are few bullets I extracted:

• The nastiest of all startup sins: failing to involve customers and their feedback from day 1
  
  

  

  How dinosaurs ruled the earth

 (No one thinks what to do with the dinosaurs after they are built)

... and how they look today

• Customer Discovery happen in two separate, distinct phases: “problem” discovery and, later, “solution” discovery
  How can the teams even think of customer discovery, when they are blinded by a mystical, unintelligible-to-mortals LINPACK benchmark? Who cares about FLOPS?

I have followed the list for the past several years. I think I can sum it as follows, more nodes, more cores, more InfiniBand, Blue Gene, bigger HPL number. Yawn.

• Building a solution to a problem of moderate or lukewarm interest to users is a long-term death sentence for startups,
  Do you remember who was the #1 rank in TOP500 5 years ago, without looking at the list? What they do with machine today, now that other 10x to 100x machines were built?

The ISC'12 panelists had the right insights , but all I did, is to format what they said in customer discovery format. After ISC'09  I wrote about the TOP500 : Does it still make sense? Top 500 Supercomputing list

The LINPACK test, even in its' most refined modern form - tells little or nothing of how useful the supercompter is. Yet, adding insult to injury, the Top 500 judges claim that

Any system designed specifically to solve the LINPACK benchmark problem or have as its major purpose the goal of a high Top500 ranking will be disqualified..

Then, who will use this system? It is like buying a Formula One car for personal use. I can not even drive it to buy milk without cohorts of mechanics supporting me. Never mind I can not take any passengers... and the ten million dollars price tag..

By analogy, the computers we need to make money with are the computers our customers will make money with.

We must adapt SC competitions to stimulate as many startups as possible. How? Where is a wish, there a way.

Notes added September 23, 2012.

Some readers of this blog, suggested to add two more columns to the TOP500 list. I have no influence whatsoever on these decisions, but here are my comments.

1. Add the price of the system listed to calculate the price performance 

   We do not know the price. We know the cost. The price assumes there is market for it, and it is not .

2. Calculate the ROI

   The ROI assumes there are customers who buy and create steady revenues . But  there are no customers.