Friday, January 21, 2011

From HPC In The Cloud "Univa Revving the Grid Engine Revival"

There is a sort of happy ending to the Grid Engine saga following Oracle’s formal acquisition of Sun in January 2010, which I described in my post The Fate of Oracle’s Grid Engine.
Word of this arrived today (January 19, 2011)  via a press release from Univa,

Tuesday, January 18, 2011

Tell us your preferences: Cloud IaaS providers

To all the readers of Part 2 and Part 1 , I prepared a survey to learn how you select the infrastructure cloud providers

Click here to take survey

I will publish the results on this blog periodically. I want to thank to all the people who read this blog  and amazed me.

A million thanks!

The following IaaS providers are included. List is not comprehensive, but I may cover others in a follow up survey:

Friday, January 07, 2011

Part 2. Getting out of the Trough of Disillusion Will cloud computing be adopted massively in 2011?

Dilbert strip licensed by United Feature Syndicate, Inc.

We have seen  in Part 1 of this post  that Cloud Computing is the least  considered (15%) and most rejected (46%) new  emerging technology in Data Center space

This statement is  based on research from the largest organization of Data Center professionals - AFCOM.  Their survey has 436 respondents members from 27 Countries worldwide. 83.3%  in the U.S., 16.7% Overseas. representing: Private Industry 84.5%,  Government Agencies 8.1%, Colleges or Universities 7.4%,  Respondent’s Personal Budget Responsibility: $3M+ 41.9%, $5M+ 29.3%  and for  $10M+ 19.4%.  

For 2011 there must be a real osmosis between the Data Center professionals and the cloud computing community. I discovered a total disconnect between  Cloud Computing providers- who talk in terms like PaaS, IaaS and SaaS, which is the focus of the technology creation, -  and Data Center owners - who are traditional consumers of the Data Center enterprise technologies and  who own the most significant  budgets that cloud computing must leverage to gain widespread adoption.

We are , using Gartner Hype Cycle, in the  the  Trough of Disillusion and we need to reach the Plateau of Productivity . You can follow the answers on Quora to the question Has Cloud Computing reached its peak in terms of hype? According to J. Herman - artist, writer, producer -   "Cloud computing is still PO, meaning Pre -Oprah. The moment Oprah declares cloud computing cool... then the hype will peak"


In the eyes of the Data Center folks we are just the philosophers, technologists and artists of the cloud computing.  We get just as much credibility as tiniest middle character from Dilbert strip at the top of this page. The size hints we are little blah-blah  dwarfs for now. Our rhetoric - which we call "messages"- did not reach our target audience. 

Cloud Computing  hopes for 2011

Here are the important companies for cloud computing, according to Gartner's  Magic Quadrant for Cloud Infrastructure as a Service last updated on December, 2010This  paper is one of the best analyst documents I have come across. The distinct groups in  the Magic Quadrant are

  • Self-managed IaaS, for cost-effective agile replacement of traditional data center infrastructure.
  • Lightly managed IaaS, for customers who wish to primarily self-manage but want the provider to be responsible for routine operations tasks.
  • Complex managed hosting, for customers who want to outsource operational responsibility for the infrastructure underlying Web content and applications.

For a vendor to be included above, here are Gartner's criteria
  • They must sell on-demand hosting as a stand-alone service, without the requirement to bundle it with application development, application maintenance or other outsourcing.
  • Their services must be enterprise-class, offering 24/7 customer support (including phone support), SLAs, and the ability to scale an application beyond the capacity of a single server.
  • They must have significant market presence, as indicated by Web-hosting-related revenue of at least $50 million in 2009, or an on-demand hosting revenue run rate of at least $25 million in 2010.
  • They must have demonstrable global presence. They must have reference customers in North America, Western Europe and Asia. They must have data centers in North America as well as either Western Europe or Asia, or they must derive at least 20% of their hosting revenue from customers outside the region in which they have their headquarters.
The third bullet eliminated many  promising  companies who are no-where near $25M in annual revenues.

If you look at the 10 best financed companies in Cloud Computing out of 10 companies, only one is listed: Joyent . This is one of the most hopeful Palo Alto cloud companies. They host what they preach and are nice people to do business with.

Companies like Rightscale, Cloudera, Eucalyptus are not listed on the Magic Quadrant.  One may argue they are not in the infrastructure business, But this is a fuzzy distinction as we will see later on. What it matters here that these companies attracted top investment funds and that the VC community believes they will deliver the expected revenues.





Ability to Execute and  Completeness of Vision

These are two axes of the Magic Quadrant. This is in a nut shell how the companies are evaluated
  •  Market Understanding and Product Strategy are the highest ratings possible in Completeness of Vision
  • Product/Service and Customer Experience are the highest ratings for Ability to Execute
To get into the Data Centers, we need to understand this market, the way it is now, setting aside our belief-in-Nirvana that cloud computing will bring. In fact, we need to completely forget any solutions we have in our mind when interviewing for product management purposes a significant customer. We need to be humble   and respect for the all data center classic practitioners.Then, as follow up, we need to test the Data Center Customer experience, when compared  to the one that have today.

As vision of infrastructure providers, one of the better companies having this approach is RackSpace, whose philosophy is  expressed in the graphic below.

They seem to offer pure hosting, managed hybrid clouds and clouds as fluent transition among the products.

Making IaaS revenues in 2011

John Considine from CloudSwich writes on his blog on January 4, 2011
From IAAS providers, we’re seeing a trend to offer more PAAS services. This is apparent in Amazon’s offerings
These tendencies of adding PaaS creates a complexity that hinders adoption. Amazon, by far the market leader is, according to Gartner  "Amazon is a thought leader; it is extraordinarily innovative, exceptionally agile and very responsive to the market. It has the richest cloud IaaS product portfolio, and is constantly expanding its service offerings and reducing its prices"

However Gartner has some reservations on the Amazon business model:
  • Amazon does not offer any managed services.
  • Amazon is the only evaluated vendor that does not also offer the standard options of colocation, dedicated nonvirtualized servers (often used for databases), and private non-Internet connectivity (although Amazon will negotiate peering). These components are critical for many customers, who need hybrid, not pure cloud, environments.
  • Amazon has the weakest cloud compute SLA of any of the evaluated competing public cloud compute services, even though its uptime is actually very good.
  • Amazon is a price leader, but it charges separately for optional items that are often bundled with competitive offerings. Prospective customers should be careful to model the costs accurately, especially network-related charges. Support is not included — it is a 10% to 20% uplift to the price, and it is geared primarily toward technically knowledgeable, expert users.
  • Amazon's offering is developer-centric, rather than enterprise-oriented, although it has significant traction in large enterprises...Amazon will negotiate and sign contracts known as Enterprise Agreements, but customers often report that the negotiation process is frustrating.
  • In other words, the Amazon model is not Data Center friendly. To port a Data Center center on Amazon, is an extraordinary feat of complex engineering, requiring top experts. Netflix port was breakthrough but it gives headaches even a year later . See 5 lessons a we have learned using AWS
Talking about Amazon AMI (Amazon Machine Interface), most people pretend they understand them and that are simple to write. My experience is 99% of mainstream cloud users have not a clue on how to write an AMI. So we are ashamed to admit it, and we praise the new clothing of the emperor, ever if we can not see it.




 How much is it worth to enlarge the market share for IaaS?

 The Economist's article Tanks in the  Cloud tries to figure outhow much money IaaS makes. Why IaaS? Because "the most interesting layer—the only one that really deserves to be called “cloud computing”, say purists—is “infrastructure as a service” (IaaS)."

Economist, quoting among others the well known cloud expert Randy Bias estimate AWS revenues for 2011 at $750 millions per year. Taking a total estimate for all companies from Gartner Magic Quadrant, maybe we get $1.5 billion per year (being generous). So we have a long to go up and up and up.

The most recent and conservative estimates from 451 Group  is 16.7 billions for 2013. Other Analysts including IDC estimated 50 billion per year plus. There are at least $10 billions per year to be made. The study includes SaaS revenues The entire companies from the Magic quadrant can not have a combined growth to reach the 50% of the 16.7 billions, unless significant new players will appear in the next year or two.
The time is ripe for a Google-like or Facebook-like player on IaaS space,  Economist says
"Computing clouds—essentially digital-service factories—are the first truly global utility"
We are close to the Plateau of Productivity or - as Quora's contributors call  it - the Oprah-ready Cloud Computing
David, Regina  and  Miha

Tuesday, January 04, 2011

Getting out of the Trough of Desilusion Will cloud computing be adopted massively in 2011? Part 1

For 2011, will cloud computing be adopted massively in enterprises? There are many definitions in cloud computing, the most common being National Institute of Standards (NIST), but no one adopts a new business model based on a definition. To me cloud computing is about making money with IT by delivering the infrastructure (IaaS), the platforms (PaaS) and the software applications (SaaS) as pay-per-use services. Many times I read that cloud computing reduces the costs, the so-called Total Cost of Ownership (TCO) – one of the most difficult to measure and calculate real-time. If the cost reduction is the objective, then we must shut down the IT to have zero costs.

This is not obviously the goal. The goal is to make money, to maximize the Return for Investment (ROI) for IT. If the IT pays-per-use its' suppliers, and then invoices the internal and external customers also as pay-per-use, one can calculate instantly the profit. If by spending $1M to have revenues of $3M makes sense, the IT might discover that doubling the costs to $2M might lead to $7M revenues. In this case we doubled the TCO (expressed for simplicity as pay-per-use to suppliers) to generate a much higher profit


In real life, the transition from silo-ed Data Centers to cloud computing is much slower than expected by pundits. The actual studies from the field show that cloud computing is still little understood and perceived as risky by data center rank and file. These are the folks that be convinced by Cloud Computing and they are not.

The InformationWeek 2010 Data Center Trends ranks cloud computing the last trend

  1. Top of the Rack (ToR) switching model
  2. Converged networks (Converged Enhanced Ethernet)
  3. In-server cooling
  4. Regulation and efficiency
  5. Built-in Power Management
  6. Cloud computing
The list of the trends shows that the Data Center crowd thinks and acts based on classic, static, silo-ed business models they are used from the past. There is no paradigm shift yet. I quote:
For the vast majority of organization with data centers, public or private infrastructure services will be restricted to extending systems in place today. Few can afford to jump wholesale into outsourcing, even if security and availability concerns permitted--and that's a really big "if."
Assuming resource management tools are in place, the relationship with the cloud will not be one of simple "resource on demand." Like the steel mill's relationship with the power grid, the future relationship between the private enterprise data center and the public cloud will likely be one of constant interaction, monitoring where processing is most cost effective for your particular service-level agreements. In all cases, dynamic control and monitoring will be a significant part of the data center staff's responsibilities.
Many IT departments are defensive when discussing cloud technology, and they raise legitimate concerns. Disaster recovery becoming a challenge for smaller companies that end up low in the pecking order for resources. According to an expert
"More facilities are being built in places with cheap power, and the server farms are sold off in blocks as 'cloud computing' to other companies," he says. There's minimal expertise on site, and all data flows through one big pipe, disguised as multiple entry points from multiple directions but all within the same country's infrastructure.
"Then, a disaster occurs," he says. "The owner of the data center uses the resources it needs first but lacks the labor to reroute network capability. Small and midsize customers that had their critical data in the cloud may find themselves well down in the 'please hold' queue."
The upshot: If "IT"in your organization translates to nothing more than "governance of your contract with the cloud computing provider," you could be in major trouble. On the other hand, companies that judiciously incorporate infrastructure-as-a-service capacity will eventually drive a higher level of IT sophistication and demand. Cloud technologies require a very clear understanding of application performance requirement, service-level agreements, operational and business priorities, and costs across all aspects of service delivery

Other reference is the AFCOM (Association of Data Center Management Professionals) 2009-2010 Data Center Trends survey. From the emerging technologies, the virtualization is the most considered (73%) and least rejected (10%)

  • Virtual Processing (72.9%)
    • considered, but rejected by 10% more
  • Web Applications (70.4%)
    • considered, but rejected by 5% more
  • Automation (54.8%)
    • considered, but rejected by 15% more
  • Cluster Computing (50.0%)
    • considered, but rejected by 12% more
  • Cloud Computing (14.9%)
    • considered, but rejected by 46% more

Cloud Computing is least considered (15%) and most rejected (46%) new emerging technology in Data Center space.

AFCOM was so surprised by the findings, that it prepares a new study exclusively on Cloud Computing can be adopted in Data Center. The study is targeted for publication for March 2011

Before Cloud Computing, we had Grid Computing, which failed to reach mainstream the enterprises, in spite at the benefits. It was too complex to set up (or perceived as too complex) in spite of a nirvana of benefits promised and proven, if some expensive consultants were hired for life.
Gartner Hype Cycle for high performance workplace reveals importance of enterprise portals, cloud-based grid computing, hyperconnectedness and media tablets. That go through a inflated rags to riches fuzz stages. Grid Computing never made it beyond the Trough of Disillusion.

Can we avoid that Cloud Computing does not end as Grid Computing and we start the Slope of Enlightment towards the Plateau of Productivity?

We see right now , in spite of AWS success, in spite of market sizes of up to $160B per year in size coming from analyst reports, the Data Center strategists simply ignore Cloud Computing. We need to face this challenge and work directly the traditional IT enterprise staff.
The human factor requires a symbiosis with cloud providers. Making enterprises use public clouds, for example should not be geared by profits only. Cloud Computing should be part of a new social contract, as described by Umair Haque  in his book  The New Capitalist Manifesto: Building a Disruptively Better Business
The twenty-first century capitalist’s agenda, in a nutshell, is to rethink the “capital”—to build organizations that are less machines, and more living networks of the many different kinds of capital, whether natural, human, social, or creative.

Nothing fits better these attributes than cloud computing. I will try to answer more  questions in Part 2 of this post



Blog Archive

About Me

My photo

AI and ML for Conversational Economy