Wednesday, April 25, 2012

The True Story of the Grid Engine Dream

In Sun,  they called me Mr. Grid Engine. I was the product manager of Sun Grid Engine for a decade, until January 2010. This blog is a story of my journey. Where are  we today?

Defying many predictions, Grid Engine, formerly CODINE, is the Distributed Resource Management software most used in High Performance (HPC), and it refuses to yield to the cloud business model. One day HPC will be delivered as a cloud. Not yet

If you want a metaphor, the Volkswagen Beetle created the cult car that made successful the entire company. Similarly, Grid Engine can be viewed as a not perfect, but fascinating product. It can become the launching pad for something much bigger, with a much wider adoption.   
Grid Engine is, yes - not a typo error - a cult product, an enviable positioning, which defies logic. Apple, Java, Facebook are cult products. Grid Engine has the potential grow from this reputation well beyond it is today.

Commercial supported distributions are Univa Grid Engine and Oracle Grid Engine. Both claim to be the successor of Sun Grid Engine. The Open Source  distributions are Son of Grid Engine, the latest release being 8.0.0e on April 19, 2012, and Open Grid Scheduler .

Here is a quote from Inc. magazine latest article 8 Core beliefs of Extraordinary Bosses

·         Average bosses see business as a conflict between companies, departments and groups. They… demonize competitors as "enemies," and treat customers as "territory" to be conquered. 
·         Extraordinary bosses see business as a symbiosis where the most diverse firm is most likely to survive and thrive. They naturally create teams that adapt easily to new markets and can quickly form partnerships with other companies, customers ... and even competitors.
  Grid Engine  started with an extraordinary entrepreneur, Dr Wolfgang Gentzsch, who founded Genias  Software in 1991 later re-named Gridware in 1999.

 Wolfgang  says there is only one Grid Engine community, which forms an ecosystem, which he calls a symbiosis of diversity. It all originated in Genias  It implied a huge physical and mental effort, going through Sun acquisition of Gridware in 2000 and later - when Oracle took over Sun in 2011 the creation of the GE ecosystem.

After Wolfgang left Sun, - many fine people in Sun had to leave at that time - it was frustrating to see how our efforts to have two Sun Grid Engine products (one available by subscription and one available as free Open Source) failed because of management veto. On one hand we were under pressure to be profitable as a unit, on the other hand, our customers appeared to have no reason to pay even one cent for a subscription or license.

Oracle still has IP control of Grid Engine. Both Univa and Oracle decided to make no more contributions to the open source. While in Oracle open source policies are clear, Univa, a champion of open source for many years, has surprised the community. This has created an agitated  thread on Grid Engine discussion group.

Quoting from Inc. again:
Extraordinary bosses see change as an inevitable part of life. While they don't value change for its own sake, they know that success is only possible if employees and organization embrace new ideas and new ways of doing business.
The paradox is the companies who make real big money  do things as if not interested in money. Quora has a new thread, titled  What is Mark Zuckerberg's true attitude towards money? Here is a quote from highest ranking answer:
 Mark's main motivations were pretty clearly based around materially changing the world and building technology that was used by everyone on the planet.... My impression back then was that if he had to choose, he'd rather be the most important/influential person in the world rather than the richest. And I think that's visible in how he directed the company to focus on user growth and product impact rather than revenue or business considerations. Even today, while Facebook makes a ton of money, it could probably make magnitudes more if that were its primary goal.
The Zuckerbergs

What people usually say? 

   "Well I am not Zuckerberg." or 
   "I am not Steve Jobs." or
   "We can never make Grid Engine a business of this magnitude."

   "Yes we can!"

This my answer.

Oracle is part of the Grid Engine Ecosystem, They are one of the most powerful high tech companies in the world. In September 2008, Larry Ellison dismissed the concept of Cloud. In 2012, Cloud Computing is one of the main initiatives in Oracle. 
Univa web site points out that Oracle does not have a Grid Engine roadmap. This can change any moment, as Big Data becomes the buzz of the decade. Since Oracle takeover, Grid Engine is part of the Ops Center, a business unit whose culture is not in sync with Grid Engine. This may change

Rackspace announced at the OpenStack Design Summit and Conference that it’s ready to run its public cloud service on OpenStack, an open source software they own and made accessible to anyone.  55 companies worldwide, including IBM support implementations. Oracle may get some inspiration here for Grid Engine

The Grid Engine ecosystem has extremely gifted contributors. In addition to Univa team (Gary Tyreman, Bill Bryce, Fritz Ferstl), we have a superb team from open source including Chris Dagdigian from BioTeam (the creator of the legendary domain), Daniel Templeton, now with Cloudera. 

We have Rayson Ho, Ron Chen, Chansup Byun, Stephen Dennis,  John Tseng , Dave Love, Chi Chan, Reuti and many more from either Open Grid Scheduler/Grid Engine or the Son of Grid Engine.

Sweden's Gridcore Computing has joined Open Grid Scheduler, They pre-announced that an update version of Grid Engine 2011.11 will be released at the Gompute User Meeting on May 12 2012 in Gothenburg, Sweden. They add  Dev Subramanian, a top notch Grid Engine architect to the open source Grid Engine resources.

Wolfgang Gentzsch tops the list to restore the soul of Grid Engine.  He has made it before. He might do it again.

I believe in miracles. When Steve Jobs returned to Apple, they had a month or so to dismantle and liquidate the company. But...

Tuesday, April 10, 2012

Telcos will dominate the clouds tomorrow?

The Telcos are saying  they are the right incarnation to make the cloud trusted and ubiquitous.  Telecoms Cloud Services Summit will take place in two weeks, 23-25 April 2012 in Berlin

According to Heavy Reading  only 13% of the enterprise IT is committed to cloud in 2013 leaving a plenty 87% traditionally spent IT budget to be disrupted by the cloud.

Amazon Web Services, RackSpace, OpenStack, Citrix, Eucalyptus, KVM, VMware and other gods of the cloud are not complete solutions. In a recent twiter exchange illuminaries like Lydia Leong (Gartner),  Randy Bias (CloudScaling) and George Reese (enStratus) agreed that the IaaS platform is only 10% mature.

This is my blog on this incompleteness, underlying the future opportunities.

"The cloud is not the cloud without the network." Those were the words of NTT America CTO Doug Junkins at Cloud Connect conference in Santa Clara, CA.  I said the same thing in this blog Why Include the Network Virtualization to the Cloud . As the Telcos own the networks, this is a beginning of an idea that will change both the Enterprise IT and Telcos for ever.

The Ericsson discussion paper The Telecom Cloud Opportunity acts as a Telco 2.0 Manifesto
Ericsson envisions a Networked Society, a world of 50 billion connected devices, by the end of this decade – and cloud computing, along with mobility and ubiquitous broadband, is playing an instrumental role in its creation.

Telecom operators have recognized it as a source of new revenues, with global investments in cloud services projected to more than double from an estimated US$55 billion in 2011 to almost US$130  billion annually by 2015.

By leveraging existing network-based features and their expertise in managed services, telecom operators enjoy many unique advantages over other cloud providers. The explosive growth of mobile access into the cloud also plays strongly to operators’ strengths. Ericsson sees three major roles through which telecom operators can align themselves in the cloud value chain:
  • Managing cloud connectivity,
  • Delivering cloud-based capabilities, 
  • Leveraging network assets to enhance cloud offerings.
In addition, telecom operators ...  can also choose to simultaneously  commercialise those cloud applications to open up opportunities in traditional markets such as outsourced billing for over-the-top (OTT) players, or as a vehicle to enter emerging segments like cloud-based machine-to-machine (M2M) platforms for vertical industries. Operators must consider cloud computing as an integral part of their strategies. 
This is beautiful to read and it will happen. But the technology usable by a telco needs to go a long way

Today most cloud IaaS interfaces look similar to AT&T Cloud Architect Create My Cloud

Data Center
Storage (GB) 2
Additional Storage (GB)3
Operating System
OS Specific Add-On

Hourly:  $0.100
Monthly: $50.00

This embodies the idea of a how a buyer on line acquires infrastructure. Most buyers have no idea which data center to choose (perhaps closest,? or larger capacity), how many cores, what storage to get, and so on. Also, if I select something (say 20 16 x 2.0 cores). The price is  $15,800 per hour or $10,220 per month.  How do I know whether I did not buy an oversize cloud? Should I need more resources, do I ask for them manually or the cloud is autonomic?

If a Telco takes over the cloud business , they must make the access to clouds (public, private) in a more more intuitive. Perhaps AT&T message with a price per hour 50% higher than a price per month, is "forget about cloud, let's talk about hosting." Prices of  fifteen grand per 10 hours are not manageable via web with credit cards. The AT&T hourly prices not an incentive to buy, au contraire they are a deterrent to buy, because at this stage in life they are not able to provide fast enough clouds for a duration of one day only

The cloud IaaS offerings today  are  processors, memory, storage while the cost   of the bare iron is peanuts compared to cost of networking.  Network virtualization (about 60% of the cost of the cloud) can expose as billable, premium services like (1) secure socket layer (SSL), (2) packet inspection,  (3) quality of service (QoS), traffic prioritization or data encryption. The network revenues can double and triple the cloud revenues

I suggest pre-configured clouds,  labelled like in  the fashion industry S, M, L XL, XXL . Or clouds optimized for Big Data analytics  or for SAP Oracle enterprise apps. Clouds for Games and Personal Clouds. Etc.

To develop these products I would use modern product management methodologies,-  see Working Backwards in Product Management - based on the post Steve Jobs era.

Addison Snell, the CEO of Intersect360 Research has tweeted: "You can have the best technology, but without skilled people it probably won't work out for you."

This is the largest hurdle the large Telco companies have to overcome. They have to attract and retain creative people and make them decision makers.

As I wrote elsewhere in this blog:
 There is an extraordinary connection between entrepreneurship in creativity!  The bottom line, creative must come out of some tolerable chaos.  The creation involves contraction of the known corporate and HR dominated world, to make room for a new, imperfect oasis of apparent misfits that will generate creativity so we all can be all what we can be.
Large companies - and Telco were large companies who re-invented themselves - thrived on prestige plus  a false sense of security. They learned their lesson that "prestige is just fossilized inspiration". (Paul Graham). Telecom cloud creation involves a new breed of executives a-la-Steve-Jobs (1) the ability to spot adequate cloud technologies, made by founders who never knew how a telecom provider works, (2)  to orchestrate these technologies in order to (3) deliver cloud services as easy to use as building a car on the web, offering always the option "let us find it for you".
Ease of use and ease of billing plus affordability is name of the game

Blog Archive

About Me

My photo

AI and ML for Conversational Economy