Thursday, January 28, 2016

An Interview with Derek Collison, Apcera’s CEO and founder

I interviewed  Derek Collison, This is how describes him
Derek Collison is founder and CEO of Apcera. An industry veteran and pioneer in large-scale distributed systems and enterprise computing, he has held executive positions at TIBCO Software, Google and VMware. At TIBCO, he designed and implemented a wide range of messaging products including Rendezvous and EMS.
Derek is not from an Ivy League University (he graduated in University of Maryland) , yet his career is a string of visionary projects in distributed computing that brought immediate tangible benefits.

As Walter Isaacson, the biographer of Steve Jobs wrote in New York Times,
Smart and educated people don’t always spawn innovation. America’s advantage, if it continues to have one, will be that it can produce people who are also more creative and imaginative, those who know how to stand at the intersection of the humanities and the sciences.
Paraphrasing another comment from Walter Isaacson, "intuition was based not on conventional learning but on experiential wisdom. He also had a lot of imagination and knew how to apply it"

Mr Isaacson spent months next to Steve Jobs. I spent a few hours with Derek Collison and it felt as if these words were written for him as well.

Apcera majestic Art Deco office building in San Francisco 

The User Problem 

Miha: What is the user problem you are trying to solve?

Derek: At Tibco Software,  between 1999 to 2003, we were experimenting with high-speed messaging, integrations, middle-ware and so on. Even at that time we started to see how we can get a lot of resources together, to get something done. The message bus or integration bus was put together in a big fat app – for Wall Street.

Miha: Then you went to work for Google.

Derek:  Yes. I spent six years (2003 to 2009) there I was watching the development of web, and mobile, coming out. In addition to building survey apps and  windows apps, engineers had to build web apps. At the time, it was kind of horrid. Ruby on Rails came around this time, to try to solve and simplify building these new web apps. It became interesting to me to watch how the deployment become harder (and I mean very, very much harder) while the development became faster and easier.

Miha: This is was really a game changer

Derek: Google came up with new rules

  1. Hardware does not talk to software
    • Hardware was not an issue; we had a sea of infrastructure 
  2. There is no hardware for testing or hardware for production but just one big cloud.

And software people like me, we did not talk to machines, we talked to an Intermediary platform called the Borg.

Miha's note

See Wired Return of the Borg: How Twitter Rebuilt Google’s Secret Weapon ...the software system that orchestrates the whole thing ... is called Borg, and it’s one of the best-kept secrets of Google’s rapid evolution into the most dominant force on the web. [Google engineer] John Wilkes won’t even call it Borg. “I prefer to call it the system that will not be named,” he says.
 In 2014 Google decided to offer Kubernetes  as an evolution of Borg The developers from Borg moved to  Kubernetes  team. Google's Kubernetes is now part of  OpenStack.  

Derek:  The Borg was very rudimentary at the time, but it did something that made me feel empowered, Google trusted the BORG, not me ☺ . They trusted that platform of technology to protect their core business, and allowed us in software development to move fast.

Miha: How did Borg work?

Derek: The Borg principle was "Convention over Configuration".  It means a developer only needs to specify the unconventional aspects of the application
Google started to do the same thing to deployment. I was not involved, but they had a project called Google App Engine  which as a platform offers a limited number of choices for deployment, but it is much easier to use

Miha: How did it work in practice?


  1. Let Google worry about database administration, server configuration, sharding and load balancing. With Traffic Splitting, you can A/B test different live versions of your app. 
  2. Choose the storage option you need: a traditional MySQL database using Cloud SQL, a schemaless NoSQL datastore, or object storage using Cloud Storage. 

Multitenancy support lets you compartmentalize your application data.
[This is called “opinionated stacks” like “opinionated software”]
Miha: How come you then left Google to join VMware?

Derek: When Paul Maritz succeeded Diane Green as CEO,  he knew of our team at Google. He came over and said:  “I need you guys to come over and do something cool in VMware”. I personally was not interested in virtualization, so I came up with a new idea called project B29. This was  an enterprise PaaS (Platform as a Service). It had to do with Java applications, with appservers and databases using some sort of opinionated stacks.

As soon as the platform was launched I sat back and said to myself, “I missed the bigger and harder problem”. The bigger problem was not to speed up deployment, but to deliver a trusted platform, seamlessly and transparently.  In Google we had Borg, but what about the outside world? Here we have devs and devops, running separately from each other, deploying the most modern technology, which today is Docker containers, tomorrow who knows what that will be. This is not the hard problem. The hard problem is how to get over security and trust which allows both devs and devops to deliver faster.

Miha: Docker security is a big issue now,

Derek: Of course. When we started the company, I said: “Hey, the security system the way they work today (firewalls, VLANs), they are not going to work or scale to fit in this new world. They are too slow to respond, we need a better way to do this. This is not just about container security standards, like in the OCI (Open Container Initiative) that Apcera joined. It is not about scheduling smarts (BORG was not very smart, it just matched some constraints), but it did the job since 2003 and it was still a preferred tool inside Google.

How Apcera Started

Miha: When did you start Apcera?

Derek:  In 2012, VMware open sourced the project, delivering a technology that did what it said it did. From the interaction with the customers, I discovered that we solved the wrong problem, yet the train had left the station. It was not like Cloud Foundry 2.0 where I can do a re-write and try to solve a different set of problems. Devs and devops are all about speed and innovation and all new tools. But in my opinion, they do not see the forest for the trees. I realized this was not about speeding things up; it is how to have a system that reasons and decides those things transparently, and enforces those decisions and policies  to drive trust. How do I reason if you are allowed to deploy something? How do I reason to whom you are allowed to talk?

Derek paused for a few seconds:

Derek: Back to VMware, they created a system that in my opinion  allowed OpenStack to exist.  By that I mean not race to reduce costs as much as possible, but offering a class of  services that attract  me to move from one cluster to another. I think most of those services will be around data services, human machine interfaces, and Artificial Intelligence and Machine Learning.

I have two options: I can either (1) Hire expensive experts from IBM or Amazon, for example, to create all these services, and greatly increase the costs, or (2) create a platform that overrides the complexity to  move services  from one cloud to another

Bottom line, you can have a platform that over-rides all this costly and complicated stuff. For example, we support containers from Docker, directly from Docker, there is no code from us, that allows transparency.  Once you are running on that platform you have a true, trusted app mobility and scalability. in Vancouver at the OpenStack summit 2014 we were showing how we change the policy to run on Google, then move to Amazon, and then back to OpenStack, all in about a minute. Before it took months. Docker solved a lot of the porting stuff. Apcera can help you move the workloads from one cloud to another and will enforce trust: where things are allowed to run, what it is allowed to use and to access, and who is allowed to talk to the workload, etc.

Apcera's new office inside

What Apcera Does

Miha: Who sees Apcera? Devs? Devops? Users?

Derek: Great question. If devs and devops see us, we would have failed, because we would have had exposed the complexities of managing policies and operations and slow down the development and deployment.  We enable the IT Operations to become the heroes, who almost on a daily basis put a platform together, so dev and devops can go ahead with no worries.

Miha: Some engineers don’t believe in devops. What is your opinion?

Derek:  We believe in Apcera that we are taking on some really nasty and hard problems, to make it transparent and enable IT ops to say, “yes, you have an interface to our system and you understand policies, watch the change in policies.”  In an enterprise, the devs and devops are just a target for the Docker containers, of for the Kubernetes ecosystem… but it is  Apcera who designs and enforces all the rules of the platform.

Miha: Kubernetes  is now part of OpenStack, Apcera wouldn’t have any problems administering their workloads?

Derek: Yes. The devops have no direct access to the Apcera platform, but they benefit from it. Devops are very smart and hard to find. But the issue is not smartness, it’s awareness. The guy who runs IT Operations, he reports to a chain of command to privacy gatekeepers, who are aware.  You don’t have to encrypt the database. We just make a rule that says, any data in the database has to be encrypted.

There are two way of doing this. One is to go to every dev and devops and say, please recompile your apps to implement this and that, or you can a have a platform technology to enforce the policy. That is a very hard problem technically to do, but if you can pull it off, there is this #1 consumers electronics company who says: “Guys can you do this for me?”  and we can.

 There are some immediate problems, while not very sexy, they are very important. For example, a customer said to us, “I have this simple problem:  we had this task of porting 4,800 applications in Java 1.6 to port  to Java 1.7 which had an implementations plan for 18 months, using three groups of about 200 people. The customer was not sure how they would verify whether they run on Java 1.6 or Java 1.7 with confidence after the 18 months.  Can your platform tell me who is running 1.6?” I said yes. He says, “how can you help this process?” And we answered, “if the platform enables you to cut down the implementation time from 18 months to 2 months to have all this stuff done. Then you can trust the platform that the data goes in the right place, etc.”

Apcera's team "Nice people to do business with"

Some thoughts

Peter Linder, a Networked Society evangelist at Ericsson, says Distributed data centers are  the new network end-point. "These distributed data centers will be a mission-critical infrastructure for Networked Society."

Derek seems ahead of his time, in the right direction, because Apcera, as it is now, might be just the first stop in his journey.


I don’t say anything online that I wouldn’t say in person.  What I say are exclusively my thoughts, views, opinions or understanding of a topic or issue, and not my employers'. I can be wrong even though I try hard not to be. I will admit to mistakes, correct them promptly and even apologize where it is appropriate.

Sunday, January 17, 2016

John Sculley and Steve Jobs. Some random thoughts

In the Steve Jobs chapter of Apple Computer mythology, the name of the villain is John Sculley, ex-president of Pepsi-Cola. He was hired by Steve Jobs himself, In 1983  he became the CEO and stayed for 10 years as the highest paid CEO on Silicon Valley. He is the guy who pushed Steve aside.

He then tried to run Apple Computer as a commodity corporation. One lesson the world learned is that one can not replace a resourceful, genial  founder with an equally resourceful bureaucrat.

The need to hire non-conformists and underdogs

This is a  blog I wrote in April 2014, in essence long quote from David Brooks, the New York Time columnist and thinker.
Bias hiring decisions against perfectionists. If you work in a white-collar sector that attracts highly educated job applicants, you’ve probably been flooded with résumés from people who are not so much human beings as perfect avatars of success. They got 3.8 grade-point averages in high school and college. They served in the cliché leadership positions on campus. They got all the perfect consultant/investment bank internships. During off-hours they distributed bed nets in Zambia and dug wells in Peru.
When you read these résumés, you have two thoughts. First, this applicant is awesome. Second, there’s something completely flavorless here. This person has followed the cookie-cutter formula for what it means to be successful and you actually have no clue what the person is really like except for a high talent for social conformity. Either they have no desire to chart out an original life course or lack the courage to do so. Shy away from such people.

Who won?

Steve was born in 1955 and died in 2011 at the age of 56. He was 16 years younger than Sculley. who was born in 1939 and is turning 77 years old this year

Steve was an university drop-out. John was a graduate from the top Ivy League East Coast schools: Brown University  in Rhode Island, whose graduates  are John D. Rockefeller Jr., John F. Kennedy Jr., Ted Turner and the like, plus he is a  Wharton MBA graduate.

Steve Jobs married only once

Marriage with Laurene Powell in a Buddhist ceremony in Yosemite in 1991  

 John Sculley had three wives. His latest marriage at the age of 74 took place in 2013

Since the beginning of his career, the marriages helped his career
He married Ruth Sculley, stepdaughter of PepsiCo president Donald Kendall in 1960, which ended in divorce in 1965... Then in 1978, he then married Carol Lee Adams, ex-wife of a former PepsiCo vice president, ultimately divorcing in 2011.
Ruth Sculley is his 3rd wife and she is featured on Mr. Sculley twitter home page.

Steve Jobs' family on the father side is from Syria. Now, in this political climate of 2016, not accepting immigrants from Syria could have meant not having an American Apple Computer.

Steve Jobs rests in a non-denominational cemetery in Palo Alto

Is life just?

Those who only judge Steve Jobs as legendary, almost supernatural hero. have a distorted view of this world. John Sculley is completely different from the mythology, And so is Steve Jobs. Both men are exceptional and both made mistakes. One of them proved to be a Messiah. After all, Steve invited John to join Apple, in the same way as he saw the iTunes  later on. Both were good ideas, at one time,

John Sculley works with two other brothers investing in startups. Having worked at Apple is his greatest asset. He wrote new book and he  is available as a speaker, consultant for some hefty fees. The former "bad" guy" is now "a good guy".

We never know whether we are going to be winners or losers. We simply do what we do best, but it is not for us who decide.  This is one of my answers at what it makes us happy every day

Read Miha Ahronovitz's answer to What are 5 daily things to do to feel happy each day? on Quora

Monday, January 11, 2016

A Trusted Cloud Platform

Why this video is on Apcera home page?

The first message a visitor reads on Apcera’s home page is:
Innovation, Meet Trust: Transform your business with the most secure and trusted cloud platform
This is not just a marketing claim. The founder of Apcera, Derek Collison, has a rich experience from working over many years at Tibco and Google, then initiating the Cloud Foundry project. When he worked at Google, Derek and his team used an internal tool called Borg. No Google developer had direct access to hardware, only to Borg. Derek discovered that Google trusted Borg to protect their core business, while allowing devs to move fast.

This is why

Apcera is the ultimate platform that Derek created to build on this concept. He placed trust first in the design priorities. In this video from Apcera, - that Apcera just placed on their web site - Jason Hoffman, Head of Product Area Cloud Systems at Ericsson, talks about trust.
In Ericsson, we are responsible for mission-critical infrastructures globally. Those range from your ability to make a 911 call to just being able to connect to your kids when you are on the road and tell them you miss them. When our infrastructure doesn’t work, presidents and prime ministers call our CEO, upset.

What is trust? 

Any company feels in a way  honored that the highest dignitaries are upset when their products are facing a temporary service interruption. A Hasidic teaching says: “Provide love; trust will be born from it. Demonstrate your trust, and it will awaken love.” In business, customer love leads to loyalty. In 140 years, Ericsson has a fantastic reputation and sense of responsibility towards its loyal customers.
Trust also means dealing with ethical and professional people
When you are looking at what you are doing professionally or in your personal life, it does boil down to whether you can trust the system you’re working with.
What is a professional? A professional is someone who does not simply know about something. A professional does something with what he knows. Professionals are not judged by what they know, but by what they produce as a result of that knowledge.

We value both the technology and the people

When we start thinking of the scenarios where people start using multiple clouds, or multiple technologies, when we start thinking of that type of machinery, that’s exactly why we were so attracted to the Apcera platform, because this is a layer can be rolled out across the top of everything. We have to deliver trusted infrastructures, and the basis of that trust from us is the Apcera platform.

The blockchain technology as a measure of trust

There is a technology that hit the headlines last year, but it is not new, called blockchain. According to Economist magazine cover story:
 In essence it is a shared, trusted, public ledger that everyone can inspect, but which no single user controls. The participants in a blockchain system collectively keep the ledger up to date: it can be amended only according to strict rules and by general agreement. Bitcoin’s blockchain ledger prevents double-spending and keeps track of transactions continuously. It is what makes possible a currency without a central bank.
Apcera trust is the result of steady automated policies enforcement.

 But the blockchain technology can measure the trust. Reading   Mike Gault - the  CEO of Guardtime - article in Wired magazine:

A keyless signature provides an alternative method to key-based technologies, and delivers proof and non-repudiation of electronic data using only hash functions for verification. By using hash functions, the technology can prove the time, authenticity, and origin (machine, organization, individual) of the input data. 
In addition, keyless signature technology provides mass-scale, non-expiring data validation while eliminating the need for secrets or other forms of trust. Thus, it eliminates the need for complex certificate-based solutions which carry certificate management issues, including expiration and revocation.
In practical terms:
This technology helps organizations to validate, verify and self-authenticate their big data. The keyless nature of the technology reduces the security and administrative footprint because it removes the need for cipher keys and passwords, which can be lost or mislaid. For data authentication to be align squarely with data authenticity, taking signatures keyless is key in our new brave big data world.

Some interesting tweets 

Friday, January 01, 2016

Why Docker containers are not used widely in Enterprises?

2015 was the year of the Docker explosion

I am a great admirer of Solomon Hykes, whose company dotCloud seed financed by Trinity Ventures, went south and was on the verge being sold for peanuts. Then Solomon and close collaborators did something crazy: they decided to open source the container technology they had built for dotCloud.

His investors were against it, yet Dan Scholnick , an open minded VC investor from Trinity wrote:
Thanks to Solomon and his team, we have a business called Docker that’s on fire and changing the world of software infrastructure....
 Solomon Hykes is an exceptional entrepreneur.  None of the success Docker is having today would have been possible without his insight and courage, to say nothing of the technical accomplishments.  He knew when to cut bait and when to go with his gut.  No combination of good idea and great investors can accomplish anything without exceptional, visionary founders at the helm.  And of course, Solomon couldn’t have done it alone.
You can make a Docker container after a 30 minutes tutorial with a Mac, Windows or Linux machine.

Yet in January 2015, lots and lots of people talked about Docker, some few people did some proof of concept, almost no one did production environment docker processing

Analyst ravings

Here is a sample from Forrester predictions for 2016 as described in this blog dated December 29, 2015.
Containers are all the rage!
Over the past year Containers such as Docker have generated tremendous interest and uptake among well-known cloud providers, who use them to deliver some of the largest and most popular cloud services and applications. Container adoption is being driven by the promise that containers deliver the ability to “build once and run anywhere", allowing increased server efficiency and scalability for technology managers.
Perhaps we can say this at the end of 2016,  But we are a few miles away from this vision to be reality. Please read this blog further and you decide what is the reality in December 2015

 Security Concerns

One of the best recent articles on this issue is What can go wrong when security is ignored during development? by  Filippos Raditsas  Some things he says:
  • New technologies expand the attack surface of a system. 
  • Attackers look at the new features and technologies as potential areas of weakness.
  • A false belief that the utilization of certain technologies are more secure out-of-the-box solutions. This does not necessarily mean that a system will still be secure after a new technology is integrated.
  • Early phases of the development life-cycle, such as analysis or design, where the requirements are gathered and the architectural components and technologies are chosen, may introduce security flaws.
For these reasons, the most effective solution is:
..exposing developers to security from an attacker's point of view. More specifically, instead of just training them about secure coding practices, they should be guided through the discovery and exploitation of software vulnerabilities, as if they were the attackers.

The Rugged Manifesto

This group,  exists since 2012 and helps bridge the gap between the developers' community and the security community so that we can interact and collaborate more efficiently towards the common goal of reliable and secure applications.

Why Docker is still not widely used in Enterprises?

Here are a few quotes from December 4, 2015 blog of  Julian Dunn a deep thinker, cool writer and seasoned coder.
This post is simply about the horrifying realization that containerization opens up a whole new playing field for folks to abuse. Many technology professionals in the coming decades will bear the brunt of the mistakes people are making today in their use of containers. Worse, these mistakes will be even more long-lived, because containers — being portable artifacts independent of the runtime — can conceivably survive in the wild far longer than, say, a web application written using JSP, Struts 1.1 and running under Tomcat 3.
Here are the reasons:

Using containers without really understanding why

"The reason that “Docker Docker Docker” is such a meme is because as with any new technology, there are large swaths of unqualified technology “professionals” ready and eager to rub it all over everything. There’s no shortage of developers in startups insisting that Docker is the “future” and that it must be used for everything.

 Blindly containerizing legacy applications 

".. jamming legacy applications into containers. This makes no sense. Containers are designed for per-process isolation, ergo, one process per container. Containers aren’t going to be particularly useful for running Oracle 12c or SAP HANA that have a million processes. You could containerize them, but why? If a service is not something you’ll be ripping and replacing frequently, there’s really no point.
 I also see container technologies being used to keep legacy  applications alive forever, which is even more terrifying. We can therefore expect to still be running today’s applications, inside containers, in 2050 or beyond."

 Ignoring devops principles

In a containerized world, however, it gets way worse for teams that haven’t adopted devops practices. Now, these application teams aren’t just throwing Java bytecode over the wall; they’re throwing entire machine images. Think of all the things that can go wrong on a Linux system and how much maintenance needs to be done to keep it running smoothly, like security patching, rebooting, performance tuning, and so on. Now apply all that work to a microservices world, where you could have tens or hundreds of thousands of containers — which are really full-blown Linux systems — running in production. Keeping the lights on will become a nightmare without good cooperation and shared responsibility between dev and ops.. This will come down crashing down on both groups when production failures occur.

 Lack of standard quality assurance principles

How are enterprises to know that the image functions correctly? Or is compliant, or secure? Or that it will continue to be that way even if changes are made? Until container adopters realize that everything they learned about ensuring safety in the delivery of software products doesn’t get thrown away in a container universe, we can expect a lot of buggy, insecure applications being deployed to production or pushed to the Docker Hub where, like a virus, they will get used by others.

 Giant container images containing all of userland

By default, most containers provide a full Linux system, with all of /bin, /usr, and so on. Too few people build base images, or start container builds from scratch (literally FROM scratch in Docker). 
I believe that the “large attack surface” is a fundamental design problem with containers being an evolutionary, not a revolutionary step from VMs and bare metal. Container technology has been so successful purely in its ability to efficiently de-duplicate and package an entire Linux userland in a portable, run-anywhere image. But the application process still believes it is running inside a full multi-user Linux OS


 "Technologies last a very long time. Mikey Dickerson of the U.S. Digital Service spoke recently about how Medicare still primarily runs on 7000 COBOL jobs written piecemeal over the last 30 years. Nobody has a complete map of the dependencies between these jobs, or where to even start replacing them. Because patterns are so commonly copied, it’s important that for future generations, we try to minimize the bad solutions and make more good ones."
 Sadly, technology hype creates a reality distortion field that stops people from clearly assessing whether their use of that technology is a) appropriate and b) architected well,

Apcera Docker Solution

Josh Ellithorpe from Apcera presented this demo on DockerConEU. What is Apcera? The best intuitive description  is in this short blog 10X Savings

What is Apcera Docker solution? IMHO it addresses many (if not all) the concerns of Julian Dunn. Apcera proposes a process implemented as part of their platform capabilities, Because containers vulnerabilities are a combination of human error, intuition and judgement on what is best for a container and what is not.

I use screen shots from Josh's video to illustrate:

Enterprises moving to production are looking to extend Docker security and policy control using three bullets from the picture

In this 2nd slide, Apcera recommends ad-ons for Docker answering specific questions that Docker so far does only at registry level.

If my name is Mr. Container, then is natural for me to ask "What route am I allowed to take?" "How many resources I can use?" "Where am I allowed to land and deploy?", "What software package shall I use - and which ones  I should never never touch?" "How someone always knows what I am doing, who can help and correct my mistakes before it is too late?"

The first symptom of enterprise suspicions is asking them to use other tools for authentication, different from what they use now. So Apcera has a new command "apc docker run" to replace "docker run". It uses all the authorization tools Enterprises are familiar and trust them


This last slide lists a sample of essential policies Apcera can provide for native workloads pertinent to Docker

The work that Apcera has done in collaboration with FlawCheck to test Docker images for known vulnerabilities before deploying them in production is described in this webinar

After reading Julian Dunn blog, I can see how Apcera  can minimize the Enterprise fear of doom and instill trust when deploying Docker or any other new technology.

If they are more companies like Apcera, then Forrester prediction (see above)  will become an instant reality.

I am familiar with Apcera, but if you know of similar companies, I will be glad to hear from you. I know Docker acquired a few companies, like Tutum, and I interviewed Borja Burgos its' former CEO in February 2014 (see Tutum is set to dockerize the Enterprise. ). However Docker has so many directions to focus that the elegant Apcera solution is still my #1 preference.

Blog Archive

About Me

My photo

AI and ML for Conversational Economy