2014-09-17

An Open Letter to the OpenStack Foundation

I have recently started to regularly follow the mailing lists and the conversations are quite interesting.

It is quite evident that that OpenStack is starting to go through growing pains. It was quite evident as well from OpenStack Silicon Valley 2014 that was held yesterday.

OpenStack has grown from a minimal amount – where most of the developers knew each other personally, knew each others phone numbers, a good personal community – the way it should be. But as all good things, like all successful thing, it grows. This is what OpenStack looks like today.

Current OpenStack programs are listed below:

New capabilities under development for Juno and beyond:

These are the core components. I do not think that I am exaggerating if I was to say that there are at least another 30 projects or Github repositories either on the OpenStack repo or the Stackforge repo, and a good portion of them are actively being developed.

Ideally each and every project should be independent from the other. This of course has its upside but also problems as well.

Let’s take an example.

Most of the core components use a central database, typically MySQL. Keystone, Neutron, Glance and so on. You could install a database server for each and every component – but that would probably be major overkill, so usually what will happen is you create different tables on the same database server for each component. This now becomes a shared dependency for all the components now using that Database server. The larger the number of components use it, the more things that need to be considered. It has to be made highly available, the MySQL queries of a single component should not have any major effect on any other component, and so on and so forth.

Dependecy Hell? Spaghetti

So you would think that if most components have decided on using the same database then we should do the same for all of the components? That would be a logical assumption. No need to over complicate things.

Along comes Ceilometer. I will not go into the reasons why, but it uses MongoDB, not MySQL. That just created a complication. I now need to know how to manage another database, back it up, replicate it, and make it highly available. In addition to MySQL.

I have seen this many times, my organization is as guilty as any other for doing things like this. A development group starts working on a product and look for what is the easiest way for them to deliver their solution. Sometimes a solution is chosen because it is a “hot technology” and other times, because the standard “just doesn’t cut it”.

(As a side note – a whole other discussion could be held here, is adhering to standards the right way – even if it is more difficult or takes longer, or get the job done in the best and fastest way possible – ideally it should be a balance of the two – IMHO)

Without clear direction from the product leadership, without a proper standard – things can go wrong very quickly.

That is how I came across a product (with about 30 components) that had 6 different database technologies:

  • Cassandra
  • Oracle
  • CouchDB
  • MySQL
  • MongoDB
  • Derby

That is

  1. 6 different connection strings
  2. 6 different database clients
  3. 6 different backup procedures
  4. 6 different High availability models

And so on and so on…

Is this a bad thing? Depends who you would ask. The people who developed it – they are happy – because they got the job done, product works and they did not create a dependency.

For those who have to deploy, manage, troubleshoot and support this product – THIS IS A NIGHTMARE! for obvious reasons, some of them mentioned above.

I think that the same is happening in OpenStack. The proliferation of projects, the number of developers and just the sheer size of the solution is becoming very hard to manage. Goals of one projects do not necessarily align with that of other projects, and cross project collaboration is not the “best” at the moment between the projects.

Randy Bias said yesterday in his talk at OpenStackSV (not an exact quote):

The problem  is that OpenStack does not currently have a unifying vision or product strategy. With the growth in programs, OpenStack’s more consistent mission starts to be degraded and less meaning around OpenStack occurs.

Each program team tends to have their own view of what OpenStack is. This does not mean there needs to be a dictator in OpenStack, but there does need to be product leadership.

(Source – vElemental)

Adrian Ionel also raised some thought yesterday on the subject.

The general developer does not care about things below or around the application like monitoring software, storage, network, or the hypervisor. They care about API quality and ease of use, feature velocity (not about OpenStack plumbing), and portability (devs want to write things once). Is OpenStack too intrinsic and poorly focused? Are infrastructure vendors moving focus away from critical areas?

In conclusion, he believes that there are some tangible things that can be done.
– Focus on API (awesome, well documented, easy to use, consistent, backwards compatible)
– Invest in ease of use vs flexbile plumbing (could have too many options)
– Don’t move up stack partner instead (LB, DBaaS, etc)
– Reshape upstream engineering to foster open competition inside OpenStack projects (engineering come up with solutions and compete while staying in framework and let market forces choose) vs central planning

(Source – vElemental)

I am not sure I agree with everything that Adrian said, especially not the part about competition. Yes this does produce better products in the end – due to the competition, but it also generates a lot of work (and lines of code) that will probably go down the drain in the end. I do not think that the OpenStack Community is currently in the position to allow themselves that luxury. Code reviews and code fixes are not being completed in the fastest time possible. Added to that extra code in two competing projects for the same purpose will only emphasize this point.

One last thing. The OpenStack WTE (Win the Enterprise) initiative is something that should have been kicked off 2 years ago.

No matter how good the product is – if people cannot use it or it does not provide the functionality that people need, it will not be adopted.

There are things, basic things that the Enterprise needs today, and have been asking for a number of years and can get them today in competing products.

I am not saying that OpenStack has to cater for every single kind of workload, for every application, for every single scenario - not at all. But the vision, the direction has to be there. Lay down the basic foundation of what OpenStack is going after, when you think it will happen, and more importantly what will not be supported or worked on in the upcoming future.

The feeling that I have (and I am not the only one) is that each project is doing what is best for them – and not necessarily what is best for OpenStack.

The Technical Committee’s charter:

The OpenStack Technical Committee provides technical leadership for OpenStack as a whole. Responsibilities include enforcing OpenStack ideals (such as Openness, Transparency, Commonality, Integration and Quality), deciding on issues that impact multiple programs, providing an ultimate appeals board for technical decisions and general oversight. It is a fully-elected Committee that represents the contributors to the project, and more details about membership and programs may be found in its charter.

I think at the upcoming Summit this is going to be a hot discussion topic. The important thing is that it should be an open discussion, and I think taking into consideration not only the development part of the community – but also the User and operators as well.

As always I would be happy to help out in any way I can.

Labels: , , ,