I have been working on a project for a while that includes the deployment of a large number of moving parts that are in a significant state of flux. Drops every two weeks, new features added all the time, and, of course, with a system this size there is a great amount of complexity involved. Complexity in the Continuous Integration stage, complexity with the end-to-end testing, and, definitely, complexity with the Continuous Deployment.

A good part of the intricacies comes from the fact the development team wants to assure that the deployment will be cloud-agnostic. But before I go into if this would be a good or a bad thing, let me first explain what this means, and offer a few examples.

It is no secret that almost no OpenStack cloud is identical to another. The network setup could be different (provider networks vs. private networks). Some clouds have Swift installed by default, while others do not. There are nuances and differences between an on-premises OpenStack deployment and using an OpenStack cloud provider, such as RackSpace. APIs are different, versions are different. This makes things very difficult for people writing software to interact with the cloud to address a fully cloud agnostic solution. APIs, authentication mechanisms, and the way you can access resources will change from one cloud to another.

Read the rest of the article here

Two days ago I decided to submit my candidacy for one of the two spots up for election (for the first time!) on the OpenStack User committee.

I am pasting my proposal verbatim (original email link here)…

Good evening to you all.

As others have so kindly stepped up - I would also like to self-nominate myself for as candidate for the User committee.

I have been involved in the OpenStack community since the Icehouse release.

From day 1,  I felt that the user community was not completely accepted as a part of the OpenStack community and that there was a clear and broad disconnect between the two parts of OpenStack.

Instead of going all the way back - and stepping through time to explain who I am and what I have done - I have chosen a few significant points along the way - of where I think I made an impact - sometimes small - but also sometimes a lot bigger.

• The OpenStack Architecture Design Guide [1]. This was my first Opensource project and it was an honor to participate and help the community to produce such a valuable resource.
• Running for the TC for the first time [2]. I was not elected.
• Running for the TC for the second time [3]. Again I was not elected.
  
  (There has never been a member of the User community elected to a TC seat - AFAIK)

In my original candidacy [2] proposal - I mentioned the inclusion of others.

Which is why I so proud of the achievement of the definition of the AUC from the last cycle and the workgroup [3] that Shamail Tahir and I co-chaired
(Needless to say that a **huge** amount of the credit goes also to all the other members of the WG that were involved!!) in making this happen.

Over the years I think I have tried to make difference (perhaps not always in the right way) - maybe the developer community was not ready for such a drastic change - and I still think that they are not.

Now is a time for change.

I think that the User Committee and these upcoming election (which are the first ever) are a critical time for all of us that are part of the OpenStack community - who contribute in numerous ways - **but do not contribute code**.

The User Committee is now becoming what it should have been from the start, an equal participant in the 3 pillars of OpenStack.

I would like to be a part, actually I would be honored to be a part, of ensuring that this comes to fruition and would like to request your vote for the User Committee.

Now down to the nitty gritty. If elected I would like to focus on the following (but not only):

1. Establishing the User committee as significant part of OpenStack - and continue the amazing collaboration that has been forged over the past two years. The tangible feedback to the OpenStack community provided by the Working Groups have defined clear requirements coming from the trenches and need to be addressed throughout the community as a whole.
   
2. Expand the AUC constituency - both by adding additional criteria and by encouraging more participation in the community according to the initial defined criteria.
   
3. Establish a clear and fruitful working relationship with Technical committee - enabling the whole of OpenStack to continue to evolve, produce features and functionality that is not only cutting edge but also fundamental and crucial to anyone and everyone using OpenStack today.

Last but not least - I would like to point you to a blog post I wrote almost a year ago [5].

My views have not changed. OpenStack is evolving and needs participation not only from the developer community (which by the way is facing more than enough of its own challenges) but also from us who use, and operate OpenStack.

For me - we are already in a better place - and things will only get better - regardless of who leads the User committee.

Thank you for your consideration - and I would like to wish the best of luck to all the other candidates.

--
Best Regards,
Maish Saidel-Keesing

[1] http://technodrone.blogspot.com/2014/08/the-openstack-architecture-design-guide.html

[2] http://lists.openstack.org/pipermail/openstack-dev/2015-April/062372.html

[3] http://lists.openstack.org/pipermail/openstack-dev/2015-September/075773.html

[4] https://wiki.openstack.org/wiki/AUCRecognition

[5] http://technodrone.blogspot.com/2016/03/we-are-all-openstack-are-we-really.html

Elections open up on February 13th and only those who have been recognized as AUC (Active User Contributors) are eligible to vote.

Don’t forget to vote!

I am on my way to the summit – cutting it fine, as I will only be arriving after the keynotes have started on Day 1. That is part of my life being a religious orthodox Jew.

Just a few hours ago, I finished the festival of Sukkot, a festival where we ‘leave’ our homes for 8 days and move to a temporary house. Well not really leave the house – but we eat all our meals in the Sukkah for the whole festival.

It symbolizes the fact that we have faith in G-d and remember that life is was not so easy and that we left slavery in Egypt many years ago to become a free nation.

Going back to the title of the post.

The main reason I am going to this summit is because I am co-presenting a session on the work
Shamail Tahir and myself have been leading with the Active User Contributor (AUC) working group over the past few months.

If you are interested in more details about the work – I will not rehash the post that has already been published - Recognizing active user contributors to OpenStack.

Please take a minute or three to read it over.

Great!! You are back.

I think that we are at a time where OpenStack is about to change, and for the better. Traditionally Operators and Users have been neglected left out of many of the decisions made in the projects, something which I have vocally opposed many a time (you can find most of my posts here) especially this one - We Are All OpenStack! Are We Really????.

A number of things have happened recently – and will continue to evolve over the next few months – which will enable Users and Operators to actually have a voice (at least that is my true hope and belief).

Proper representation, real recognition, and hopefully some more influence in directions and perhaps also priorities.

If you are going to be at the summit, make sure you join Shamail and myself presenting the work that was done over the past few months.

I hope that one day we will be able to look back on the past and reflect on a time once past, and understand that we are all now one community.

Looking forward to see you there!

Background

OpenStack has a vast community, globally distributed, spanning multiple time zones and all sorts and kinds. The community has a culture - a charter, a way of doing things. It is something that you have to learn how to get used to - because the lay of the land is not always as straight forward as you would expect, not what you are accustomed to, and sometimes it might see downright weird. But as the saying goes, "When in Rome, do as the Romans".

A thread on the OpenStack mailing list as of late week (actually two of them) caught my eye, and attention - so much that I feel the need to write this post.

Ever since I have started to get involved with OpenStack, I have noticed that there is a very obvious divide between the Developers (those who write and contribute code to OpenStack) and the User community (this is split into two parts - those who actually consume OpenStack - the end users, and those who deploy and maintain OpenStack - the operators). But before the thread - let me explain current situation in OpenStack for those who might not be acquainted with the way things work.

Anyone who wants can sign up as an OpenStack foundation member. All you need to do is to sign up on the site, fill in a form, accept an agreement - and you are a full fledged OpenStack foundation member.

The 'benefits' that come with this are not many - and the obligations are not that demanding either.

2.2 Individual Members.
(a) Individual Members must be natural persons. Individual Members may be any natural person who has an interest in the purpose of the Foundation and may be employed by Platinum Members or Gold Members.
(b) The application, admission, withdrawal and termination of persons as Individual Members are set forth in the membership policy attached as Appendix 1 (“Individual Member Policy”).

As an foundation member - you can run as a candidate for the Individual Member Director elections and of course you can vote in the above elections. You are invited to participate in the User surveys that are published on a regular basis. And of course you consider yourself a member of the OpenStack community. You are also eligible to run as a candidate for the Technical Committee elections (but you are not allowed to vote – see below).

Next come the Active Technical Contributors (ATC). These are the people that actually write the code (well at least that was the intention - I will explain later). The criteria for becoming an ATC are as follows:

(i) An Individual Member is an ATC who has had a contribution approved for inclusion in any of the official OpenStack projects during one of the two prior release cycles of the Core OpenStack Project (“Approved Contribution”). Such Individual Member shall remain an ATC for three hundred and sixty five days after the date of acceptance of such Approved Contribution.
(ii) An Individual Member who has made only other technical contributions to the OpenStack Core Project (such as bug triagers and technical documentation writers) can apply to the chair of the Technical Committee to become an ATC. The final approval of such application shall be approved by a vote of the Technical Committee. The term shall be for three hundred and sixty five days after the date of approval of the application by the Technical Committee.

This comes with obligations and benefits as well. The obligations are that you are expected to contribute code according to the guidelines and rules of the projects, code quality must be upheld, and etc. etc.

The benefits of being an ATC are as follows.

• You get a free pass to the OpenStack summit ($600 value).
• You are eligible to vote in the bi-annual Technical Committee Elections.
• You are also eligible to vote for the Project Team Lead Elections - which occur every six months - providing you contributed to that specific project over the last cycle.
• You are considered an integral member of the OpenStack community. Up until last summit - you were also granted access to the OpenStack Design summit - something that was closed for only ATC's (but this has changed since last summit - or two summits ago - I cannot remember).

There are two other groups who are partially recognized by the OpenStack community. There are people that are contributors to the community. They submit bugs, review patches, but do not actually submit code to the projects. I assume that the idea is that once they start getting into the code - then they actually will contribute code back into the projects in the long run and will become ATC's.

And then there are the end-users, and then the Operators. They are the people who are trying to deploy, upgrade and maintain what we all call OpenStack.

• They are the ones who are trying to get things working in OpenStack.
• They are the ones who have to provide creative ways to supply solutions to their clients - because OpenStack does not do what it was supposed to - does not do it well enough, does it really badly - or has not got there yet.
  Two clear examples (for me) would be - bad telemetry for all instances (which is now being re-written) or Load balancing that is not enterprise ready.
• They participate in an Ops meetups (see the outcome from the last one in Manchester a month ago)
• They contribute code - to the Ops repositories - which include scripts, tools, and much more.
  They participate in working groups - such as Enterprise Working group, Operator Tags, Product marketing - there are many. The technical (a.k.a. Developer) community see these participants as part of the user community. They do not see these contributions as part of the OpenStack product, they are deemed Upstream, or Downstream - but not part of the OpenStack products.

 

The Problem

The thread I am talking about - was titled "Recognizing Ops contributions". It has been voiced many a time (and not only by me - even though I have been very vocal about it) that Operators should be recognized as part of OpenStack. I have actually run for the Technical Committee elections on the basis of this stand - but I was not at all successful.

OpenStack works on the 4 Opens (and this is an official OpenStack resolution!)

I would like to bring your attention to the following points in that resolution

The community controls the design process. You can help make this software meet your needs.

The technical governance of the project is a community meritocracy with contributors electing technical leads and members of the Technical Committee.

The obvious question though is - what is considered a contributor?

If the four opens are to be upheld - then I would expect that the all contributors are equal.

Today that is not the case.

There have been many a time where I personally and many other operators as well have asked for things to change in OpenStack, have asked to be included in the community, have asked to provide feedback, and have been constantly told that everything should be done the way that developers have been doing it. Sometimes with success, sometimes with less, but I think that OpenStack has started on the right track - but still there is a whole lot of resistance to allow Operators a full and equal say in what happens in the development process.

When I see comments like this (from Thierry Carrez - a long time TC member), I come across what has been happening for many years already.

Yeah, we can't really overload ATC because this is defined and used in governance. I'd rather call all of us "contributors".
<..snip..>
Upstream contributors are represented by the Technical Committee and vote for it. Downstream contributors are represented by the User Committee and (imho) should vote for it.

Yes Thierry continues to say that all of the contributors should be equal - but keep the two separate - only people who contribute Upstream (which I understand as those who write code) should be part of the TC, and all the others - part of the User Committee.

What is this User Committee? Looking on the OpenStack site there is a clear definition of its mission.

As the number of production OpenStack deployments increase and more ecosystem partners add support for OpenStack clouds, it becomes increasingly important that the communities building services around OpenStack guide and influence the product evolution. The OpenStack User Committee mission is to:

• Consolidate user requirements and present these to the management board and technical committee.
• Provide guidance for the development teams where user feedback is requested.
• Track OpenStack deployments and usage, helping to share user stories and experiences.
• Work with the user groups worldwide to keep the OpenStack community vibrant and informed.

To me it seems that the user committee is a mantlepiece body which has absolutely no influence on what happens in OpenStack and was formed to say, "Hey we have a group of people that represent our users. They can't really do anything - but at least we can say they have representation."

And how do I know that the User committee - has no teeth? I looked at the bylaws (4.14).

4.14 User Committee. The User Committee shall be an advisory committee to the Board of Directors and shall be comprised of at least three (3) Individual Members, one (1) appointed by the Technical Committee, one (1) appointed by the Board of Directors, and one (1) appointed by the appointees of the Technical Committee and Board of Directors. The User Committee shall organize its meetings and may, on approval of the Board of Directors, create elected seats to be filled by a vote of the Individual Members. On request of the User Committee, the Board of Directors shall invite the User Committee to attend each regular quarterly meeting and shall allocate at least one (1) hour during the regular quarterly meeting following the annual election to hear the report and recommendations of the User Committee. On request of the User Committee, the Technical Committee shall invite the User Committee to attend a regular meeting and shall allocate at least one (1) hour during such meeting up to four times each calendar year to hear the report and recommendations of the User Committee.

The UC (User Committee) has to request to be invited to attend the BOD (Board of Directors) meeting. The UC has to request to attend the TC (Technical Committee) regular meetings and will be given at maximum 4 hours a year to give recommendations and report.

There is not a single word about what the UC actually can do besides participate in meeting and pass recommendations. Do these recommendations have to be accepted? Can they change anything? That is at the discretion of the TC and the Board.

What does the TC do - or what can they do? First the TC Charter:

The Technical Committee (“TC”) is tasked with providing the technical leadership for OpenStack as a whole (all official projects, as defined below). It enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality...), decides on issues affecting multiple projects, forms an ultimate appeals board for technical decisions, and generally has technical oversight over all of OpenStack.

Again back to the bylaws (4.13)

4.13 Technical Committee.
(a) The Technical Committee shall be selected as provided in the Technical Committee Member Policy in Appendix 4.
(b) (i) The Technical Committee shall have the authority to manage the OpenStack Project, including the authority to determine the scope of the OpenStack Technical Committee Approved Release subject to the procedures set forth below. No changes to the OpenStack Technical Committee Approved Release which deletes all or part of the then current Trademark Designated OpenStack Software. shall be approved by the Technical Committee without approval as provided in the Coordination Procedures. After such approval, the Secretary shall post such description to the Foundation’s website.

Do you notice the difference in tone? This is the committee that decides what happens within the OpenStack projects, who is added, who is not, who is removed, what directions are taken, etc. etc.

Sorry for jumping around, but back to the thread.

Right, this brings up the other important point I meant to make. The purpose of the "ATC" designation is to figure out who gets to vote for the Technical Committee, as a form of self-governance. That's all, but it's very important (in my opinion, far, far, far more important than some look-at-me status on a conference badge or a hand-out on free admission to an event). Granting votes for the upstream technical governing body to people who aren't involved directly in upstream technology decisions makes little sense, or at least causes it to cease being self-governance (as much as letting all of OpenStack's software developers decide who should run the User Committee would make it no longer well represent downstream users).

Again - I read this is as - let the developers write their code and keep Operators far away from having anything to do with how OpenStack projects are managed. Because they have no business here.

The Solution?

Which led me to write this answer. I would like to re-iterate what I said on that thread.

Operator contributions to OpenStack are no less important or no less equal than that of anyone writing code or translating UI's or writing documentation.
By saying that someone who contributes to OpenStack - but doing so by not writing code are not entitled to any technical say in what directions OpenStack should pursue or how OpenStack should be governed, is IMHO a weird (to put it nicely) perception of equality.

So I see two options.

Ops Contributors are considered Active Technical Contributors - just the same as anyone writing code - or fixing a spelling mistake in documentation (and yes submitting a patch to correct a typo in a document - does give you ATC status). Their contributions are just as important to the success of the community as anyone else.
or
Give Ops contributors a different status (whatever the name may be) - and change the governance laws to allow these people with this status a voting right in the Technical committee. They have as much right as any other contributor to cast their opinion on how the TC should govern and what direction it should choose.

By alienating Operators (and yes it is harsh word - but that is the feeling that many Operators - me included - have at the moment) from having a say in - how OpenStack should run, what release cycles should be - what the other side of the fence is experiencing each and every day due to problems in OpenStack's past and possible potential trouble with the future - reminds me of a time in the not so far back history where not all men/women were equal.

Where some were allowed to vote, and others not - they were told that others could decide for them - because those others knew what was best.

OpenStack's 12th release - Mitaka - is coming up very soon. That means that OpenStack will soon be 6 years old.
I think it is time for OpenStack - including the developer community - to accept that they are no longer alone in this - there is a whole lot of information, knowledge and experience that can be reaped from all those who are actually using the products that the community produces and it is now time to accepts all contributors - in whatever way they may choose to do so - as equal members - in every way.

Yes there may be disadvantages, there also could be many advantages as well. But it time to stop treating everyone that does not write code as a "second degree citizen". Because at the moment OpenStack Technical community flaunt that everyone is part of OpenStack - but de-facto - they are most certainly not.

There will be a new working group formed (Non-ATC Recognition Working Group) in the very near future. I plan to be a very active member of this group - with my personal end goal of finally getting all contributors to the OpenStack community - and not only those who actually contribute code - as equals - with equal say - in all aspects of OpenStack - yes - including Technical leadership and decisions.

(I consider myself an Operator - my view could be biased)

A short while back I tweeted the following:

Has hell frozen over? There is actual talk about prioritizing stability over landing new features. Times are a changin' #OpenStack

— Maish Saidel-Keesing (@maishsk) November 5, 2015

This was a result of me reading the minutes of the OpenStack Technical committee from November 3rd, 2015 (full log is here)

What pleasantly surprises me is that this might finally becoming a viable option.

Let me first go into the background of my trail of thoughts.
I really enjoy working with OpenStack, but I also really hate working with OpenStack. A good old love/hate relationship.

There are amazing things about this community, the way we work, and the software we produce (just to name two).

On the other hand there are really bad things about this community, namely – the way we work, and the software we produce.

One might say that is an oxymoron (I am not calling anyone stupid though). But yes it is true. The OpenStack community produces amazing things – or at least that is until someone (let’s call them an Operator) tries to actually use this in a production environment, and then they find out until it is completely and totally, not ready, in any way for production use.

So this Operator tries to raise his concerns, why this is not useable, these are valid concerns and maybe these should be fixed. But unfortunately they have no way of actually getting this done, because – the feedback loop is completely broken. Operators have no way of getting that information back in. The community has no interest in opening up a new conduit into the developer community – because they are fixated on working in a specific way.

I must stress and honestly say, that this was the case up until approximately a year ago. Since then the OpenStack community has embarked on a number of initiatives that are making this a much easier process, and we are actually seeing some change (see the title of this post).

For me the question actually is – what sparked this change? Up until now I was under the impression that the OpenStack community (and I mean the developers) was of the opinion that they are driving the requirements and development goals, but it seems as of late – this could be shifting.

To me this is going to have a significant effect of what OpenStack is, and how it moves forward.

For better and for worse. Time will tell.

What do you all think? Please feel free to leave your thoughts and comments below.

I love diving into numbers – especially when it has something to do with technology conferences.

But before I do that I would to bring to your attention my two sessions that I have submitted for the upcoming summit (Shameless Plug.. )

Me Tarzan, you Jane (or Operators are not Developers)

Welcoming Operators to the OpenStack Jungle.

A year ago I set out on a journey on trying to help the OpenStack developer community understand the other (and sometimes not well understood) side of the OpenStack community, its users.

Users is a subjective term, depending on who you ask - it could mean the end user using an API or a GUI to deploy a new instance but it also includes those who operate the cloud, maintain it and sweat blood an tears to just allow the end-user to do what he wants.

There is distinct separation today between the two entities - but the good part is that they are slowly coming together.

This talk will describe how you an Operator can make a difference - be it small or large - in OpenStack.

The topics we will go over here are:

• Initiatives
• User Committee
• WTE
• ISV
• Monitoring & Logging
• Large Deployments
• ... and many more....

Operator specific activities:

• Ops Tags
• IRC
• Heaven forbid - committing code

Expect some interesting stories, some horror stories but we are all aiming for the same thing.

The pot of gold at the end of the rainbow.

OpenStack - High Availability (as a Service) - Fact? Fiction (or maybe something in between)?

Installing an OpenStack cloud used to be a complex task. We have evolved over time and made this a lot easier and more palatable for operators. Operating an OpenStack cloud on the other hand is whole different ball game..

Operators want stable systems and resilient systems, and if the infrastructure services can scale that would only be an added benefit.

But today, OpenStack as a result of its culture, and its history, is a collection of parts, pieces and solutions using multiple different technologies, and architectures.

One of the pain points is naturally high availability for the services which are provided today in a number of different ways.

This talk will propose one possible future direction with which this could be addressed.

By providing a central HA service for all OpenStack projects.

This session will describe a proof of concept for such solution showing making use of cloud friendly technologies that can could take the level of operations to whole new dimension.

 

I did this kind of an exercise for VMworld a few years ago - By the Numbers - VMworld 2013 Session Voting, and I thought it would be interesting to see what the numbers are for this OpenStack Summit.

Of course without some insight as well – the numbers would be quite boring.

There are a total of 1504 sessions that you can cast your vote upon.

That is a huge number of sessions. Perhaps too many. There is no way to go over the list in a defined amount of time. I think that most of the people are going to vote only if they are sent directly to a specific link. Which means this will be targeted votes as a result of someone asking you specifically vote for a specific session. Not really an ideal process but I guess that is the price we all have to pay, as a result of OpenStack becoming more and more popular.

Here are the number of sessions submitted for each track:
(Please Note – these are based on the pure number of submissions – and not what has been accepted. This is not an exact science – there could be a number of reasons why they are ranked like this – but these are my thoughts and ideas on the data below)

What is the most popular track? 

Operations - the one part that the OpenStack community is still struggling to get their input back into the projects. For a number of reasons. Be it inclusion, methodology, culture or mindset.

For me this is / should be (also for everyone that is involved in OpenStack as well) a bright and shiny beacon. Claiming that this is either the most important aspect or the most pressing need that people want to hear about or want to talk about might be going a bit far, but it is way, way up there.

OpenStack Summits should be about the technology, not about how keep the bits and bytes up and running, deployed and working in an efficient manner.

Next up on the list. Community and How to Contribute. Way down there in at the bottom. Is that because people already know how to do it? Because people have given up on trying?

This is something that the community as a whole should invest more in making it part of the culture and making the bar much more accessible to all.

Hands on Labs. The number of sessions proposed as labs is growing. Maybe it is time to think about a centralized solution specifically for the summit?

Neutron (a.k.a. Networking). The hardest part about a cloud is the networking part. I have said this before and will continue to preach it from the rooftops.

I took the liberty of creating a word-cloud of from all the words in all the submissions according to recurrence.

It makes you think..   :)

You have only one more day to vote – go and make your voice heard!

This was a big piece of news last week. There are some that even are suggesting (but not really) that the Google stock jumped drastically as a result of this announcement, I personally find that just a good joke (and some wishful thinking).

A snippet from the Google post:

As we look to the future of computing in the enterprise, we see two important trends emerging.

The first is a move towards the hybrid cloud.  Few enterprises can move their entire infrastructure to the public cloud. For most, hybrid deployments will be the norm and OpenStack is emerging as a standard for the on-premises component of these deployments.

For me this is a gauntlet thrown directly in the face of AWS.

Amazon has always pushed that everything should and can run in the public cloud. They have never believed in the hybrid cloud model. It was all AWS or nothing. I have never agreed with this statement, and still don’t.

There will always be things that need to run in house – for a number of possible reasons:

• security
• regulation
• the nature of the workload
• politics

AWS has tried over the years to provide adaptations/solutions and tools to ease the journey into the private cloud – such as VPC or Direct Connect to enable you to extend your datacenter to the public cloud – but still have it feel as if it was in-house. But that is not hybrid.

I think that this is a great move on Google’s part and their ‘bear-hug’ of OpenStack.

It will be interesting to see what this collaboration brings into the OpenStack world.

Even though we still do not know what the next release of OpenStack will yet be called (due to some community naming issues) – this is still an event I am very much looking forward to.

Registration is open for the event.

Early bird tickets are $600 (until August 31st, 2015)

A question was just posted to the OpenStack mailing list – and this is not the first time I have seen this request.

Can openstack conference video files be downloaded?

A while back I wrote a post about how you can download all the vBrownBag sessions from the past OpenStack summit.

Same thing applies here, with a slight syntax change.

You can use the same tool – youtube-dl (just the version has changed since that post – and therefore some of the syntax is different as well).

Download youtube-dl and make the file executable

curl https://yt-dl.org/downloads/2015.06.25/youtube-dl \
-o /usr/local/bin/youtube-dl
chmod a+rx /usr/local/bin/youtube-dl

The videos are available on the OpenStack Youtube channel.

What you are looking for is all the videos that were uploaded from the Summit, that would mean between May 18th, 2015 and May 30th, 2015.

The command to do that would be

youtube-dl -ci -f best --dateafter 20150518 \
--datebefore 20150529 https://www.youtube.com/user/OpenStackFoundation/videos

The options I have used are:

-c - Force resume of partially downloaded files. By default, youtube-dl will resume downloads if possible.
-i  - Continue on download errors, for example to skip unavailable videos in a playlist.
-f best - Download the best quality video available.
--dateafter - Start after date
--datebefore – Up until date specified

Be advised.. This will take a while – and will use up a decent amount of disk space.

Happy downloading !!

I have been home for just over a week from my trip to Vancouver for the OpenStack Kilo Summit (or Liberty Design Summit – take your pick).

It was a whirlwind of week, jam packed with sessions, conversations, meetings, presentations and community events.



There were a number of insights that I took with me and I would like to share with you in this post, and also in some upcoming posts in the future.

1. OPs is a real thing

There was a dedicated OPs track at the summit. Before we go into what actually happened there – I would like to clarify what I mean as OPs and if this is any different to the perception of how it is defined by the OpenStack community.

For me an operator is primarily the person that has to actually maintain the cloud infrastructure. This could be a number of things:

• Create packages for installing OpenStack
• Actually installing OpenStack
• Making sure that OpenStack keeps up and running
• Monitors the infrastructure
• Upgrades the infrastructure

There are also end-users, and these are the people that actually use OpenStack:

• Provision instances
• Deploy applications on top of those instances
• Create tenants/users/projects

For the OpenStack Community – sometimes these two groups are one and the same, and in my honest opinion they definitely are not – and should not be treated as such. It is taking time, but I think that the community is starting to understand that there are two distinct groups here, which have very different sets of needs, and should be catered to quite differently

There was a significant amount of discussion of how Operators can get more involved, and honestly I must say that the situation has improved – drastically – in comparison to the situation 12 months ago.

There are a number of working groups, the Win The Enterprise WG, the Product WG, the Monitoring and Logging WG, all of these have been meeting over the last year to try and hash out ways of getting more involved.

One of the interesting discussions that came out as a result (well perhaps not directly but I assume that I had something to do with it) of me running for the OpenStack TC was how does the OpenStack Community want to acknowledge those people that are not committing code but are contributing back into the community. To get some more background – I refer you to these threads on the mailing list.

• Who is allowed to vote for TC candidates
• TC Candidacy

Something that kept on coming up again and again in sessions was that the Project groups are looking for more and more feedback from the people operating and deploying OpenStack, but the process of getting that feedback is broken/not working/problematic.

I do understand that the TC (and OpenStack) would like to protect the most valued resource that OpenStack has – and that of course is the people writing the code.

But there has to be an easier way of allowing people to submit the feedback – and perhaps there is…
A way for Operators/Users to submit feature requests.

2. Vendors are involved in OpenStack – and they are here to stay

They are not there for the good of their hearts. They are there because they want to make money, and a lot of it. That is one (but not the only) reason why they contribute to open source projects.

OpenStack is no different. Each and every one of the vendors involved (and I will not name companies – because the sheer size of the list is just too long) are there to increase their market share, their revenue, their influence.

And that is a difficult dance to master. They are the ones providing resources to commit code and there are times where the agenda behind that is not purely community driven. This post – sums it up pretty well.

As OpenStack has grown he says its turned into a corporate open source project, not a community-driven one. He spent a day walking around the show-floor at the recent OpenStack Summit in Vancouver and said he didn’t find anyone talking about the original mission of the project. "Everyone’s talking about who’s making money, who’s career is advancing, how much people get paid, how many workloads are in production," McKenty says. "The mission was to do things differently."

OpenStack is not a small community project any more – where everyone knows each other by name/face/IRC handle. It has grown up, come of age.

For better or for worse. Stay tuned for more.

As always please feel free to add your thoughts and comments below.

This is a re-post of my interview with Jason Baker of opensource.com

Continuous integration and continuous delivery are changing the way software developers create and deploy software. For many developers, Jenkins is the go-to tool for making CI/CD happen. But how easy is it to integrate Jenkins with your OpenStack cloud platform?

Meet Maish Saidel-Keesing. Maish is a platform architect for Cisco in Israel focused on making OpenStack serve as a platform upon which video services can be deployed. He works to integrate a number of complementary solutions with the default out-of-the-box OpenStack project and to adapt Cisco's projects to have a viable cloud deployment model.

At OpenStack Summit in Vancouver next week, Maish is giving a talk called: The Jenkins Plugin for OpenStack: Simple and Painless CI/CD. I caught up with Maish to learn a little more about his talk, continous integration, and where OpenStack is headed.

Interview

Without giving too much away, what can attendees expect to learn from your talk?

The attendees will learn about the journey that we went through 6-12 months ago, when we looked at using OpenStack as our compute resource for the CI/CD pipeline for several of our products. I'll cover the challenges we faced, why other solutions were not suitable, and how we overcame these challenges with a Jenkins plugin that we developed for our purposes, which we are open sourcing to the community at the summit.

What affects has CI/CD had on the development of software in recent years?

I think that CI/CD has allowed software developers to provide a better product for their customers. In allowing them to continuously deploy and test their software, they can provide better code. In addition, it has brought the developers closer to the actual deployments in the field. In the past, there was a clear disconnect between the people writing the software and those who deployed and supported it at the customer.

How can a developer integrate OpenStack into their Jenkins workflow?

Using the plugin we developed it is very simple to integrate an OpenStack cloud as part of the resources that can be consumed in your Jenkins workflow. All the users will need is to provide a few parameters, such as endpoints, credentials, etc., and they will be able to start deploying to their OpenStack cloud.

How is the open source nature of this workflow an advantage for the organizations using it?

An open source project always has the benefit of having multiple people contributing and improving the code. It is always a good thing to have another view on a project with a fresh outlook. It improves the functionality, the quality and the overall experience for everyone.

Looking more broadly to the OpenStack Summit, what are you most excited about for Vancouver?

First and foremost, I look forward to networking with my peers. It is a vibrant and active community.

I would also like to see some tighter collaboration between the operators, the User Committee, the Technical Committee, and the projects themselves to understand what the needs are of those deploying and maintaining OpenStack in the field and to help them to achieve their goals.

One of the major themes I think we will see from this summit will be the spotlight on companies, organizations and others using the products. We'll see why they moved, and how OpenStack solves their problems. Scalability is no longer in question: scaling is a fact.

Where do you see OpenStack headed, in the Liberty release and beyond?

The community has undergone a big change in the last year, trying to define itself in a clearer way: what is OpenStack, and what it is not.

I hope that all involved continue to contribute, and that the projects focus more on features and problems that are fed to them from the field. It is fine line to define, and usually not a clear one, but something that OpenStack (and all those who consider themselves part of the OpenStack community) have to address and solve, together.

At all technology conference I always like to go on to Floor / Marketplace / Solutions Exchange – where vendors try to get your attention and market their product.

Going over the list of vendors from Summit site, the list below are some of the less know companies (at least to me) that caught my eye and I would like to go over during the summit and see what they have to say.

** The blurb I posted is something that I found on each of the respective sites, and does not necessarily provide a comprehensive overview of what each company offers **

Stackato
Stackato allows agile enterprises to develop and deploy software solutions faster than ever before and manage them more effectively. Stackato provides development teams with built-in languages, frameworks and services on one single cloud application platform, while providing enterprise-level security and world-class support.

Akanda
Akanda is the only open source network virtualization solution built by OpenStack operators for real OpenStack clouds. Akanda eliminates the need for complex SDN controllers, overlays and multiple plugins for cloud networking by providing a simple integrated networking stack (routing, firewall, load balancing) for connecting and securing multi-tenant OpenStack environments.

Appcito
Appcito Cloud Application Front-End™ (CAFE) is an easy-to-deploy, unified and cloud-native service that enables cloud application teams to innovate faster and improve user experiences with their applications.

Appformix
Operators and developers can use AppFormix’s versatile software to remove and prevent resource contention among applications from the infrastructure without being invasive to applications. The real-time, state driven control provided by AppFormix’s intuitive dashboard allows efficient management of all I/O resources. For deeper control and customization, access to API driven controls are also easily accessible. Plan infrastructure intelligently and remove the guess work involved in managing finite server resources to create fully optimized data center infrastructure.

Caringo
Caringo Swarm leverages simple and emergent behavior with decentralized coordination to handle any rate, flow or size of data. Swarm turns standard hardware into a reliable pool of resources that adapts to any workload or use case while offering a foundation for new data services.

Cleversafe
Cleversafe’s decentralized, shared-nothing storage architecture enables performance and capacity to scale independently, reaching petabyte levels and beyond.

GuardiCore
Covering all the traffic inside datacenters, GuardiCore offers the only solution combining real-time detection of threats based on deep analysis of actual traffic, real time understanding, mitigation and remediation.

OneConvergence
One Convergence Network Virtualization and Service Delivery (NVSD) Solution takes a policy driven approach and brings in the innovative concept of “Service Overlays” to go along with “Network Overlays” to virtualize networks and services. The solution innovates and extends SDN with Service Overlays for delivering L4 to L7 services with higher-level abstractions that are application friendly.

Quobyte
Quobyte turns your servers into a horizontal software-defined storage infrastructure. It is a complete storage product that can host any application out-of-the-box. Through fault-tolerance, flexible placement and integrated automation, Quobyte decouples configuration and operations from hardware.

Scality
The RING is a software-based storage that is built to scale to petabytes with performance, scaling and protection mechanisms appropriate for such scale. It enables your business to grow without limitations and extra overhead, works across 80% of your applications, and protects your data over 200% more efficiently at 50–70% lower cost.

Scalr
The Scalr Cloud Management Platform packages all the cloud best practices in an extensible piece of software, giving your engineers the head start they need to finally focus on creating customer value, not on solving cloud problems.

StorPool Storage
StorPool is storage software. It runs on standard hardware – servers, drives, network – and turns them into high-performance storage system. StorPool replaces traditional storage arrays, all-flash arrays or other inferior storage software (SDS 1.0 solutions).

Stratoscale
Stratoscale’s software transforms standard x86 servers into a hyper-converged infrastructure solution
combining high-performance storage with efficient cloud services, while supporting both
containers and virtualization on the same platform.

TransCirrus
Core, storage and compute nodes connecting via Extreme Networks

Tufin
Security Policy Orchestration for the World's Largest Enterprises.
Managing security policies on multi-vendor firewalls & cloud platforms.

The OpenStack community is converging on Vancouver next week for the bi-annual summit for all things OpenStack.

I am glad to be joining the event and I would like to share with you a short outline of what public events and activities I will participating in.

• Monday May 18th, 13:30 – OSpod with Niki Akosta and Jeff Dickey
• Monday May 18th, 15:00 - vBrownbag Presentation, “The Oh..!! moment in OpenStack”
• Tuesday May 19th, 17:30 - The Jenkins Plugin for OpenStack: Simple and Painless CI/CD
• Wednesday May 20th, 09:00 – OpenStack and Beyond Podcast

The rest of my time will be spread out over the Cross-Project workshops, the Ops sessions, other sessions and activities.

I am really looking forward to this event and please feel free to come and say hello.

As of late I have been thinking long and hard about if I can in some way contribute in a more efficient way into the OpenStack community.

Almost all of my focus today is on OpenStack, on its architecture and how to deploy certain solutions on top such an infrastructure.

What is the Technical Committee?

It is a group of 13 elected people by the OpenStack ATC’s (Active Technical contributors – a.k.a the people that are actively contributing code to the projects over the last year). There are seven spots up for election for this term, in addition to the six TC members that were chosen 6 months ago for a term of one year.

The TC’s Mission is defined as follows:

The Technical Committee (“TC”) is tasked with providing the technical leadership for OpenStack as a whole (all official projects, as defined below). It enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality...), decides on issues affecting multiple projects, forms an ultimate appeals board for technical decisions, and generally has technical oversight over all of OpenStack.

On Thursday I decided to take the plunge. Here is the email where I announced my candidacy.

This is not a paid job, if anything it more of a “second” part-time job – a voluntary part-time job. There are meetings, email discussions on a regular basis.

There are a number of reasons that I am running for a spot on the TC.

Diversity

In my post The OpenStack Elections - Another Look, I noted that operators were not chosen to for the board. This is something that I think is lacking in the OpenStack community today. The influence that the people who are actually using and deploying the software is minimal if at all. The influence they have is mostly after the fact (at best) and not much of an input of what they would like to have put into the product.

I am hoping to bring in a new perspective to the TC, to help  them understand the needs of those who actually deploy the software and have to deal with it day in and day out. There are valid pain points that they have, and in my honest opinion they feel they are not being heard or not being taken into consideration, at least not enough in their eyes.

Acceptance of others

The people who vote are only those who contribute code. Those who have committed a patch to the OpenStack code repositories. That is the definition of an ATC.

It is not easy to get a patch committed. Not at all (at least that is my opinion). You have to learn how to use the tools that the OpenStack community has in place. That takes time. I tried to ease the process with a Docker container to help you along. But even with that, it still seems (to me) that to get into this group of contributors takes time.

It is understandable. There is a standard of doing things (and rightfully so) so the chances of you getting your change accepted the first time are slim, for a number of reasons that I will not go into in this post.

I think that the definition of contributor should be expanded and not only limited to a those who write the code. There are a number of other ways to contribute.

I know that this will not be an easy “battle to win”. I am essentially asking the people to relinquish the way they have been doing things for the past 5 years and allow those who are not developers, those who do not write the code, to steer the technical direction of OpenStack.

I do think this will be in the best interest of everyone to extend the reach of OpenStack community, to branch out.

More information on the actual election that will run until April 30th can be found here. If you are one of the approximate 1,800 people who is an ATC, you should have received a ballot for voting.

It will be interesting to see the results which should be out in a few days.

As always your thoughts and comments are appreciated, please feel free to leave them below.

I would like to bring to your attention that the voting for the sessions for the upcoming OpenStack Israel Summit on June 15th, 2015 is now open.

Make your voice heard and participate in setting the agenda for the event!

You can find more information and the presentation that I gave last year in this post Recap - Openstack Israel 2014 #OpenStackIL and for your convenience I have embedded the recording below.

I usually have a Linux VM that I use to perform some of my remote management tasks, such a OpenStack CLI commands.

But since I now have a Mac (and yes I am in enjoying it!!) I thought why not do it natively on my Mac. The official documentation on installing clients is on the OpenStack site.

This is how I got it done.

Firstly install pip

easy_install pip

Now to install the clients (keystone, glance, heat, nova, neutron, cinder, swift and the new OpenStack client)

pip install python-keystoneclient python-novaclient python-heatclient python-swiftclient python-neutronclient python-cinderclient python-glanceclient python-openstackclient

First problem – was no permissions

Yes you do need sudo for some things…

sudo –H pip install python-keystoneclient python-novaclient python-heatclient python-swiftclient python-neutronclient python-cinderclient python-glanceclient python-openstackclient

Success!

Or so I thought…

Google led me here - https://github.com/major/supernova/issues/55

sudo –H pip uninstall six

And then

sudo –H easy_install six

And all was good

Quick and Simple!! Hope this is useful!

VMware announced last week the launch of VIO and there are a number of things that I think people are missing and should be pointed out.

The information I have taken is from the Datasheet and publicly available information.

Networking

A great part of the the functionality and flexibility that people use is the option for flexible networking, i.e. creating private networks, routers for example.

That is great – all the functionality is there – with NSX. But how many people are actually using NSX today? How many people have deployed NSX? In a previous article I went through the reasons why this will not be an easy path. So how does that work with what I currently have in my datacenter?

I think it is safe to assume that this will be deployed in an environment with a Distributed virtual switch – we are talking about environments that are using Enterprise Plus after all (and VMware is giving this away for free to everyone with the Ent+ licenses).

So how does OpenStack work with a DvSwitch today?

Well I am sorry to surprise you – but it does not. It only works with nova-network (which is supposed to be deprecated – so caveat emptor). VMware themselves have said that most customers that are using OpenStack and VMware are using NSX (and that they don’t really have much experience with nova-network).

** Edited February 11th, 2015 **

According to a twitter conversations with @hui_kenneth and @danwendlandt last night – VIO GA will support the dvSwitch, only that information is currently not public and is only available to the Beta users. The functionality still will not bet the same as that of NSX.

@maishsk @hui_kenneth full docs are limited to beta customers. GA is very soon. Up to you if you want your blog to be accurate or not :)

— dan wendlandt (@danwendlandt) February 10, 2015

So it boils down to this. The only way to really use OpenStack with vSphere today – in any kind of semi-normal way, is to do it with NSX. Any demo you have seen – Hands-on Labs, presentations all use NSX. And it always something that already exists in the environment you are working with, that is the assumption.

So VMware is giving this away for free (unless you are interested in support – which will cost you another $200 per socket) – but this essentially is giving you a hobbled product – which does not have functionality that you get out of the OpenStack box – because you are using vSphere networking.

So what features will you not be able to use – without NSX?

• No GRE – standard VLANs only
• No LBaaS
• No VPNaaS
• No FWaaS
• No security groups

I will say that the number of people that are actually using FWaaS and VPNaaS are not the majority of OpenStack users – but on the other LBaaS – is more or less an essential part of any automated cloud. And even more so – security groups are definitely an essential part of any cloud.

But of course we would like to use all the bells and whistles – actually I would really only like to use Neutron with vSphere – so my options are only going to be to use NSX (until they manage to get this working as will with a dvSwitch).

So what is this going to cost me?

This is going to hurt (and by no means am I licensing expert – and yes I know that no-one really pays list price – but here goes).

You have two options to buy NSX – per vm or per socket. Now we all know that the per-vm model – usually does not run in the customers favor – and the last time a per-vm model was proposed – the was a huge disturbance in the force. So I am going to assume that you will want a per CPU based license.

1 CPU license of NSX for vSphere (NS-VS-C) – $5,996
1 CPU SNS Basic support (NX-VS-G-SSS-C) for 1 CPU – $1,259
1 Year SNS Production support for VMware Integrated OpenStack for 1 CPU – $200
1 CPU license of VMware Integrated OpenStack (assuming you have Enterprise Plus) – $0

Total cost for 1 CPU of VMware Integrated OpenStack – $5,996
Annual support costs for 1 CPU of VMware Integrated OpenStack (and NSX) – $1,459

So let me lay this out in simple terms with an example.

** Post Updated February 11th **

It was brought to my attention that there is a minimum purchase of 50 CPU licenses for OpenStack support as part of the FAQ notes.

I did not change my assumption that you would only be using 4 hosts but the purchase of additional OpenStack CPU support s required.

I have therefore amended the numbers below.

If you are interested in really using (i.e. with neutron and NSX) OpenStack on a 4 host (8 socket) cluster this will cost you:

• Initial cost
  • NSX licensing – 8x$5,996 = $47,968
  • SnS first year – 8x$(1,259 + 200) = $11,672
  • SnS first year – 8x1,259 + 50x200 = $20,072
  • Total – $59,640
  • Total – $68,040
• Annual costs
  • SnS per year – $11,672
  • SnS per year – $20,072

That is above and beyond the regular licensing fees that you pay for Enterprise Plus licenses (which I have not factored in here – because I am assuming that you already have them. But if you do not, then that is even more of a hit to your CAPEX.

Again I would like to stress – that this is MSRP – and not including any bundles.

My Take

VMware would like to see the whole world run on their platform (obviously), and they have started to make a move to minimize the impact that I think they are starting to feel – due to people moving over OpenStack. This offering is a foot in the door to minimize the business they could lose from people moving off of their platform (seriously speaking though vCloud is a competing product – and I do not know how much longer they can continue to sell competing products). There are a number of benefits of running on top of vSphere of course, the underlying platform – and the hooks and insight into vRO is another one of course.

That is one side of the story. The other side is NSX adoption – I do not think that VMware is seeing the market share that they were hoping to gain with NSX – network virtualization is still not a mainstream concept. Companies are starting to dabble and try – but no – we are not there yet.

The ironic thing is that even with VIO integrated with NSX when it is released – it still will not support native LBaaS, VPNaaS and FWaaS out of the box (you probably will be able to integrate with 3rd party vendors) – that will probably come in a future release.

So even with their flagship product – it still will not have all the functionality that OpenStack Operators/Users are accustomed to have in their environments today.

There are benefits of having “one neck to throttle” so to speak – but that comes with a price tag – and hefty one. It certainly is not a free product as it is being made out to be – or at a minimal cost (VMware support).

The devil is always in the details.

What do you say? Is it financially viable? Would you use VIO? Why? Or would you rather rough it and go with another vendor?

I would be happy to hear your thoughts, and comments. Please feel free to leave them in the comments below.

The Board of directors and the bylaws were approved. Summary posts can be found here (2015 Individual Director Election results) and here (Bylaws amendments approved).

Individual Directors

• Tim Bell
• Russell Bryant
• Alex Freedland
• Rob Hirschfeld
• Vishvananda Ishaya
• Kavit Munshi
• Egle Sigler
• Monty Taylor

The voting numbers can be found here.

Congratulations to all the new and re-elected board members. Well deserved!

I have a few things I would like to add about the data that was presented – and my thoughts.

1. Tim Bell 

Tim received the highest number of votes. CERN is a huge OpenStack user and was showcased at the summit in Paris. He is one of only 3 the board members that are not officially affiliated with a vendor directly involved in OpenStack. I see this as a big vote of confidence by the members of the foundation – that are interested in seeing more representation from non-affiliated members. Something I personally would like to see as well.

2. Participation Numbers

Only just over 16% of the members voted in the election. That to me seems to be very low. and I would like to address the OpenStack community to ask why this is so? This is actually something the Board and Foundation should also be actively looking into as well (perhaps they already are).

• Is it because people are not interested in participating?
• Is it because the importance of the process was not made clear enough?
• People did not find the candidates suitable?

More people actually voted for the amendment changes that in the election itself – which I find quite strange. They were already on the page and did not bother to vote for any of the candidates.

3. Operators were not chosen

Neither Jesse Proudman and Randy Bias were voted in – both seemed to me that they were very vocal in their campaign as to why they wanted to get on the board – and that was to bring a change into the way OpenStack is currently “run”. I do personally think this is pity – because I would have liked to see more representation from the Operator standpoint, something which I still feel is still lacking in the OpenStack community.

4. Participation in Amendment changes

The quorum was achieved – but only by 315 votes. My previous post The OpenStack Foundation – 2015 Individual Director Election – spoke about how I see this change as a problematic one. I personally do not think that the quorum would have been reached – if it was not for the “aggressive” marketing campaign that the Foundation embarked upon in order to reach this quorum. Without the countless number of posts from Board members, Foundation members (myself included) and anyone that cared about this election - on their blogs, social media and everywhere that was possible. Jonathan Bryce even promised to remove his beard to achieve this goal..

If we hit quorum in time, @jbryce will shave his beard! If you haven't voted, vote NOW! pic.twitter.com/ys0g3Q60XS

— OpenStack (@OpenStack) January 16, 2015

And he did!

@jbryce's beard is gone! If you haven't voted, you have 5 MINUTES & soon we'll post the 2015 election results pic.twitter.com/ATFdOe5pQX

— OpenStack (@OpenStack) January 16, 2015

A huge effort indeed, from the foundation and the community at large – but my question is..

How long will it be until it is needed again?

Yes the quorum needed is now officially only 10% (instead of 25%) but I do foresee the day that even that will not be reached (and that will not be too far in the future). That is why I think the change should have been to remove the quorum all together.

5. People were not happy with the change in the quorum

On both the first two amendments, the approval rate was 93-95% – almost everyone agreed. Changing the quorum – only had 80% of the people that approved. Of course that is still a majority and perfectly valid and acceptable as a decision, but still it is interesting to see that more people were not happy with the change.

It would be interesting to know if it was the lowering of the percentage to 10% or was it that the proposed change should have been to remove the quorum altogether. I personally voted against this change because I think that the quorum should be removed completely and not lowered to 10%.

I am very pleased that the changes were made – because it allows OpenStack to continue to grow, but I do think that planning should start now – for when the changes made in these amendments will not suffice and need to be changed again.

If you are willing to share - I would be interested in hearing your thoughts about the points above. Please feel free to leave them in the comments below.