2014-01-02

Some Thoughts About Openstack High Availability

As of late I have been putting some thought into how the underlying infrastructure that is needed for running and OpenStack environment and the lack of built-in high availability solutions to provide a robust (and yes I might even go as far as saying - an "Enterprise-Ready") solution.

First let's go over what are the components/services that run in Openstack - the easiest would be to quote the OpenStack documentation.

openstack_havana_conceptual_arch

To simplify this I am not going to go into each and every single service that needs to be redundant but rather to go at a slightly higher level (not that much) and propose that the the following architecture could be a possible solution.

I will not go into the exact steps needed to provide the clustering mechanism for each component, I think there are more than enough resources that have done an amazing job already ( I will say though that it is not straight forward, and multiple technologies and solutions are required for the components).

  1. Introduction to OpenStack High Availability
  2. Practical Lessons from Building a Highly Available Openstack Private Cloud
  3. High Availability Update
  4. Openstack HA @Paypal

(Just as a side note - it would be a great idea to write out an article as to what operations will not be possible/what will not be available, if one of the above services/components are down)

If we were to use the basic model above - we would need at minimum 6 servers/instances/VM's - and to simplify for the sake of the article - the amount of resources allocated to each of these VM's are not really the issue.

What has been occupying me lately - is where do you should these workloads be placed? The workloads do not warrant their own physical hardware (in my humble opinion) - because they could easily run as virtual machines. This could you bring you to a solution such as the following (and by no means is this the correct/optimal way)

Openstack HA

Let's see how we can answer that $1,000,000 question.

Obviously you cannot run them under Openstack itself. Firstly the concept of building Openstack hosted on Openstack is quite problematic. In a vSphere environment this is possible because each and every host can be managed on it's own and you can install a full vCenter / vCloud environment on a standalone host - and have that managed by the VM's it is running. Is this a not always a good idea - and the VCAT documentation states that a Management Cluster is a good design practice. Separating the management layer from the actual layer that runs the resource workloads is a very good idea (if it is a viable option for your environment)

I have yet to find a way that will enable you to import instances that are already running on a nova-compute node.
Not possible.

The way I would design this environment is something like the following architecture as a management cluster.

Openstack HA1

It is not a necessity to have an underlying HA solution on the hypervisor layer (although the benefits are obvious). The components themselves will be using an application clustering solution and if one of the hosts go down (with the control nodes), the services will continue to function - albeit in a degraded state - but you should not have any major loss of functionality.

So what could you use as the host for these control nodes? vSphere, Hyper-V, or perhaps KVM? I am sure there are pros and cons for any of the above solutions, and this is again, is something that you need to take into account when deploying your infrastructure.

Some food for thought on the dawn of this new year.

I would be grateful to hear your comments and ideas on this matter. Please feel free to leave them in the comments below.