2018-08-19

A Triangle is Not a Circle & Some Things Don’t Fit in the Cloud

Baby Blocks

We all started off as babies, and I am sure that not many of you remember that one of the first toys you played with (and if you do not remember - then I am sure those of you with kids have probably done the same with your children) was a plastic container with different shapes on the lid and blocks that were made of different shapes.

A triangle would only go into the triangle, a circle in the circle, a block in the block and so on.

This is a basic skill that teaches us that no matter how hard we try, there are some things that just do not work. Things can only work in a certain way (of course coordination, patience and whole lot of other educational things).

It is a skill that we acquire, it takes time, patience, everyone gets there in the end.

And why am I blogging about this – you may ask?

This analogy came up a few days ago in a discussion of a way to provide a highly available database in the cloud.

And it got me thinking….

There are certain things that are not meant to be deployed in a cloud environment because they were never meant to be there in the first place. The application needed an Oracle database and it was supposed to be deployed in a cloud environment.

What is the default way to deploy Oracle in highly available configuration? Oracle RAC. There are a number of basic requirements (simplified) you need for Oracle RAC.

  1. Shared disk between the nodes.
    That will not work in a cloud environment.
    So we can try using dNFS – as the shared storage for the nodes – that might work..
    But then you have to make an NFS mount available to the nodes – in the cloud.
    So let’s deploy an NFS node as part of the solution.
    But then we have to make that NFS node highly available.
  2. Multicast between the nodes - that also does not work well in the cloud.
    So maybe create a networking environment in the cloud that will support multicast?
    Deploy a router appliance in the cloud.
    Now connect all the instances in the cloud into the router.
    But the router poses as a single point of failure.
    Make the router highly available.

And if not Oracle RAC – then how about Data Guard – which does not require shared storage?

But it has a steep licensing fee.
And you have to find a way for managing the virtual IP address – that you not necessarily will have control over.
But that can be overcome by deploying a VRRP solution with IP addresses that are manually managed.

ENOUGH!!!

Trying to fit a triangle into a square – yes if you push hard enough (it will break the lid and fit).
If you cry hard enough – Mom/Dad will come over and put it in for you.

Or you come up with half-assbaked solution like the one below…

blocks

Some things will not fit. Trying to make them fit creates even more (and sometimes even bigger) problems.

In this case the solution should have been - change the code to use a NoSQL database that can be deployed easily and reliably in a cloud environment.

As always your thoughts and comments are welcome.

2018-08-15

Saving a Few Shekels on your AWS bill

I have a jumpbox that I use to access resources in the cloud – and I use it at work, only during work hours and only on workdays.

There are usually 720 work hours in the month or 744 in months that have 31 days. Assuming that I want to run the instance for 12 hours a day and for 5 days a week. In order to calculate how many hours exactly – we will need an example.

The month of August, 2018

image

The work week in Israel is Sunday-Thursday (yeah – I know – we are special…).

August would have 22 work days. Total number of hours in August (31*24 = 744). 220 working hours in the month (22 working days multiplied by 10 hours per day).

The math is simple 220/744 – I only need the instance for 30% of the month – so why would I pay for all of it?

744 hours * $0.0464 (for a t2.medium instance in us-east-2) = $34.5216 and if I was to only pay for the hours that I was actually using the instance that would be 220 * $0.0464 = $10.208. A third of the cost. Simple math.

So there are multiple ways to do this – as a Lambda script, Cloud custodian – each of these work – very well and will work brilliantly at scale. For me it was a single machine and honestly I  could not be bothered to set up all the requirements to get everything working.

Simple solution – use cron. I don’t pay for resource usage by hour in my corporate network (If someone does – then you have my sympathies..) so I set up a simple cron job to do this.

To start up the instance:

0 8 * * 0,1,2,3,4 start-jumpbox

And to stop the instance at night

0 18 * * 0,1,2,3,4 stop-jumpbox

And what is the start/stop-jumpbox comand you might ask – really simple aws cli command

aws ec2 start-instances –region <__REGION__>  --instance-ids <__INSTANCE_ID__>

aws ec2 stop-instances –region <__REGION__>  --instance-ids <__INSTANCE_ID__>

Of course in the background the correct credentials and access keys are set up on my linux box – not going to go into how to that here – AWS has enough documentation on that.

The last thing that I needed to solve was the jumpbox has a public IP (obviously) and If I really wanted to save the money – I do not want to have pay for a static Elastic IP provisioned and sitting there idle for 70% of the month (because the instance is powered down

After doing the calculation – it was chump change for my use case (524hrs * $0.005=$2.62) so maybe I should have not worried about it – but the resulted script is still useful.

I wanted to use the allocated IP address that AWS provides to the instance at no cost. The problem with this is – every time you stop the instance the IP address is reclaimed by AWS and when you power it on – you will get a new one.

Me being the lazy bum I am – I did not want to have to lookup up the IP each and every time so I went down the path of updating a DNS record on each reboot.

Take the Public IP allocated to the instance and update a known FQDN that I would use on a regular basis.

The script can be found here (pretty self explanatory)



Now of course this is only a single instance – but if you are interested in saving money this is one of the considerations you think about looking to save. (of course this should be managed properly at scale a single cron job will not suffice…)

For example – if you have a 1000 development machines that are not being used after working hours (and I know that not everything can be shut down after hours there are background jobs that run 24/7),  and they are not a measly t2.medium but rather an m4.large 

1000 (instances) * 220 (work hours) * $0.1 (cost per hour) = $22,200

1000 (instances) * 744 (hours in the month) * $0.1 (cost per hour) =  $74,400

See how you just saved $50k a month on your AWS bill?

You are welcome :)

(If you want spend a some your well saved cash on my new book – The Cloud Walkabout – feel free).

If you have any questions about the script / solution or just want to leave a comment – please feel free to do so below