2018-10-15

How Long Until you Get the New Shiny Toys from re:Invent?

re:Invent is coming - and the frenzy of releases that will build up to the event is just around the corner.

I have always had in the back of my mind that all the products announced at re:Invent are great for the press releases and the small digs at other vendors, but sometimes it takes a while until we actually get what was announced on stage in front of ~20,000 people and the rest of the world.

And I went out to look for some data. It is obvious that not everything that we heard about on stage was baked and ready for production use.

Andy Jassy - re:invent 2017 keynote

Here are some examples from last years re:Invent


re:Invent 2017

EKS (188 days)

https://aws.amazon.com/blogs/aws/amazon-elastic-container-service-for-kubernetes/
https://aws.amazon.com/blogs/aws/amazon-eks-now-generally-available/ (June 5, 2018)

 
Bare Metal (170 days)

https://aws.amazon.com/blogs/aws/new-amazon-ec2-bare-metal-instances-with-direct-access-to-hardware/
https://aws.amazon.com/about-aws/whats-new/2018/05/announcing-general-availability-of-amazon-ec2-bare-metal-instances/ (May 17, 2018)

 
Serverless App repo (83 days)

https://aws.amazon.com/blogs/aws/aws-serverless-app-repo/
https://aws.amazon.com/blogs/aws/now-available-aws-serverless-application-repository/ (Feb 21, 2018)

 
Neptune (183 days)

https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-neptune-fast-reliable-graph-database-built-for-the-cloud/
https://aws.amazon.com/blogs/aws/amazon-neptune-generally-available/ (May 30, 2018)

 
Aurora Multi-master (Still not released)

https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/
Yet to be released (Oct 14, 2018)

 
Aurora Serverless (254 days)

https://aws.amazon.com/blogs/aws/in-the-works-amazon-aurora-serverless/
https://aws.amazon.com/blogs/aws/aurora-serverless-ga/ (Aug 9, 2018)

 
IOT 1-click (169 days)

https://aws.amazon.com/about-aws/whats-new/2017/11/aws-iot-one-click-now-in-preview/
https://aws.amazon.com/about-aws/whats-new/2018/05/aws-iot-1-click-generally-available/ (May 16, 2018)

 
Translate (127 days)

https://aws.amazon.com/blogs/aws/introducing-amazon-translate-real-time-text-language-translation/
https://aws.amazon.com/blogs/aws/amazon-translate-now-generally-available/ (Apr 4, 2018)

 
Transcribe (127 days)

https://aws.amazon.com/blogs/aws/amazon-transcribe-scalable-and-accurate-automatic-speech-recognition/
https://aws.amazon.com/blogs/aws/amazon-transcribe-now-generally-available/ (Apr 4, 2018)

 
Appsync (137 days)

https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/
https://aws.amazon.com/about-aws/whats-new/2018/04/aws-appsync-now-ga/ (Apr 13, 2018)

 
S3 Select (126 days)

https://aws.amazon.com/blogs/aws/s3-glacier-select/
https://aws.amazon.com/about-aws/whats-new/2018/04/amazon-s3-select-is-now-generally-available/ (Apr 3, 2018)


re:Invent 2016

Lex (141 days)

https://aws.amazon.com/blogs/aws/amazon-lex-build-conversational-voice-text-interfaces/ https://aws.amazon.com/blogs/aws/amazon-lex-now-generally-available/ (Apr 19, 2017)

 
PostgreSQL for Aurora (329 days)

https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgresql-compatibility/ 
https://aws.amazon.com/blogs/aws/now-available-amazon-aurora-with-postgresql-compatibility/ (Oct 24, 2017)

 
GreenGrass (190 days)

https://aws.amazon.com/blogs/aws/aws-greengrass-ubiquitous-real-world-computing/
https://aws.amazon.com/blogs/aws/aws-greengrass-run-aws-lambda-functions-on-connected-devices/ (Jun 07, 2017)

 
X-Ray (140 days)

https://aws.amazon.com/blogs/aws/aws-x-ray-see-inside-of-your-distributed-application/
https://aws.amazon.com/blogs/aws/aws-x-ray-update-general-availability-including-lambda-integration/ (Apr 19, 2017)

 
Batch (36 days)

https://aws.amazon.com/blogs/aws/aws-batch-run-batch-computing-jobs-on-aws/
https://aws.amazon.com/about-aws/whats-new/2017/01/aws-batch-now-generally-available/ (Jan 5, 2017)

 
Lambda Edge (229 days)

https://aws.amazon.com/blogs/aws/coming-soon-lambda-at-the-edge/
https://aws.amazon.com/about-aws/whats-new/2017/07/lambda-at-edge-now-generally-available/ (Jul 17, 2017)

 
At a glance it looks like the average amount of time from the list above was about 5 months.

Now don’t get me wrong. For all of the above items that were not actually available at re:Invent - I would estimate that there were the same number of products (if not more) that were available (at least in a limited number of regions) the same day they were announced. Above and beyond - the problems that AWS is trying solve and really complex - and a almost all of them have never been done before - so please AWS take your time in developing the game changing technology that you have been giving to the world.

So when Andy Jassy and Werner Vogels get up on stage at the end of November, and announce whatever wonderful stuff they are going to announce - we should all take into account that it could take anything from 1 day to almost a year until we can actually use it in all the AWS regions that we are consuming today.


Werner Vogels - re:invent 2017 keynote

How can this / does this affect you? I can give an example from the EKS announcement. We were actively looking at a kubernetes deployment on AWS and were contemplating whether we should deploy our own or wait for the managed solution that was announced at re:Invent.

Since we did not have an official release date - we decided to roll our own - and not wait for some some unknown time in the future.

It is nice to know what is coming. You will need to evaluate how long you can wait - are you ready to go with a version one product (that could / will probably have a good number of limitations) or come up with a contingency plan to solve your issues.

2018-10-08

#AWS PrivateLink vs. NAT Gateway from a Pricing Perspective

A customer came to me with a request. They do not want to use a NAT gateway from their VPC to access the AWS API's. They had a number of security concerns regarding the use of a NAT gateway (no control, logs, auditing - but that is a for a different post) and they asked for a solution.

The AWS API's that they needed access to were:Endpoints

  • S3
  • KMS
  • SSM
  • Cloudwatch
  • Cloudformation

Last year at re:Invent AWS announced the option to create VPC Interface endpoints using PrivateLink and have steadily been adding more endpoints over the past year.

With the use of these endpoints you can actually have a VPC with instances that will not have any internet access (at least not through AWS) and still be able to interact with all the AWS API's.

This is technically possible - and can easily be automated, but I wanted to look at the cost perspective.

The VPC in us-east-1 has 2 Availability Zones (you should always have at minimum 2).

That would mean deploying 2 NAT gateways in your VPC (Pricing)

I am going to assume that you have the same amount of data going through both options - so I will not factor this into the price.

Usually you have 730 hours in a month.

Each NAT gateway will cost you 0.045*730 = ~$33.

Total for 2 NAT Gateways would be $66 per month (not including traffic).

What does this look like for Interface Endpoints? (Pricing)

Each Endpoint will need to be deployed in both AZ's in pairs.

Each Interace Endpoint will cost 0.01*730*2 = ~15

Total for all the endpoints above (4 Interface Endpoints - KMS, SSM, CloudWatch and Cloudformation) would be $60 per month.
The S3 endpoint is a Gateway endpoint - and therefore does not cost you any extra.

As you can see - it is not that much cheaper.

Take into account the following scenario - you need API access to 15 out of the 21 possible interface Endpoints

This would run you the steep amount of $225 per month - which is a lot more than just a NAT Gateway.

Design decisions always have tradeoffs - sometimes you prefer security and other times it will be cost. I hope that this will enable you to make an informed decision in your VPC design.