tag:blogger.com,1999:blog-58196406943858434902024-03-05T18:32:34.272+02:00TechnodroneGoing Virtual In The Physical WorldMaish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comBlogger700125tag:blogger.com,1999:blog-5819640694385843490.post-15734484168232951182020-01-06T22:15:00.001+02:002022-03-24T15:21:03.548+02:00This will be my last post... On this blogWait...<br />
What??<br />
Where are you going???<br />
Is everything OK????<br />
You are stopping to write blog posts?????<br />
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<iframe allowfullscreen="" class="giphy-embed" frameborder="0" height="392" src="https://giphy.com/embed/spfi6nabVuq5y" width="480"></iframe><br /></div>
<div style="text-align: center;">
<br /></div>
<span style="color: red; font-family: "times" , "times new roman" , serif; font-size: x-large;"><b>HELL NO!!</b></span><br />
<br />
My <a href="https://technodrone.blogspot.com/2007/11/welcome-to-blog.html" target="_blank">first blog post</a> brings back memories. When I decided to start this blog way, way back in November <u style="font-weight: bold;">2007</u>, I decided to go for a free platform, because it suited my needs at the time, and honestly it has served me really well over the last 12 years.<br />
<br />
But it is time for a change.<br />
<br />
I have evolved over the last 12 years. The way I work, has evolved over the last 12 years, and this platform is no longer in line with my daily routines and practices.<br />
<br />
Which is why I have decided to move to a new blog, a new domain, and a new beginning.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blog.technodrone.cloud/" target="_blank"><img alt="Technodrone" border="0" data-original-height="501" data-original-width="800" height="400" src="https://maishsk.com/blog/images/20190625_last_post/technodrone.png" title="Technodrone" width="640" /></a></div>
<br />
<br />
<div style="text-align: center;">
<a href="https://blog.technodrone.cloud/" target="_blank"><span style="font-size: x-large;">https://blog.technodrone.cloud</span></a></div>
<br />
<span style="font-size: large;">Q: </span>Are you closing access to this site?<br />
<span style="font-size: large;">A: </span> No all content will be remain here - although I have set up redirects for all the posts that will now point to the new blog.<br />
<br />
<span style="font-size: large;">Q: </span>You have Visio Stencils on your site - can I still get them?<br />
<span style="font-size: large;">A: </span>Yes, the old links still are there - and the posts are now redirected to the new site..<br />
<br />
<span style="font-size: large;">Q: </span>Will the URL and rss feed stay the same?<br />
<span style="font-size: large;">A: </span>Nope, new domain, new urls, <a href="https://blog.technodrone.cloud/index.xml" target="_blank">new feed</a>.<br />
<br />
<span style="font-size: large;">Q: </span>Are you still using the blogger platform??<br />
<span style="font-size: large;">A: </span>You weren't paying attention were you... I am moving off of blogger. The site is going to be hosted as a static site on <a href="https://aws.amazon.com/s3/" target="_blank">AWS S3</a>, powered by <a href="https://gohugo.io/" target="_blank">Hugo</a> - more on how the sausage is made in the future..<br />
<br />
<span style="font-size: large;">Q: </span>Will this site design stay the same?<br />
<span style="font-size: large;">A: </span>No - I am going to change the design to something much more simple and minimal, the info is outdated and needs to be changed to reflect the times.<br />
<br />
Here is a small time lapse of the site over the years.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190625_last_post/timelapse.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="380" data-original-width="800" height="304" src="https://maishsk.com/blog/images/20190625_last_post/timelapse.gif" width="640" /></a></div>
<br />
<br />
Thanks Google for the hospitality over the years, I am extremely grateful, you have served me well.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190625_last_post/stats.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="706" data-original-width="800" height="564" src="https://maishsk.com/blog/images/20190625_last_post/stats.png" width="640" /></a></div>
<br />
<div style="text-align: center;">
<div style="text-align: left;">
Start your engines and add this to your bookmarks.... <a href="https://blog.technodrone.cloud/2020/01/welcome-to-blog-2.0/" target="_blank">Welcome to Blog 2.0</a></div>
</div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-69299293287610367562019-06-12T18:59:00.000+03:002019-06-12T18:59:00.132+03:00Starting a new Journey #AWS<span style="font-family: inherit;">Simon Sinek has a <a href="https://www.ted.com/talks/simon_sinek_how_great_leaders_inspire_action" target="_blank">great talk</a> - about how great leaders inspire great action. I learned something really important from this talk even through it is almost 10 years old.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/qp0HIF3SfI4" width="640"></iframe>
<span style="font-family: inherit;"><br /></span><span style="font-family: inherit;"><br /></span><br />
<span style="font-family: inherit;">By explaining things in the wrong way - we miss the opportunity to make a great impact, to change the world.</span><br />
<ol>
<li><span style="font-family: inherit;">We usually start with the <b>What</b>.</span></li>
<li><span style="font-family: inherit;">Then the <b>How</b>..</span></li>
<li><span style="font-family: inherit;">And only at the end - we get into the <b>Why</b>...</span></li>
</ol>
<br />
It should be the other be the reverse.<br />
<br />
Following Simon's advice I will start with the why..<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20191206_new_journey/how_why_what.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="How? Why? What?" border="0" data-original-height="621" data-original-width="800" height="310" src="https://maishsk.com/blog/images/20191206_new_journey/how_why_what.png" title="How? Why? What?" width="400" /></a></div>
<br />
<h3>
<b><u>Why?</u></b></h3>
I firmly believe that the future is the public cloud. I believe that we can accomplish so much more, so much faster, when we leave heavy lifting for others. This allows all of us to focus on providing the <b>actual</b> value to our customers without having to worry about the underlying infrastructure.<br />
<br />
I know that I have a huge amount still to learn, but I also have a huge amount of knowledge, experience and insight that I can share with others. I have been doing this for many years, and see this not only as a way to put bread on the table, but also a way to make a real change in the world.<br />
<br />
I want other people to benefit from what I have to give.<br />
<b><u><br /></u></b>
<br />
<h3>
<b><u>How?</u></b></h3>
I work with teams on how to start their journey to the cloud, how to make use of the technologies available to them. This includes, writing code, continuously learning (myself included), gaining more knowledge, and ultimately sharing that knowledge with others. I have built pipelines, migrated workloads into the cloud, failed miserably in some cases, continuously improved and iterated to get better the whole time.<br />
<br />
Working on a regular basis with customers to help them on their journey, through their challenges along the way, celebrate their success stories with them, experience the pain and anguish with their failures / disasters - but above all - to be an advisor for my clients - with their best interests in mind.<br />
<b><u><br /></u></b>
<br />
<h3>
<b><u>What?</u></b></h3>
The change I have decided to embark on (and the challenge I have decided to accept) is moving my skills and energy in a direction where I feel I can make even more of an impact, help more people, help even more organizations, and not only focus on a single company, but make even a bigger impact.<br />
<br />
Starting July 15th I will joining <b>Amazon (Web Services) </b>as a <b>Senior Solutions Architect</b>.<br />
<br />
I will be working with an amazing team of solutions architects and talented people in a company that I really believe can change the way we use technology, make it better, more efficient, and do amazing things.<br />
<br />
My last day at CyberArk will be June 30th, then I go on a long deserved and well earned vacation for two weeks.<br />
<br />
I have learned a huge deal at my time here at CyberArk, worked with amazing people, learned a lot about the security space, their challenges, their fears, their constraints. None of it is easy. It is not a cloud native world and the problems this industry faces are not easy ones to solve, especially in what could be termed as "legacy" environments. For all this knowledge, the insight and experiences over the last year - I am extremely grateful.<br />
<br />
I cannot wait for <a href="https://www.youtube.com/watch?v=fTwXS2H_iJo" target="_blank">day 1</a> on July 15th!!!Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-76687276473521451102019-06-10T09:29:00.005+03:002019-06-12T12:24:00.108+03:00Book Review: Mastering AWS Cost OptimizationI dabble in AWS every now and again :) and a new book just came out - so obviously I wanted to go through it and give it a read.<br />
<br />
<a href="https://www.amazon.com/Mastering-AWS-Cost-Optimization-operational/dp/965572803X" target="_blank">Mastering AWS Cost Optimization: Real-world technical and operational cost-saving best practices</a> <br />
(Eli Mansoor and Yair Green)<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://images-na.ssl-images-amazon.com/images/I/41T1Ftn020L.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="500" data-original-width="405" height="200" src="https://images-na.ssl-images-amazon.com/images/I/41T1Ftn020L.jpg" width="161" /></a></div>
<div>
<br />
<div>
<br /></div>
<div>
<i><b>So first some disclosure - I have met with Eli a few times throughout my career - we had some business discussions during his Rackspace days. Eli Reached out to me and asked me to read the book and post a review. </b></i></div>
<div>
<i><b><br /></b></i></div>
<div>
<i><b>I received a free paperback copy. </b></i></div>
<div>
<br /></div>
<div>
I finished the book in two days, (it was chag and I had a lot of time to read). It quite clear that a lot of knowledge and detail went into the writing of the book. </div>
<div>
<br /></div>
<div>
Eli and Yair took a methodological approach throughout the book. They focused on three main aspects of your AWS cost (with a strong emphasis on Compute and Storage, but also Networking).</div>
<div>
<br /></div>
<div>
They used a methodology which they name KAO (Knowledge, Architecture, Operation) which in my honest opinion provided a logical and clear flow for the book and made it an easy read. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190610_aws_cost_optimize/kao.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="KAO Methodology" border="0" data-original-height="188" data-original-width="659" height="182" src="https://maishsk.com/blog/images/20190610_aws_cost_optimize/kao.png" title="KAO Methodology" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<div>
They go into detail on how each of the services are used - sometimes in really great detail. There are a number of examples in the book used to explain how things exactly work. </div>
<div>
<br /></div>
</div>
<div>
The last section of the book is focused on Operations with different suggestions and recommendations on how to adjust you current practices, to become more "cost aware/optimized". I for one would have preferred that this section would have been more of the focus of the entire book - but that could just be me. There are a great number of gems in this section - that has something new for everyone (me included!!) </div>
<div>
<br /></div>
<div>
As I said, this was an easy read, well structured and very informative. Eli and Yair have done a great job, diving deep on a topic that is important to us all (but has no real good source of information - besides experience) - but have also left enough space to expand on this book and to provide a more detailed deep dive and focus on more specific subjects in the future.</div>
</div>
<div>
<br /></div>
<div>
I would definitely give it a read!</div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-24026963973482958452019-05-27T10:00:00.000+03:002019-05-27T13:47:16.160+03:00(Not) Real Scientific Proof that AMI has #3syllables<a href="https://aws.amazon.com/" target="_blank">AWS</a> has 26, (yes) I counted them, different products with exactly 3 letters in them (or derivatives of) - lets go through them one at a time.<br />
<br />
<br />
<ul>
<li><b>A-C-M</b> AWS Certificate Manager - Is not pronounced ac-em (also not hack-em) </li>
<li><b>D-M-S</b> Database Migration Service - Is not pronounced dems nor dee-miss (and also not dimms)</li>
<li><b>E-B-S</b> Elastic Block Store - Is not pronounced ebbs (and we are not being washed back out to sea), nor ee-bzz (people might be allergic to bees) </li>
<li><b>E-C-2</b> (Well it should actually be E-C-C - but EC2 sounds so much sexier) Elastic Compute Cloud - Is not pronounced ek-2 (or even eck - otherwise people might get confused with "what the heck2")</li>
<li><b>E-C-R</b> - Elastic Container Registry - Is not pronounced Ecker-R (sounds too much like pecker) </li>
<li><b>E-C-S</b> - Elastic Container Service - Is not pronounced eh-ckes neither ee-cees nor Ex (People would be wary to use a product named Amazon X - they might think that AWS is taking after Google with their Alphabet) </li>
<li><b>E-F-S</b> - Elastic File System - Is not pronounced ef-s neither ee-fees nor eefs</li>
<li><b>E-K-S</b> - Elastic Container Service for Kubernetes - pronouncing this x-kay (ECS-K) would sound too much like Xray (another AWS product). Also see above about E-C-S </li>
<li><b>E-M-R</b> Elastic MapReduce - We don't call it ee-mer - nor emmer (otherwise all the <a href="https://m.interglot.com/nl/en/emmer" target="_blank">Dutch people</a> might think that this is an S3 look-alike) </li>
<li><b>F-S-X</b> - I can't find what this stands for - except for FSx :) - not ef-sex (that is not politically correct..) </li>
<li><b>I-A-M</b> - Identity and Access Managment - no-one uses I-AM - (Dr. Suess would be happy with I-AM-SAM - <a href="https://en.wikipedia.org/wiki/Sam_I_Am" target="_blank">SAM-I-AM</a>) </li>
<li><b>I-O-T</b> - Internet Of Things - Not eye-ot (people might think there are more than <a href="https://www.youtube.com/watch?v=HI0x0KYChq4" target="_blank">7 dwarfs in the service</a> - eye-o, eye-o it's off to work we go..) </li>
<li><b>K-M-S</b> Key Management Service - Is not pronounced kems - nor kee-mes (keemes - the new AWS meme-as-a-service product is probably not a good idea either) </li>
<li><b>L-E-X</b> - this is actually the product name - Amazon Lex - even though the French might have enjoyed it if it was actually Le'X (but then again people don't like having their Ex in the spotlight) </li>
<li><b>M-S-K</b> - Managed Streaming for Kafka - Is not pronounced musk (Elon might not like it), em-sek (could be too fast for us to use). And of course AWS had to name a product after <a href="https://www.linkedin.com/in/maish/" target="_blank">me</a>.</li>
<li><b>P-H-D</b> - Personal Health Dashboard - Is not pronounced pee-hud and phud - would get them in trouble with spreading Fear Uncertainty and Doubt</li>
<li><b>R-A-M</b> - Resource Access Manager - Not (a battering) ram (nor the the ancient Indian king <a href="https://en.wikipedia.org/wiki/Raam" target="_blank">Raam</a>) </li>
<li><b>R-D-S</b> - Relational Database Service - Is not pronounced ar-dis, nor ar-dees (and definitely not the new <a href="https://en.wikipedia.org/wiki/TARDIS" target="_blank">time machine</a> service - tardis) </li>
<li><b>S3</b> - Simple Storage Service - This is a 3 letter product - S-S-S (S3 is so much sexier) - Not sss (people might think there are snakes) - here I conceded - ess-ess-ess brings up <a href="https://www.jta.org/2019/05/26/global/dangerous-to-wear-a-kippah-in-public-germanys-anti-semitism-czar-says" target="_blank">really bad</a> vibes </li>
<li><b>S-E-S</b> - Simple Email Service - Is not pronounced Sess nor sees (otherwise us customers might think this is a new tax in eu-west-1 or ap-south-1) </li>
<li><b>S-N-S</b> - Simple Notification Service - Is not pronounced S-ness, neither sneeze nor Sans (and not <a href="https://en.wikipedia.org/wiki/Loch_Ness_Monster" target="_blank">nessie</a> either - she is still somewhere in the Loch) </li>
<li><b>S-Q-S</b> - Simple Queue Service - Is not pronounced see-ques - nor squeeze </li>
<li><b>S-S-O</b> - Single Sign On - Is not pronounced sa-so neither ses-o nor se-so (just because I say so) </li>
<li><b>S-W-F</b> - Simple Workflow Service - Is not pronounced see-wiff - nor Swiff </li>
<li><b>V-P-C</b> - Virtual Private Cloud - Is not pronounced vee-pic, neither ve-peec nor veep-see </li>
<li><b>W-A-F</b> - Web Application Firewall - I concede - this one is #1syllable - there I said it! BUT IT IS NOT #2syllables !!</li>
</ul>
<div>
<br /></div>
Except for three exceptions (S3, LEX and WAF) - <b><u>all</u></b> the three letter products in AWS - are all pronounced with three syllables!!!!<br />
<div>
<br /></div>
<div>
Just like A-M-I - which has <a href="http://ami-has-3-syllables.online/" target="_blank">#3syllables</a> </div>
<div>
<br /></div>
<div>
I rest my case. </div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-52421396114328982432019-03-18T10:53:00.001+02:002019-03-18T10:53:44.617+02:00The #AWS EC2 Windows Secret SauceNow that I have got your attention with a catchy title - let me share with some of my thoughts regarding how AWS shines and how much your experience as a customer matters.<br />
<br />
Deploying instances in the cloud is something that is relatively fast - at least when it comes to the deployment of a Linux instance.<br />
<br />
Windows Operating Systems - is a whole different story.<br />
<br />
Have you ever thought why it takes such a long amount of time to deploy a Windows instance in the cloud? There are a number of reasons why this takes so much longer.<br />
<br />
Let me count the ways:<br />
<ol>
<li>Running Windows in the cloud - is a dumb idea - so you deserve it!! (just kidding :) ) </li>
<li>Seriously though - Windows images are big - absolutely <b>massive</b> compared to a Linux image - we are talking 30 times larger (on the best of days) so copying these large images to the hypervisor nodes takes time.</li>
<li>They are slow to start.. Windows is not a thin operating system - so it takes time. </li>
</ol>
With all the above said - it seems that AWS has created a really interesting mechanism with which they can reduce the amount of time it takes for an instance to start. Yes they say it can take anything up to 4 minutes for you to be able to remotely connect to the instance - but if you think about it - that is really a very short amount of time.<br />
<br />
I started to look into the start time of Windows (for a whole different reason) and found something really interesting.<br /><br />
<blockquote class="tr_bq">
<i><b>This is not documented anywhere - and I doubt I will receive any confirmation from AWS on the points in this post - but I am pretty confident that is the way this works.</b></i></blockquote>
<br />
<br />
It seems that there is a standby pool of Windows instances that are just waiting in the background to be allocated to a customer - based on customer demand. <br /><br /><b>Let that sink in for second, this means there is a <u>powered-off</u> Windows instance - somewhere in the AZ <u>waiting for you</u>. </b><br /><br />When you request a new Windows EC2 instance, an instance is taken from the pool and allocated to you. This is some of the magic sauce that AWS does in the background.<br />
<br />
This information is not documented anywhere - I have only found a single reference to this behavior on one of the AWS forums - <a href="https://forums.aws.amazon.com/thread.jspa?threadID=221700" target="_blank">Slow Launch on Windows Instances</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/forum_post_slow.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="forum_post_slow" border="0" data-original-height="357" data-original-width="787" height="180" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/forum_post_slow.png" title="forum_post_slow" width="400" /></a></div>
<br />
<br />
<br />
I did some digging of my own and went through the logs of a deployed Windows instance and this provided me with a solid picture of how this actually works. This is what I have discovered about the process (with the logs to back it up).<br /><br />The date that this was provisioned was the 17th of March.<br />
<ol>
<li>On the 17th I launched a Windows instance in my account at 13:46:41 through the EC2 console.<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/ec2_launch.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="ec2_launch" border="0" data-original-height="58" data-original-width="522" height="41" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/ec2_launch.png" title="ec2_launch" width="400" /></a><br /></li>
<li>You can see that AWS does not make the instance available for about 4 minutes - until then you cannot login <br /><i><br />(have you ever wondered why?? - hint, hint carry on reading.. )<br /></i><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/4-minutes_user-data.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="4_minutes" border="0" data-original-height="406" data-original-width="696" height="231" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/4-minutes_user-data.png" title="4_minutes" width="400" /></a></li>
<li>After waiting for just under 4 minutes I logged into the instance and from the Windows event log - you will see that the first entry in the System log is from <b><span style="color: red;">February 13th at 06:52</span></b> (more than a month before I even requested an instance). <br /><br />This is the day that the <a href="https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/windows-ami-version-history.html#amis-2019" target="_blank">AMI was released</a>.<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/first%20boot-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="1st_boot" border="0" data-original-height="634" data-original-width="800" height="316" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/first%20boot-1.png" title="1st_boot" width="400" /></a><br /></li>
<li>At 06:53 that same day the instance was generalized and shutdown<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/sysprep-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="sysprep" border="0" data-original-height="290" data-original-width="800" height="145" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/sysprep-2.png" title="sysprep" width="400" /></a><br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/sysprep-shutdown.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="shutdown" border="0" data-original-height="292" data-original-width="800" height="145" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/sysprep-shutdown.png" title="shiutdown" width="400" /></a><br /></li>
<li>The next entry in the log was at <b>04:55</b> on the <b>17th of March</b> - which was just under <br /><b>8 hours</b> <b><u>before</u></b> I even started my EC2 instance!!<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/startup-in-pool.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="start_in_pool" border="0" data-original-height="276" data-original-width="800" height="137" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/startup-in-pool.png" title="start_in_pool" width="400" /></a><br /><br /><span id="goog_1079751587"></span><span id="goog_1079751588"></span></li>
<li>The hostname was changed at 04:56<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/rename-generalize.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="rename_generalize" border="0" data-original-height="275" data-original-width="800" height="137" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/rename-generalize.png" title="rename_generalize" width="400" /></a><br /></li>
<li>And then restarted at 04:57<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/rebbot-generalize.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="reboot_generalize" border="0" data-original-height="304" data-original-width="800" height="151" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/rebbot-generalize.png" title="reboot_generalize" width="400" /></a><br /></li>
<li>After the instance came back up - it was shutdown once more and returned to the pool at <b>04:59</b>.<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/shutdown-return-to-pool.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="shutdown-return-to-pool." border="0" data-original-height="346" data-original-width="800" height="172" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/shutdown-return-to-pool.png" title="shutdown-return-to-pool." width="400" /></a><br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/shutdown-return-to-pool-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="shutdown-return-to-pool2" border="0" data-original-height="330" data-original-width="800" height="165" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/shutdown-return-to-pool-2.png" title="shutdown-return-to-pool2" width="400" /></a><br /></li>
<li>The instance was powered on again (from the pool) at <b>11:47:11</b> (30 seconds after my request)<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/poweron-from-pool.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="power-on-from-pool" border="0" data-original-height="282" data-original-width="800" height="140" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/poweron-from-pool.png" title="power-on-from-pool" width="400" /></a><br /><br /><b><i>More about what this whole process entails further on down the post.<br /><br /></i></b></li>
<li>The secret-sauce service then changes the ownership on the instance - and does some magic to manipulate the metadata on the instance - to allow the user to decrypt the credentials with their unique key and allow them to log in.<br /><br /><a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/ssm-agent.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img alt="ssm_agent" border="0" data-original-height="229" data-original-width="800" height="113" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/ssm-agent.png" title="ssm_agent" width="400" /></a><br /></li>
<li><span style="color: blue;"><b>The user now has access to their instance</b>.</span></li>
</ol>
<br />
I wanted to go a bit more into the entity that I named the <i>"Instance Pool"</i>. Here I assume that there is a whole process in the background that does the following (and where the secret sauce really lies).<br />
<br />
This is is how I would assume how the flow would be:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190307_aws_ec2_secret/ec2_sauce_diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="677" data-original-width="800" height="541" src="https://maishsk.com/blog/images/20190307_aws_ec2_secret/ec2_sauce_diagram.png" width="640" /></a></div>
<br />
There are two different entities here at work - one is the AWS backbone service (in orange) and the User/Customer (in blue). Both of the sequences work in parallel and also independent of each other.<br /><br />
<ul>
<li>AWS pre-warm a number of Windows instances in what I named the <i>"Instance pool"</i>. They preemptively spin up instances in the background based on their predictions and the usage patterns in each region. I assume that these instances are constantly spun up and down on a regular basis - many times a day.<br /></li>
<li>A notification is received that a customer requested an instance from a specific AMI (in a specific region, in a specific AZ and from a specific instance type - because all of these have to match the customers request).<br /></li>
<li>The request is matched to an instance that is in the pool (by AMI, region, AZ, instance type)<br /></li>
<li>The instance is then powered on (with the correct modifications of the instance flavor - and disk configuration)<br /></li>
<li>The backend then goes and makes the necessary modifications<br /></li>
<ul>
<li>ENI allocation (correct subnet + VPC)</li>
<li>Account association for the instance</li>
<li>Private key allocation</li>
<li>User-data script (if supplied) </li>
<li>Password rotation</li>
<li>etc.. etc..</li>
</ul>
</ul>
I know that this sounds simple and straight forward - but the amount of work that goes into this <i>"Instance Pool"</i> is probably something that we cannot fathom. The predictive analysis that is needed here to understand how many instances should be provisioned, in which region, in which AZ - is where AWS shines and have been doing so for a significant amount of time.<br />
<br />
This also makes perfect sense that when you deploy a custom Windows AMI - this process will not work anymore, because this is a custom AMI and therefore the provisioning time is significantly longer.<br />
<br />
And all of this is done why?<br />
<br />
To allow you to shave off a number of <b>minutes / seconds</b> wait time to get access to your Windows instance. This is what it means to provide an exceptional service to you the customer and make sure that the experience you have is the best one possible.<br />
<br />
I started to think - could this possibly be the way that AWS provisions Linux instances as well?<br />
<br />
Based on how I understand the cloud and how Linux works (and some digging in the instance logs) - this is not needed, because the image sizes are much smaller and bootup times are a lot shorter as well, so it seems to me that this <i>"Instance Pool" </i>is only used for Windows Operating systems, and only for AMI's that are owned by AWS.<br /><br />Amazing what you can find from some digging - isn't it?<br />
<br />
Please feel free to share this post and share your feedback on Twitter - <a href="https://twitter.com/maishsk" target="_blank">@maishsk</a><br />
<br />Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-11386252212046648302019-03-11T10:00:00.000+02:002019-03-11T10:16:34.006+02:00The Anatomy of an AWS Key Leak to a Public Code RepositoryMany of us working with any cloud provider know that you should never ever commit access keys to a public github repo. Some really bad things can happen if you do.<br />
<br />
AWS (and I assume all the cloud providers have their equivalent) publish their own best practices about how you should <a href="https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html" target="_blank">manage access keys</a>.<br />
<br />
One of the items mentioned there - is never to commit your credentials into your source code!!<br />
<div>
<br /></div>
<div>
Let me show you a real case that happened last week. </div>
<div>
(of course all identifiable information has been redacted - except for the specific Access key that was used - and of course it has been disabled)</div>
<div>
<br /></div>
<div>
Someone committed an access key to a public github repository. </div>
<div>
<br />
Here is the commit message </div>
<div>
<div>
<br /></div>
<div>
<div>
commit xxxxxxxx26ff48a83d1154xxxxxxxxxxxxa802</div>
<div>
Author: SomePerson <someone@some_email.com></div>
<div>
Date: Mon Mar 4 10:31:04 2019 +0200</div>
<div>
<br />
<div style="-webkit-text-stroke-width: 0px; font-family: "Times New Roman"; font-size: medium; font-variant-caps: normal; font-variant-ligatures: normal; letter-spacing: normal; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div style="margin: 0px;">
<b><i><span style="color: blue;">--- (All events will be counted from this point) ---</span></i></b></div>
</div>
</div>
<div>
<b><span style="color: red;"><br /></span></b>
<b><span style="color: red;">55 seconds</span></b> later - I received an email from AWS (<b>T+55s</b>)</div>
<div>
<br /></div>
<div>
<div>
From: "Amazon Web Services, Inc." <no-reply-aws@amazon.com></div>
<div>
To: john@doe.com</div>
<div>
Subject: Action Required: Your AWS account xxxxxxxxxxxx is compromised</div>
<div>
Date: Mon, 4 Mar 2019 08:31:59 +0000</div>
</div>
<div>
</div>
</div>
<div>
<br /></div>
<div>
1 second later (<b>T+56s</b>) AWS had already opened a support ticket about incident<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190305_anatomy_key_leak/compromise.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="156" data-original-width="399" height="249" src="https://maishsk.com/blog/images/20190305_anatomy_key_leak/compromise.png" width="640" /></a></div>
<br /></div>
<div>
<br />
<br />
Just over 1 minute later (<b>T+2:02m</b>) someone tried to use the key - but since the IAM role attached to the user (and its exposed key) did not have the permissions required - the attempt failed!!<br />
<br />
(This is why you should make sure you only give the minimum required permissions for a specific task and not the kitchen sink..)<br />
<br />
Here is the access attempt that was logged in <a href="https://aws.amazon.com/cloudtrail/" target="_blank">Cloudtrail</a><br />
<br /></div>
<script src="https://gist.github.com/maishsk/f79d27571a4b6500f184eec7e3f9c418.js"></script><br /></div>
<br />
<br />
Here is where I went in and disabled the access key (<b>T+5:58m</b>)<br />
<br />
<script src="https://gist.github.com/maishsk/39126ce798a64f55f184e0846f0ae57f.js"></script><br />
<br />
Here was the notification message I received from GuardDuty which was enabled on the account (<b>T+24:58m</b>)<br />
<br />
Date: Mon, 4 Mar 2019 08:56:02 +0000<br />
From: AWS Notifications <no-reply@sns.amazonaws.com><br />
To: john@doe.com<br />
<div>
</div>
Message-ID: <0100016947eac6b1-7b5de111-502d-4988-8077-ae4fe58a87c9-000000@email.amazonses.com><br />
Subject: AWS Notification Message<br />
<br />
<script src="https://gist.github.com/maishsk/3e44a17be2134df0ba8aefc7f4682d09.js"></script><br />
<br />
<h2>
Points for Consideration</h2>
<div>
There are a few things I would like to point out regarding the incident above (which we in the categorized to one of a low severity). </div>
<div>
<br /></div>
<div>
<ol>
<li>As you can see above the first thing that the attacker tried to do was to run a list keys. That would usually be the first thing someone would try - to try and understand which users are available in the system (assuming that the user has the permission to perform that action)<br />
<br />
You can read more about how a potential hacker would exploit this in <a href="https://rhinosecuritylabs.com/aws/" target="_blank">this series of posts</a>.<br /><br />
</li>
<li>I assume since the attacker saw that they do not have enough permissions - they decided this was not a worthy enough target to continue to try the exploit. Why waste the time if you are going to have to work really hard to get what you want. That is why we only saw a single attempt to use the key.<br />
<br />
If I was the hacker - I would just wait for the next compromised key and try again.<br /><br />
</li>
<li>The reason this attack was not successful - was because the role attached to the User (and its access keys) was built in such a way that they did not have permissions to do anything in IAM.<br /><br /><b>This was by design</b>. The concept of least privilege is so important - and 10 times more when you are working in the cloud - that you should implement it - <b>in every part of your design and infrastructure</b>.<br /><br /></li>
<li>AWS responded <b>extremely fast</b> - that is due to them (I assume) scraping the API of all public github commits (<a href="https://github.com/Tabcorp/node-github-credential-scraper" target="_blank">for example</a>). It could have been that I was just in time for a cycle - but based on my past experience - the response time is usually within a minute. It would be great if they could share how they do this and handle the huge amount of events that flow through these feeds.<br />
<br />
They still have to match up the exact compromised key to the account, and kick off the automatic process (email+ticket). All of this was done in less than 60 seconds. <br /><br />I am impressed (as should we all be).<br /><br />
</li>
<li>One thing I do not understand is that why AWS would not immediately disable the key. The business implications of having a key out in a public repo - are so severe - and the use case that would require a key in the open - is something that I cannot fathom as being a valid scenario. If AWS already find a compromised key, know which account it belongs to, and kick off a process - then why not already disable the key in the process??<br />
<br />
The amount of time and work that AWS would have to invest (in support tickets and calls) working with a customer to clean up the account, forfeit the charges incurred because of the leak - are above and beyond anything they would incur by automatically disabling the key in the first place.<br />
<br />
AWS has started to take a stance on some security features - by disabling thing by default (for example - public S3 buckets) to protect their customers from causing harm to themselves.<br />
<br />
I for one would welcome this change with open arms!<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
Hi, Maish. I have forwarded your feature request on to our service team to be reviewed. Thanks for the input! 😊 ^CC</div>
— AWS Support (@AWSSupport) <a href="https://twitter.com/AWSSupport/status/1102527225256206336?ref_src=twsrc%5Etfw">March 4, 2019</a></blockquote>
<script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><br />
<br />
<br />
</li>
<li>It took me over 5 minutes to actually act on the exposed credential - in 5 minutes, a malicious actor can do some real and serious damage to your AWS account.<br /><br /></li>
<li><a href="https://aws.amazon.com/guardduty/" target="_blank">GuardDuty</a> - was slow, but it obvious why this was the case. It takes about 15 minutes until the event is delivered to <a href="https://aws.amazon.com/cloudtrail/" target="_blank">CloudTrail</a> - and GuardDuty then has to analyze based on previous behavior. So this product should not be used for prevention - but rather - for forensic analysis after the fact. There is no real way to identify this data on your own and analyze against your baseline for behavior - so this product is in my honest opinion still very valuable.<br /><br /></li>
<li>How does one stop this from happening?<br /><br />There are a number of ways to tackle this question.<br /><br />In my honest opinion, it is mainly <b>raising awareness</b> - from the bottom all the way to the top. The same way people know that if you leave your credit card on the floor - there is a very good chance it will be abused. Drill this into people from day 1 and hopefully it will not happen again.<br /><br />There are tools that are out there - that you can use as part of your workflow - such as <br /><a href="https://github.com/awslabs/git-secrets" target="_blank">git-secrets</a> that prevent such incidents from even happening - but you would have to assure that every single person, and every single computer they ever work on - would have this installed - which is a much bigger problem to solve.<br /><br />Install your own tools to monitor your repositories - or use a service such as <a href="https://www.gitguardian.com/" target="_blank">GitGuardian</a> that does this for you (not only for AWS - but other credentials as well). </li>
</ol>
<div>
As always please feel free to share this post and leave your feedback on on Twitter <a href="https://twitter.com/maishsk" target="_blank">@maishsk</a></div>
</div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-26018428559078623572019-03-06T10:00:00.000+02:002019-03-06T10:00:07.549+02:00My awesome-podcasts List<span style="font-family: inherit;">I have a decent commute every day back and forth to work and I have come to enjoy listening to a number of podcasts throughout the week.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">I will try and keep the list up to date - <a href="https://github.com/maishsk/awesome-podcasts" target="_blank">here</a></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">As of today - this is my current list of podcasts</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<h3>
Grumpy Old Geeks</h3>
Two old farts (like me) that bitch about tech, and how ridiculous we have all become - <a href="http://gog.show/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#aws-podcast"></a><br />
<div>
<h3>
</h3>
<h3>
AWS Podcast</h3>
A weekly show about what is happening in the world of AWS - <a href="https://aws.amazon.com/podcasts/aws-podcast/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#the-cloud-pod"></a></div>
<div>
<h3>
</h3>
<h3>
The Cloud Pod</h3>
A podcast about what is going on the cloud - <a href="https://www.thecloudpod.net/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#screaming-in-the-cloud"></a></div>
<div>
<h3>
</h3>
<h3>
Screaming in the Cloud</h3>
Conversations about the cloud with people - <a href="https://www.screaminginthecloud.com/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#podctl"></a></div>
<div>
<h3>
</h3>
<h3>
PodCTL</h3>
Podcast about Kubernetes - with a RedHat focus - <a href="https://podctl.com/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#the-cloudcast"></a></div>
<div>
<h3>
</h3>
<h3>
The Cloudcast</h3>
Podcast about all things Cloud - <a href="http://www.thecloudcast.net/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#cloudtalk-hebrew"></a></div>
<div>
<h3>
</h3>
<h3>
Cloudtalk (Hebrew)</h3>
Hebrew Podcast about the world of cloud - <a href="https://cloudtalk.co.il/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#the-tony-robbins-podcast"></a></div>
<div>
<h3>
</h3>
<h3>
The Tony Robbins Podcast</h3>
Inspirational talk with Tony Robbins - <a href="https://www.tonyrobbins.com/podcasts/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#datanauts-packet-pushers"></a></div>
<div>
<h3>
</h3>
<h3>
Datanauts (Packet Pushers)</h3>
Podcast about tech, cloud and all things nice - <a href="https://packetpushers.net/datanauts-podcast/">Link</a></div>
<div>
<h3>
</h3>
<h3>
Rural Emergency Medicine Podcast</h3>
A Podcast about emergency medicine - <a href="https://ruralem.org/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#speaking-in-tech"></a></div>
<div>
<h3>
</h3>
<h3>
Speaking in Tech</h3>
Podcast about things happening in the tech world - <a href="http://speakingintech.com/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#the-secure-developer"></a></div>
<div>
<h3>
</h3>
<h3>
The Secure Developer</h3>
Security focused Podcast - <a href="https://www.heavybit.com/library/podcasts/the-secure-developer/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#the-full-stack-journey"></a></div>
<div>
<h3>
</h3>
<h3>
The Full Stack Journey</h3>
Interviews with people that have made a change in their technical career - <a href="http://fullstackjourney.com/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#to-be-continuous"></a></div>
<div>
<h3>
</h3>
<h3>
To Be Continuous</h3>
DevOps focused podcast - <a href="https://www.heavybit.com/library/podcasts/to-be-continuous/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#the-microsoft-cloud-show"></a></div>
<div>
<h3>
</h3>
<h3>
The Microsoft Cloud Show</h3>
A Microsoft focused cloud podcast - <a href="http://www.microsoftcloudshow.com/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#emergency-medicine-cases"></a></div>
<div>
<h3>
</h3>
<h3>
Emergency Medicine Cases</h3>
A podcast about emergency medicine - <a href="https://emergencymedicinecases.com/">Link</a><br />
<a href="https://github.com/maishsk/awesome-podcasts#techtalk"></a></div>
<div>
<h3>
</h3>
<h3>
Techtalk</h3>
A podcast in Hebrew about the cloud and tech - <a href="https://techtalk.co.il/">Link</a></div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-85104316405854411752019-03-04T09:00:00.000+02:002019-03-05T10:37:50.052+02:00AMI has 3 Syllables. A.M.I. #AWS<b><i><span style="font-size: large;">Just to make this clear </span><br />
<br />
(before someone get's the wrong idea...) <br />
<br />
<span style="font-size: large;">This 100% fun. Humor. </span><br />
<br />
<span style="font-size: large;">Not religion. Not a mission. </span><br />
<br />
<span style="font-size: large;">Just having some fun at the expense of AWS.. </span></i></b><span style="font-size: large;"><br />
</span> <span style="font-size: large;"><br />
</span>If you follow me on <a href="https://twitter.com/maishsk" target="_blank">Twitter</a> (and if you don't - your loss..) then you will know that I am one of many that are on a crusade.<br />
<div>
<br /></div>
A crusade to right a wrong.<br />
<br />
A wrong that some who work in a company called Amazon Web Services (a.k.a. AWS) have tried to indoctrinate the world with a lie, something that is just plain wrong.<br />
<br />
And the crusade about I speak - is the religious debate about how you pronounce AMI <br />
(Amazon Machine Image)<br />
<br />
You will find many references to this over the past few years:<br />
<br />
<a href="https://twitter.com/seldo/status/862704029155155968" target="_blank">Twitter Thread</a><br />
<a href="https://read.acloud.guru/last-year-in-aws-841a960ca60d" target="_blank">Last Year in AWS</a><br />
<a href="https://snarkive.lastweekinaws.com/issue_35-_An_Ounce_of_PreInvention.html" target="_blank">Last week in AWS - Issue #35</a><br />
<a href="https://twitter.com/somecloudguy/status/1067834997145399296" target="_blank">Another Twitter thread</a><br />
<a href="https://twitter.com/seldo/status/862704029155155968" target="_blank">And another</a><br />
<a href="https://twitter.com/QuinnyPig/status/1098511266002264066" target="_blank">And yet another</a><br />
<a href="https://medium.com/@abbyfuller/not-containers-101-bringing-your-own-ami-or-configuring-on-the-fly-8f66ca7d7eef" target="_blank">Abby Fuller's post</a><br />
<a href="https://s3.amazonaws.com/pronounce-ami/AMI.mp3" target="_blank">This recording</a><br />
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
A number that rhymes with pee. Definitely *not* a number that rhymes with poo</div>
— Maish Saidel-Keesing (@maishsk) <a href="https://twitter.com/maishsk/status/1098559174093479936?ref_src=twsrc%5Etfw" target="_blank">February 21, 2019</a></blockquote>
<br />
And of course the one and only <a href="https://twitter.com/quinnypig" target="_blank">Corey Quinn</a><br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="und">
<a href="https://t.co/c4sjfZ7KWj" target="_blank">pic.twitter.com/c4sjfZ7KWj</a></div>
— Ant Stanley (@IamStan) <a href="https://twitter.com/IamStan/status/1098512429158486016?ref_src=twsrc%5Etfw" target="_blank">February 21, 2019</a></blockquote>
<script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><br />
I decided that I cannot idly stand by and let this injustice continue.<br />
<br />
I took a step. I took a stand (and I started with a donation of 2 Euro for the domain name)<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://ami-has-3-syllables.online/" target="_blank"><img alt="AMI has 3 syllables" border="0" data-original-height="404" data-original-width="800" height="321" src="https://maishsk.com/blog/images/20190228-ami-has-3-syllables/ami-has-3-syllables-splash.png" title="AMI has 3 syllables" width="640" /></a></div>
<br />
<br />
<a href="http://ami-has-3-syllables.online/" target="_blank"><span style="font-size: large;">http://ami-has-3-syllables.online</span></a><br />
<br />
And in my ramblings back and forth with Corey - he enlightened me to the following fact<br />
(which is so unbelievably true)<br />
<br />
I managed to release the perfect AWS product (on a budget of $2 - really proud of myself)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190228-ami-has-3-syllables/perfect-launch.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Perfect launch" border="0" data-original-height="54" data-original-width="800" height="42" src="https://maishsk.com/blog/images/20190228-ami-has-3-syllables/perfect-launch.png" title="Perfect launch" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
1. People have no idea how to use it<br />
2. It has a stupid name that you cannot remember<br />
3. The graphics suck... (sorry I have not done HTML/CSS in - I do not know how long)<br />
4. No TLS<br />
<br />
So in the spirit of this perfect release - I thought about how this would work with a real AWS product launch, and therefore I will iterate over time to improve the product.<br />
<br />
Here is the plan (in the reverse order from above)..<br />
<ol>
<li>Implement TLS ( I actually could do that today - but I am going leave like this for the launch in the spirit of a new product)<br /><b><span style="color: #38761d;">** Edit ** - Implemented 05 March, 2019</span></b></li>
<li>Fix up the graphics<br />
(Here I am going to crowdsource and look to you all - and if anyone wants to step up and improve my crappy artwork - reach out - I would be happy to get some help.<br />
Feel free to reach out on to me <a href="https://twitter.com/maishsk" target="_blank">@maishsk</a>)</li>
<li>Plug the name to death - until people remember the name - in their sleep<br />
For this - say hello to <a href="https://twitter.com/3_syllables">@3_syllables</a> (feel free to follow)</li>
<li>Implement a bot that will interact with people who don't know how to pronounce A.M.I.<br />
(and maybe add some statistical functionality on the bot's activity to the site)<br />
</li>
</ol>
<div>
<a href="https://www.azlyrics.com/lyrics/lesmiserablescast/doyouhearthepeoplesing.html" target="_blank">Will you join in our crusade</a>?</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/SWzgrmOLxp8/0.jpg" frameborder="0" height="315" src="https://www.youtube.com/embed/SWzgrmOLxp8?feature=player_embedded" width="560"></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Feel free to share - and leave me your thoughts on <a href="https://twitter.com/maishsk" target="_blank">Twitter</a></div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-54838174380652946082019-02-25T10:00:00.000+02:002019-03-03T09:15:28.165+02:00Goodbye Docker and Thanks for all the FishBack in <a href="https://twitter.com/maishsk/status/1019115484673970176">July 2018</a>, I started to write a blog post about the upcoming death of Docker as a company (and also perhaps as a technology) but I never got round to completing and publishing the post. It is time to actually get that post out.<br />
<br />
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
OK .. Time to share my thoughts on the soon to be death of <a href="https://twitter.com/hashtag/docker?src=hash&ref_src=twsrc%5Etfw">#docker</a></div>
— Maish Saidel-Keesing (☁️🚀☁️) (@maishsk) <a href="https://twitter.com/maishsk/status/1019115484673970176?ref_src=twsrc%5Etfw">July 17, 2018</a></blockquote>
<script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><br />
<br />
<br />
<i><b>So here you go....</b></i><br />
<i><b><br />
</b></i> <br />
<div>
Of course Docker is still here, and of course everyone is still using Docker and will continue to do so the near and foreseeable future (how far that foreseeable future is - is yet to be determined). The reason I chose this title for the blogpost is because, in my humble opinion the days for Docker as a company are numbered and maybe also a technology as well. If would indulge me with a few minutes of your time - I will share with you the basis for my thoughts.<br />
<br />
A number of years ago - Docker was the company that changed the world - and we can safely say - is still changing the world today. Containers and the technology behind containers has been around for many years, long before the word docker was even thought of, even turned into a verb (“Dockerize all the things”), but Docker was the company that enabled the masses to consume the technology of containers, in a easy and simple fashion. Most technology companies (or at least companies that consider themselves to be a modern tech company) will be using Docker or containers as part of their product or their pipeline - because it makes so much sense and brings so much benefit to whole process.<br />
<br />
Over the past 12-24 months, people are coming to the realization that docker has run its course and as a technology is not going to be able to provide additional value to what they have today - and have decided to start to look elsewhere for that extra edge.<br />
<br />
Kubernetes has won the container orchestration war, I don’t think that anyone can deny that fact. <a href="https://www.servethehome.com/kubernetes-wins-docker-acquiesces-adopts-kubernetes/">Docker itself has adopted Kubernetes</a>. There will always be niche players that have specific use cases for Docker Swarm, Mesos, Marathon, Nomad - but the de-facto standard is Kubernetes. All 3 big cloud providers, now have a managed Kubernetes solution that they offer to their customers (and as a result will <a href="https://azure.microsoft.com/en-us/updates/azure-container-service-will-retire-on-january-31-2020/">eventually sunset</a> their own home-made solutions that they built over the years - because there can be only one). Everyone is building more services and providing more solutions, to bring in more customers, increase their revenue.<br />
<br />
Story is done. Nothing to see here. Next shiny thing please..<br />
<br />
At the moment, Kubernetes uses docker as the underlying container engine. I think that the Kubernetes community understood that Docker as a container runtime (and I use this term specifically) was the ultimate solution to get a product out of the gate as soon as possible. They also (wisely) understood quite early on they needed to have the option of switching out that container runtime - and allowing the consumers of Kubernetes to make a choice. <br />
<br />
The <a href="http://www.opencontainers.org/">Open Container Initiative</a> - brought with it the <a href="https://github.com/opencontainers/runtime-spec">Runtime Spec</a> - which opened the door to allow us all to use something else besides docker as the runtime. And they are growing - <a href="https://github.com/opencontainers/runtime-spec/blob/master/implementations.md">steadily</a>. Docker is no longer the only runtime that is being used. Their is a growth in the community - that are slowly sharing the knowledge of how use something else besides Docker. Kelsey Hightower - has updated his <a href="https://github.com/kelseyhightower/kubernetes-the-hard-way">Kubernetes the hard way</a> (amazing work - honestly) over the years from <a href="https://github.com/kubernetes-incubator/cri-o)%20v1.0.0-beta.0">CRI-O</a> to <a href="https://github.com/containerd/containerd">containerd</a> to <a href="https://github.com/google/gvisor">gvisor</a>. All the cool kids on the block are no longer using docker as the underlying runtime. There are many other options out there today <a href="https://github.com/clearcontainers/runtime">clearcontainers</a>, <a href="https://katacontainers.io/">katacontainers</a> and the list is continuously growing.<br />
<br /></div>
<div>
Most people (including myself) do not have enough knowledge and expertise of how to swap out the runtime to what ever they would like and usually just go with the default out of the box. When people understand that they can easily make the choice to swap out the container runtime, and the knowledge is out there and easily and readily available, I do not think there is any reason for us to user docker any more and therefore Docker as a technology and as a company will slowly vanish. The other container runtimes that are coming out will be faster, more secure, smarter, feature rich (some of them already are) compared to what Docker has to offer. If you have a better, smarter, more secure product - why would people continue to use technology that no longer suits their ever increasing needs? <br />
<br />
For Docker - to avert this outcome - I would advise to invest as much energy as possible - into creating the best of breed runtime for any workload - so that docker remains the de-facto standard that everyone uses. The problem with this statement - is that there no money in a container runtime. Docker never made money on their runtime, they looked for their revenue on the enterprise features above and on top the container runtime. How they are going to solve this problem - is beyond me and the scope of this post.<br />
<br />
The docker community has been steadily declining, the popularity of the events has been declining, the number of new features, announcements - is on the decline and has been on the decline for the past year or two. <br />
<br />
Someone told me a while back - that speaking bad about things or giving bad news is usually very easy. We can easily say that this is wrong, this is no useful, this should change. But without providing a positive twist on something - you become the “doom and gloom”. The “grim reaper”. Don’t be that person.<br />
<br />
I would like to heed their advice, and with that add something about - what that means for you today. You should start investing in understanding how these other runtimes can help you, where they fit, increase your knowledge and expertise - so that you can prepare for this and not be surprised when everyone else stops using docker and you find yourself having to rush into adapting all your infrastructure. I think it is inevitable.<br />
<br />
<b><i>That was the post I wanted to write 8 months ago...</i></b><br />
<br />
What triggered me to finish this post today was a post from <a href="https://twitter.com/fatherlinux">Scott Mccarty</a> - about the upcoming RHEL 8 beta - <a href="https://www.redhat.com/en/blog/red-hat-enterprise-linux-8-beta-new-set-container-tools">Enterprise Linux 8 Beta: A new set of container tools</a> - and my tweet that followed<br />
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
This will officially be the end of docker... It has been a while in the making <a href="https://t.co/kBTjpXAzAL">https://t.co/kBTjpXAzAL</a></div>
— Maish Saidel-Keesing (☁️🚀☁️) (@maishsk) <a href="https://twitter.com/maishsk/status/1098295411117309952?ref_src=twsrc%5Etfw">February 20, 2019</a></blockquote>
Lo and behold - no more docker package available in RHEL 8.<br />
<blockquote class="tr_bq">
If you’re a container veteran, you may have developed a habit of tailoring your systems by installing the “docker” package. On your brand new RHEL 8 Beta system, the first thing you’ll likely do is go to your old friend yum. You’ll try to install the docker package, <b>but to no avail</b>. If you are crafty, next, you’ll search and find this package:</blockquote>
<blockquote class="tr_bq">
<i>podman-docker.noarch : "package to Emulate Docker CLI using podman."</i></blockquote>
<blockquote class="tr_bq">
What is this Podman we speak of? The <u><b><i>docker package is replaced</i></b></u> by the Container Tools module, which consists of Podman, Buildah, Skopeo and several other tidbits. There are a lot of new names packed into that sentence so, let’s break them down.</blockquote>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190224-goodbye-docker/podman.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="203" data-original-width="800" height="161" src="https://maishsk.com/blog/images/20190224-goodbye-docker/podman.png" width="640" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div style="text-align: center;">
<span style="font-size: x-small;">(Source - Tutorial - Doug Tidwell (https://youtu.be/bJDI_QuXeCE)</span></div>
<div style="text-align: center;">
<br /></div>
I think a picture is worth more than a thousand words..<br />
<br />
Please feel free to share this post and share your feedback with me on Twitter (<a href="https://twitter.com/maishsk" target="_blank">@maishsk</a>)</div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-36296036939556920072019-02-22T10:00:00.000+02:002019-02-22T10:00:01.344+02:00Separate VPC's can do More Harm Than Good<div class="separator" style="clear: both; text-align: left;">
I have come across this a number of times of the past couple of months. Environments that were born in the datacenter, have grown in the datacenter - in short people who are used to certain (shall we say - ‘legacy’) deployments, and they they are in the midst of an attempt to mirror the same architecture when moving to the cloud.</div>
<br />
I remember in my old days that our server farm had a separate network segment (sometimes even more than one) when I was using physical servers, (while I write this - I actually think it has been about 4 years since I actually touched a physical server, or plugged a cable/disk/device into a physical server) for our Domain controllers, Applications servers, and users had their own network segments that were dedicated only to laptops and desktops.<br />
<br />
In the physical/on-prem world - this made sense - at the time - because what usually happened was the dedicated networking team that managed your infrastructure used access lists on the physical network switches to control which networks could go where.<br />
<br />
Fast forward to the cloud.<br />
<br />
There are people which equate VPC’s with Networks (even though it makes more sense to equate subnets to networks - but that is besides the point) - and think that segregating different kinds of workloads into multiple VPC’s will give you better security. <br />
<br />
Let me give you a real scenario that I was presented with not too long ago (details of course have been changed to protect the innocent … ) <br />
<br />
A three tier application. Database, Application and a frontend. And the requirement that was laid down from the security team was that each of the layers must reside in the their own VPC. Think about that for a minute. Three VPC’s that would be peered to ensure connectivity between them (because of course the 2/3 layers needed to communicate with each other - Database - application and application to frontend). When I asked what was the reason for separating the three different layers in that way, the answer was, “Security. If for example one of the layer was compromised - it would be much harder to make a lateral move to another VPC and compromise the rest.” <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/20190221_VPC_Security/vpc_segmentation.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="553" data-original-width="800" height="441" src="https://maishsk.com/blog/images/20190221_VPC_Security/vpc_segmentation.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
So what is lateral movement? I know that there is no such a thing as a 100% secure environment. There will always be hackers, there will always be ways around any counter measures we try and put in place, and we can only protect against what we know and not against what we do not. The concept of lateral movement is one, of compromising a credential on one system and with that credential moving to another system. For example - compromising a Domain admin credential on an employees laptop - and with that credential moving into an elevated system (for example a domain controller) and compromising the system even further. <br />
<br />
So how would this work out in the scenario above. If someone would compromise the frontend - the only thing they would be able to connect to would be the application layer - the frontend - does not have any direct interaction with the database layer at all, do your data would be safe. There would be a peering connection between the Frontend VPC an the Application VPC - with the appropriate routing in place to allow traffic flow between the relevant instances, and another peer between the Application VPC and the Database VPC - with the appropriate routing in place as well.<br />
<br />
What they did not understand - is that if the application layer was compromised - then that layer does have direct connectivity with the data layer - and therefore could access all the data. <br />
<br />
Segregating the layers into different VPC’s would not really help here.<br />
<br />
And honestly - this is a risk that you take - which is why the attack surface you have - exposed on your frontend - should be as small as possible - and secure as possible.<br />
<br />
But I came back to the infosec team and told them - what if I would provide the same security and segregation that you were trying to achieve but without the need of separate VPC’s ? <br />
<br />
I would create a Single VPC - with three subnets and three security groups, Frontend, Application and Database. Instances in the frontend security group would only be allowed to communicate with the instances in the application security group on a specific port (and vice-versa) and the instances in the application security group would only be allow to communicate with the instances in the database security group (and vice-versa). <br />
<br />
<br />
The traffic would be locked down to the specific flow of traffic and instances would not be able to communicate out of their security boundary. <br />
<br />
<br />
<a href="https://maishsk.com/blog/images/20190221_VPC_Security/security_group_segmenation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="670" data-original-width="593" height="670" src="https://maishsk.com/blog/images/20190221_VPC_Security/security_group_segmenation.png" width="593" /></a><br />
<br />
As a side note - this could have also been accomplished by configuring very specific routes between the instances that needed to communicate between the VPC’s, but it does not scale to an environment larger than a handful of instances. Either you need to ensure that the IP addresses in a manual fashion, or keep on adding multiple routes in the route tables.<br />
<br />
It goes without saying that if someone managed to compromise the frontend, and somehow managed through the application port to gain control into the application layer - they could gain access (in theory) to the data in the data layer. <br />
<br />
Which is exactly what happened in the same scenario with 3 separate VPC’s. No less secure - no more.<br />
<br />
But what changed??<br />
<div>
<br /></div>
<div>
The operational overhead of maintaining 3 VPC’s for no specific reason was removed.<br />
<br />
This includes:</div>
<div>
<br /></div>
<div>
<ul>
<li>VPC Peering (which has a limit)</li>
<li>Route tables (which has a limit)</li>
<li>Cost reduction</li>
</ul>
<br />
I could even take this a bit further and say I do not even need different subnets for this purpose - I could actually even put all the instances in a single subnet and use the same mechanism of security groups to lock down the communication. Which is true. And in an ideal world - I probably would have done so - but in this case - it was a bit too revolutionary to already have made the step of going to a single VPC - and to go to a single subnet - was pushing the limit - maybe just a bit too far. Sometimes you need to take small victories and rejoice and not go in for the jugular. <br />
<br />
I would opt into option of using separate VPC’s in some cases such as:<br />
<br />
<ul>
<li>Different owners or accounts where you cannot ensure the security of one of the sides.</li>
<li>When they are completely different systems - such as a CI system and production instances</li>
<li>A number of other different scenarios</li>
</ul>
<div>
<br /></div>
The bottom line of this post is - traditional datacenter architecture - does not have to be cloned into your cloud. There are cases where it does make sense - but there are cases where you can use cloud-native security measures - which will simplify your deployments immensely and allow you to concentrate as always on the most important thing. Bringing value to your customers - and not investing your time into the management and maintenance of the underlying infrastructure.<br />
<br />
<br />
Please feel free contact me on Twitter (<a href="https://twitter.com/maishsk">@maishsk</a>) if you have any thoughts or comments.</div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-33152033399412955362019-01-30T00:09:00.000+02:002019-01-30T12:51:52.271+02:00Empires are not built in a day (but they also do not last forever) I am currently on vacation in Rome (my first time) and during this trip I came to a number of realizations that I would like to share with you.<br />
<br />
I went to the Colosseum today - and I have to say I was in awe. The structure is magnificent (even if the remains are only part of of the original structure in all its glory). As I progressed throughout the day - I came to the following realizations.<br />
<br />
(.. and of course how they tie into our tech world today)<br />
<br />
<h3>
Acquire vs. Train</h3>
<div>
<br />
Throughout ancient history - all the great empires (or what were once considered as such) were barbarians. They left legacies that remain to this day - but none of them were earned honestly. Most of the great wonders of the worlds - from the pyramids to the Colosseum to the great wall of China - these were all built with slave labor. The Romans conquered the world, enslaved almost every country they touched - used them to build an empire. I think it is safe to say this how the world used to work. Today, this would not be acceptable. Slavery and taking advantage of the other is not correct. </div>
<div>
<br /></div>
<div>
The knowledge was there, the brains were there, but they needed working hands get the shit done. </div>
<div>
That is why people outsource development resources to places where labor is cheap (India for example) but leave the brains at home and only let the 'workers' churn out the hard stuff. </div>
<div>
<br /></div>
<div>
There are several problems with this - and we are seeing this today in many walks of life. Some companies understand that even though the labor is cheaper, the quality and speed with which the work they wished to complete - is not what they expect. In the olden days would be able to terrorize your slaves into working to their deaths to provide what you want. This happened in ancient Egypt, in ancient Rome, pretty much everywhere. But that does not and cannot happen today. So we do one of two things. Instead of working the people to their death we provide incentives to produce more, be it higher salaries, better conditions, bonuses - hoping that this will encourage (or should I rather say force) people to work harder. The other option is - we compromise on quality - or on delivery times - which either pisses off our customers because we are late, or pisses them off - because the product is not as good as we promised.<br />
<br />
It is obvious though - that the easiest way for us to produce - is not by training the talent from the ground up - but rather let someone else invest that time and effort - and when we have the opportunity, swoop in (in the olden days conquer) and reap the benefits of someone else's work. </div>
<div>
<br /></div>
<div>
In today's world we see this with most big companies acquiring smaller ones. Growth by Acquisition. Cisco has <a href="https://en.wikipedia.org/wiki/List_of_acquisitions_by_Cisco_Systems" target="_blank">built its empire</a> over the years in this way. You can't build an amazing Wireless product - <a href="https://newsroom.cisco.com/press-release-content?articleId=1118649" target="_blank">buy one</a>. VMware the <a href="https://www.vmware.com/it/company/acquisitions.html" target="_blank">same</a>. You can build a great Kubernetes offering - <a href="https://www.vmware.com/it/company/acquisitions/heptio.html" target="_blank">buy one</a>. </div>
<div>
<br /></div>
<div>
This is the way business works. Sometimes these mergers work and make the company better and sometimes they fail - <a href="https://www.haaretz.com/us-news/cisco-sells-nds-to-prior-owner-for-a-fifth-of-original-price-1.6049776" target="_blank">dismally</a>. Sometimes the talent gets incorporated but that is not always the case. </div>
It will all depend on how much you want to invest in the knowledge you acquired, and how much you become one with those people that bring that knowledge to the table.<br />
<br />
<h3>
True belief stays eternal</h3>
<div>
<br />
Religion is funny thing. I think I can say there is really only one religion that has stayed with us from the beginning and that is Judaism. Christianity became a well known religion - somewhere around the <a href="https://en.wikipedia.org/wiki/History_of_Christianity" target="_blank">4th century</a>. Islam - somewhere in the <a href="https://en.wikipedia.org/wiki/History_of_Islam#Timeline" target="_blank">7th century</a>. All the ancient kingdoms, rulers, empires, no matter how great they were, how much of the world they conquered (or tried to) - they no longer exist. The only true thing that people will cling to is an idea, a belief. Something that is emotional.<br />
<br />
The Persians built an empire - it is no more.</div>
<div>
The Egyptians , the Greeks, the Romans, the Ottoman empire, the list goes <a href="https://en.wikipedia.org/wiki/List_of_empires" target="_blank">on and on and on</a> - all gone. </div>
<div>
<br /></div>
<div>
In our technological world today, it is hard to call anything eternal. Computers have only been around for less than <a href="https://en.wikipedia.org/wiki/Computer#Modern_computers" target="_blank">100 years</a>. But even with its young age there are already religions forming around technology and it use..</div>
<div>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Editor_war" target="_blank">vim vs emacs</a></li>
<li>Windows vs Mac</li>
<li>Windows vs Linux</li>
<li>Closed source vs open source</li>
</ul>
</div>
It is very hard to convert someone from one religion to another, sometimes with works some <a href="https://www.history.com/topics/middle-ages/crusades" target="_blank">severe</a>, and <a href="https://en.wikipedia.org/wiki/Spanish_Inquisition" target="_blank">more severe</a>, and <a href="https://biblehub.com/esther/8-17.htm" target="_blank">sometimes less severe</a> persuasion but there are cases where people will change their mind.<br />
<br />
I am of the conviction that if what you believe in - is something that is connected to a deep emotion, something that is personal, it is something that will stay with you forever.<br />
<br />
Technology - is still in its infancy - we might not realize it - and the rate at which things change is grower faster and faster as we go along.<br />
<br />
<i>I think I got a bit lost in the journey and lost sight of the end goal here - so let me get to the point.</i><br />
<br />
Emotion, making it personal, and connecting with what you do - is something that will always stay with you. The technology you invest in, your day-to-day job, the tools you use - they will evolve and change - they are not eternal.<br />
<br />
You are not a Java guy. You are not a kubernetes girl. You are not a X.<br />
<br />
You are a person that learns, a person that adapts. Connect to your goal with emotion and this will allow you to succeed.<br />
<br />
That is who you should be!<br />
<br />
(Also published on <a href="https://www.linkedin.com/pulse/empires-built-day-also-do-last-forever-maish-saidel-keesing/" target="_blank">Linkedin</a>)Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-9811068170925454692019-01-11T10:41:00.003+02:002019-01-11T10:41:53.063+02:00The Year 2018 in reviewI don't always do these kind of posts but 2018 was a substantial year for me that warrants a short summary.<br />
<br />
I released the <a href="https://technodrone.blogspot.com/2018/01/the-aws-powershell-docker-container.html" target="_blank">AWS Powershell Container</a> - gauging by the number of <a href="https://hub.docker.com/r/maishsk/awspowershell.netcore/" target="_blank">pulls</a> - I guess that is was not that useful.. :)<br />
<br />
I completed my <a href="https://technodrone.blogspot.com/2018/01/5-aws-certifications-in-237-days.html" target="_blank"><span id="goog_2035050039"></span>5th AWS Certification</a>. The post was also translated into <a href="https://www.israelclouds.com/article/certificate-aws-237" target="_blank">Hebrew</a> as well.<br />
<br />
I presented a session at the DevOps Israel conference <br />
<br />
<iframe allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/C-DwITyVCQQ" width="640"></iframe><br />
<br />
I left Cisco (NDS) after 13 years and started a new position at <a href="https://technodrone.blogspot.com/2018/04/time-for-new-chapter-hello-cyberark.html" target="_blank">CyberArk</a>.<br />
<br />
I became a lot more involved in the Israel Cloud community (for example <a href="https://technodrone.blogspot.com/2018/07/encounters-in-cloud-interview.html" target="_blank">Encounters in the Cloud - Interview</a>).<br />
<br />
I went to re:Invent again this year - and it my posts <a href="https://technodrone.blogspot.com/2018/10/keeping-kosher-at-reinvent-2018.html" target="_blank">Keeping Kosher at re:Invent 2018</a> and <a href="https://technodrone.blogspot.com/2018/11/how-i-get-most-out-of-aws-reinvent-2018.html" target="_blank">How I Get the Most Out of #AWS re:Invent 2018</a> (<a href="https://technodrone.blogspot.com/2018/07/encounters-in-cloud-interview.html" target="_blank">Hebrew version</a>) were very useful not only to me - but from what I heard - to others as well.<br />
<br />
I was a guest on the Datanauts podcast - <a href="https://packetpushers.net/podcast/datanauts-143-getting-to-day-2-cloud/" target="_blank">Datanauts 143: Getting To Day 2 Cloud</a>. I found <a href="https://packetpushers.net/podcast/datanauts-155-the-mad-libs-prediction-show/" target="_blank">out</a> - that this episode was the <b>most popular episode</b> of the year 2018 on the show. Respect!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/2018_recap/datanauts_2018.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="450" height="400" src="https://maishsk.com/blog/images/2018_recap/datanauts_2018.jpg" width="223" /></a></div>
<br />
I presented an Ignite (in Hebrew) at DevOpsDaysTLV<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/X64H3vHUogQ" width="640"> </iframe> <br />
<br />
I also presented a session at the AWS Community Tel Aviv 2018<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/5PMpQvkXy7c" width="640"> </iframe> <br />
<br />
And last but not least - I released the <a href="https://technodrone.blogspot.com/2018/11/the-aws-visio-stencils.html" target="_blank">AWS Visio Stencils</a><br />
<br />
All in all - it was a good year.<br />
<br />
One thing that I neglected (badly!!), was my writing the rest of <a href="https://cloudwalkabout.com/" target="_blank">The Cloud Walkabout</a> - which is something that I will make the most effort to rectify this year.<br />
<br />
Looking forward to 2019... Upward and onward!!<br />
<br />
<br />Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-13668019579052218762019-01-04T10:00:00.000+02:002019-01-04T10:00:03.281+02:00I was not expecting this at re:InventThere was a lot to absorb during the jam packed week in Las Vegas but there were a number of things that I was truly surprised about during the conference..<br />
<br />
It was clear that AWS is going after the Enterprise market and are accommodating the on-prem / legacy / old-school way of thinking. This is the first re:Invent that you could really feel the change.<br />
<div>
<br />
Here are a few of them:<br />
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/outposts.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="AWS Outposts" border="0" data-original-height="340" data-original-width="800" height="268" src="https://maishsk.com/blog/images/reinvent_surprised/outposts.png" title="AWS Outposts" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/wellarchitected.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="AWS Well Architected" border="0" data-original-height="327" data-original-width="800" height="259" src="https://maishsk.com/blog/images/reinvent_surprised/wellarchitected.png" title="AWS Well Architected" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/lakeformation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Lake Formation" border="0" data-original-height="313" data-original-width="800" height="249" src="https://maishsk.com/blog/images/reinvent_surprised/lakeformation.png" title="Lake Formation" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/Security_hub.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Security Hub" border="0" data-original-height="318" data-original-width="800" height="254" src="https://maishsk.com/blog/images/reinvent_surprised/Security_hub.png" title="Security Hub" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/control_tower.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Control Tower" border="0" data-original-height="307" data-original-width="800" height="244" src="https://maishsk.com/blog/images/reinvent_surprised/control_tower.png" title="Control Tower" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/FSx.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="FSx" border="0" data-original-height="314" data-original-width="800" height="250" src="https://maishsk.com/blog/images/reinvent_surprised/FSx.png" title="FSx" width="640" /></a></div>
<br />
<br />
Next was containers or the lack of containers actually. There were no significant container announcements. ECS and EKS - were not mentioned once during the keynote. No new functionality, no new features. For the product that was probably the most demanded release that everyone wanted last year at re:Invent - this year - it was crickets all the way down. I was thinking that AWS was saving some glory and glitters for the Kubecon conference the week after - but all that really came out of there was the <a href="https://github.com/aws/containers-roadmap/projects/1" target="_blank">Containers Roadmap</a> (which is actually amazing - because AWS never disclose what their roadmap is - at least not publicly. I suppose it is expected of them as their keeping up the image of Opensource contribution and championship).<br />
<br />
And the last shocker was the fact that inbound traffic to S3 is now going to cost you money.. </div>
<div>
<br /></div>
<div>
Wait, What? You are now charged for uploads to S3????<br />
Well that is not entirely true. Traditionally - you do not pay for incoming traffic into S3 - it says that black on white. </div>
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/s3_pricing.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="s3 Pricing" border="0" data-original-height="125" data-original-width="800" height="100" src="https://maishsk.com/blog/images/reinvent_surprised/s3_pricing.png" title="s3 Pricing" width="640" /></a></div>
<div>
<br /></div>
</div>
<div>
<div>
<br /></div>
</div>
<div>
<br /></div>
<div>
So no you are not charged for direct uploads to S3. But if you do it through another service that acts as a proxy to S3 - then that's different.<br />
<br />
Storage Gateway was one such a service.<br />
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/storage_gateway.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Storage Gateway" border="0" data-original-height="203" data-original-width="800" height="162" src="https://maishsk.com/blog/images/reinvent_surprised/storage_gateway.png" title="Storage Gateway" width="640" /></a></div>
<div>
<br /></div>
</div>
<div>
Here you are allowed 100GB for free each month and capped at a maximum of $125 / month. For a company that transfers hundreds and thousands of TB a month - the $125 is chump change which essentially makes it pretty much free.</div>
<div>
<br /></div>
<div>
And then came <a href="https://aws.amazon.com/sftp/" target="_blank">AWS Transfer for SFTP</a> and the change that no-one really noticed.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/sftp_pricing.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="SFTP Pricing" border="0" data-original-height="122" data-original-width="800" height="96" src="https://maishsk.com/blog/images/reinvent_surprised/sftp_pricing.png" title="SFTP Pricing" width="640" /></a></div>
<div>
Whoa!! Not only are you being charged for 4x the amount of any other service, you are not capped at a maximum monthly spend, and you get no free monthly uploads either.</div>
<div>
<br /></div>
<div>
You use it - you pay (and pay for it you will).</div>
<div>
<br /></div>
<div>
Next up was <a href="https://aws.amazon.com/datasync/" target="_blank">DataSync</a></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/datasync_pricing.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Datasync Pricing" border="0" data-original-height="135" data-original-width="800" height="108" src="https://maishsk.com/blog/images/reinvent_surprised/datasync_pricing.png" title="Datasync Pricing" width="640" /></a></div>
<div>
<br /></div>
<div>
<br />
<br />
<br />
<br />
<br />
<br />
Again - same new price of $0.04/GB for transfer traffic <b><u>into</u></b> S3.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://maishsk.com/blog/images/reinvent_surprised/pricing_example.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Pricing example" border="0" data-original-height="140" data-original-width="800" height="110" src="https://maishsk.com/blog/images/reinvent_surprised/pricing_example.png" title="Pricing example" width="640" /></a></div>
<div>
<br /></div>
<div>
Their pricing example as well</div>
<div>
If you were to do the exact same thing - but with <b>regular S3 </b>upload. </div>
<blockquote class="tr_bq">
If you perform a one-time migration of 50 TB of 16 MB files into Amazon S3 in US East (Ohio), it costs you the following to use <b>S3 cli</b>: </blockquote>
<blockquote class="tr_bq">
(50 TB copied into S3 * 1024 GB * <b>$0.00</b> / GB) + (1 S3 LIST request * $0.005 / 1000) + (50 TB / 16 MB S3 PUT requests * $0.005 / 1000)<br />
= <b>$0</b> + $0 + $16.38<br />
= $16.38</blockquote>
That is one heck of a difference. Now I have not tested the difference in speed, or throughput you can get from Datasync - I am sure there is a difference in the data transfer speeds.<br />
<br />
<div>
But for me this is troubling. The whole bloody world uses S3 (granted most of the traffic is going from S3 out of AWS). Are AWS planning a change in their pricing model? Even if it is $0.04/GB - this would be a huge channel of additional revenue for them. Something to ponder on.</div>
<div>
<br /></div>
<div>
The pricing model that is now attached to S3 uploads seems strange to me - especially if you are receiving the exact same thing through another route for free. If it would have been network traffic through the service - I would have easily been able to accept.</div>
</div>
<div>
And last but not least, Werner Vogels finished his keynote on time this year. Well done and thank you for assisting in the effort of improving our experience at re:Invent this year.</div>
<div>
<br /></div>
<div>
Thoughts? Comments? </div>
<div>
Feel free to reach out to me on Twitter (<a href="https://twitter.com/maishsk" target="_blank">@maishsk</a>)</div>
<div>
<br /></div>
<div>
</div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-45490293688100368552018-12-19T13:50:00.001+02:002018-12-19T13:50:05.083+02:00AWS Client VPNSo after leaking (or not really leaking) from some of the sessions from re:Invent it seems that AWS have finally released the <a href="https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html" target="_blank">Client VPN</a><br />
<br />
<blockquote class="tr_bq">
<i><b>AWS Client VPN is a managed client-based VPN service that enables you to securely access your AWS resources and resources in your on-premises network. With Client VPN, you can access your resources from any location using an OpenVPN-based VPN client.</b></i></blockquote>
So instead of you having to provision a EC2 instance on your own and configure your own OpenVPN server - you can use this service<br />
<br />
But <a href="https://aws.amazon.com/vpn/pricing/" target="_blank">pricing</a> is <b><u>outrageous</u></b>...<br />
<br /><blockquote class="tr_bq">
$0.05 per AWS Client VPN connection hour<br />$0.10 per AWS Client VPN endpoint association hour</blockquote>
<br />
Assuming I would like to bring up a EC2 instance that would handle a 5 VPN connections and I leave the server running 24/7 for a month users connect for approximately 8 hours a day - 5 days a week<br />
LEaving this service provisioned for the entire month would cost<br />
<br />
0.10 * 750(hours in a month) = $75<br />
0.05 * 5(people) * 8(hours) * 5 (days) * 4 (weeks) = $40<br />
<br />
Total cost for one month - <b><span style="color: red;">$115</span></b><br />
<br />
If I were to roll my own on EC2<br />
<br />
Using a t3.small instance (2vCPU/2GB ram) should be more than sufficient.<br />
<br />
0.02 * 750 (hours in a month) = <b><span style="color: red;">$15</span></b><br />
<br />
<br />
OK - it is not comparing apples to apples - not by a long shot<br />
<br /><blockquote class="tr_bq">
<i>Client VPN offers the following features:<br /><br /><b>Secure</b> — It provides a secure TLS connection from any location using the OpenVPN client.<br /><b>Managed service</b> — It is an AWS managed service, so it removes the operational burden of deploying and managing a third-party remote access VPN solution.<br /><b>Highly available and elastic</b> — It automatically scales to the number of users connecting to your AWS resources and on-premises resources.<br /><b>Authentication</b> — It supports client authentication using Active Directory and certificate-based authentication.<br /><b>Granular control</b> — It enables you to implement custom security controls by defining network-based access rules. These rules can be configured at the granularity of Active Directory groups. You can also implement access control using security groups.<br /><b>Ease of use</b> — It enables you to access your AWS resources and on-premises resources using a single VPN tunnel.<br /><b>Manageability</b> — It enables you to view connection logs, which provide details on client connection attempts. You can also manage active client connections, with the ability to terminate active client connections.<br /><b>Deep integration</b> — It integrates with existing AWS services, including AWS Directory Service and Amazon VPC.</i></blockquote>
Are all these extra features worth paying so much more for this managed service?<br />
You are the only one that can answer this.<br />
<br />
I am throwing the gauntlet out there - for someone to write the code that will enable the provisioning of a VPN Endpoint on demand - based on usage - which will make this service more cost effective.Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-48906642006147841382018-12-10T10:00:00.001+02:002018-12-10T10:00:02.291+02:00#AWS Outposts - told you so..<p><a href="https://technodrone.blogspot.com/2018/07/the-aws-world-shook-and-nobody-noticed.html">I called it </a>- to me it was obvious that this was going to happen. The signs were all there. This was the direction that the market has been pushing for, and AWS has a reputation of giving the customers what they ask for.</p><p>The last announcement that was Andy Jassey made on the keynote on Wednesday - was AWS Outposts.<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Here's what AWS Outposts looks like.<br><br>Plus in to power and network in your data center.<br><br>Use public AWS APIs, specify your subnet to launch locally. <a href="https://t.co/gkKe1Q4QxW">pic.twitter.com/gkKe1Q4QxW</a></p>— Eric Hammond (@esh) <a href="https://twitter.com/esh/status/1067876590955094016?ref_src=twsrc%5Etfw">November 28, 2018</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Here was the <a href="https://aws.amazon.com/about-aws/whats-new/2018/11/announcing-aws-outposts/">announcement</a>. Usually Jeff Barr (or as of late - someone else on the Technical Evangelist team) have a detailed blog post - on a new product that was just announced. <p>For AWS Outposts - nada… The only thing that is out there - is the announcement - and a “TBD” product page - <a href="https://aws.amazon.com/outposts/">https://aws.amazon.com/outposts/</a><p><a href="https://maishsk.com/blog/images/5f6e509470f9_76E7/image.png"><img width="825" height="111" title="image" style="margin: 0px; border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="image" src="https://maishsk.com/blog/images/5f6e509470f9_76E7/image_thumb.png" border="0"></a><p>Once the announcement was made - VMware went all out with as much information as they could describing the VMware variant of AWS outposts <a href="https://cloud.vmware.com/community/2018/11/28/vmware-cloud-aws-outposts-cloud-managed-sddc-data-center/">https://cloud.vmware.com/community/2018/11/28/vmware-cloud-aws-outposts-cloud-managed-sddc-data-center/</a><p>Blog posts, interviews, sessions you name it they went all in - for a very good reason - if you ask me. This expands their VMware Cloud on AWS in a substantial way.<p>And who was missing from this announcement ? <strong>AWS</strong>. </p><p>To me this is puzzling. The one sided coverage of something that is supposed to be a joint venture, means that either - this was a pure publicity announcement - and the product has not yet been finalized - or AWS dropped the ball on this one - big time!!</p><p>So what do we know about a this product? It will come in two flavors:<ul><li><p>VMware Cloud on AWS Outposts allows you to use the same VMware control plane and APIs you use to run your infrastructure</p><li><p>A native variant of AWS Outposts allows you to use the same exact APIs and control plane you use to run in the AWS cloud, but on-premises. </p></li></ul><p><img width="778" height="287" src="https://lh3.googleusercontent.com/0ZIjN2dBjNNreHzlVW1Mt1nMnrSINy1ubK_b4cBOd4AkBc0I_6-ZI7uYPH4eqpoKiev21nAjo0TZflbAK7LFA6lYSScQRwMkc6Seqtwqe89R4G0MU7024P8eDumK-zbmG19TE4vx"><p>The AWS native variant of AWS Outposts allows you to use the same exact APIs and control plane you use in the AWS cloud, but on-premises. You will be able to run Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Elastic Block Store (Amazon EBS) on Outposts. At launch or in the months after, we plan to add services like RDS, ECS, EKS, SageMaker, EMR.<p>Not a word has been published since the announcement, of how this is going to work from the perspective of the “AWS variant” Outposts. <p><blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">If there is one thing that has left me completely confused from <a href="https://twitter.com/hashtag/reInvent?src=hash&ref_src=twsrc%5Etfw">#reInvent</a> - is the <a href="https://twitter.com/awscloud?ref_src=twsrc%5Etfw">@awscloud</a> story of <a href="https://twitter.com/hashtag/Outposts?src=hash&ref_src=twsrc%5Etfw">#Outposts</a>. VMware have been pluggin this to death this weak - but besides the announcement - there has hardly been any details about the <a href="https://twitter.com/hashtag/AWS?src=hash&ref_src=twsrc%5Etfw">#AWS</a> side.</p>— Maish Saidel-Keesing (☁️☁️☁️) (@maishsk) <a href="https://twitter.com/maishsk/status/1068554130623389696?ref_src=twsrc%5Etfw">November 30, 2018</a></blockquote><p>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
I even went as far and asked Jeff Barr - what is the story here. The funny thing is - I actually met him at Starbucks about 15 minutes after I posted the tweet. </p><blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">So I just had a quick chat with <a href="https://twitter.com/jeffbarr?ref_src=twsrc%5Etfw">@jeffbarr</a> about this outside Starbucks. Super nice guy!! Hope to hear and see more details about this in the near future. <a href="https://twitter.com/hashtag/reInvent?src=hash&ref_src=twsrc%5Etfw">#reInvent</a></p>— Maish Saidel-Keesing (☁️☁️☁️) (@maishsk) <a href="https://twitter.com/maishsk/status/1067944213713756160?ref_src=twsrc%5Etfw">November 29, 2018</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>His answer (if my memory serves me correctly) was..<blockquote><p><em><strong> “The team had not yet had the opportunity to go into detail into the new offering, and would be publishing more details about it"</strong></em></p></blockquote><p>To me Outposts - is the <strong>biggest announcement</strong> of the whole of re:Invent - if played correctly - it will remove any and all competition that is hoping to provide a Hybrid cloud story - one that enterprises can understand. <p>You want AWS - you can have it - in the cloud - and also on prem - the same exact experience - this is something that customers have been asking for years for AWS to provide (and also something that AWS have consistently been completely against - because everything and anything should run in AWS - there is no need for on-prem… - until now :) ) <p>And mark my words, once you have an Outpost in almost every single datacenter - the need for Edge locations in each and every country - <strong><em>will be no more...</em></strong> <p>I guess we will have to wait for the aftermath to die down - and wait to see exactly how this going to work….<p>And now some of my personal thoughts about this whole topic.<p>There are a lot of moving parts that AWS will now have to go into - especially regarding the logistics of providing the end service to the customer.<p>If you remember there was once another product - that provided you with a similar service - yep I am talking about the <a href="https://en.wikipedia.org/wiki/VCE_(company)" target="_blank">vBlock</a> - a joint venture from VMware, Cisco and EMC. Which went the <a href="https://www.technologyreview.com/s/529841/the-way-of-the-dodo/" target="_blank">way of the dodo</a>. The partnership fell apart for a number of reasons. <p>Customers loved the solution!! You had a single number to call - for anything and everything related to the deployment. Disk died? Called the support number. Network not working? Call the support number. vSphere doing some crazy shit? Call the same support number. One neck to throttle, and customers loved it. <p>And now you have Amazon selling you hardware - or should I rather say leasing you the hardware. You will not own it - you will pay as you go. I assume that there will be a commitment - of some kind - and you will not be able to order by the hour - the logistics on per hour would be too complicated.<p>But speaking of logistics - if there is a company that commit to having a 4 hour delivery time on a failed piece of hardware - it is Amazon - with their global presence. They have the logistical capability to ensure delivery of practically anything in their inventory to anywhere in the world - in the shortest amount of time.<p>But there are still many unknowns... here are a few that come to mind:<ul><li>Will this come with a networking component? I assume it will - what will that network component be? Software? Hardware?</li><li>By providing you (the customer) with the same experience and AWS hardware - are they risking exposure of how AWS works getting out? I assume that this will be covered in TOS and NDA that you sign as part of the upcoming service.</li><li>I assume there will be redundant network connectivity requirements in order for this to work - I will also go out on a limb and say that a Direct Connect link will be a requirement as well. This means that it will be only be suitable for a certain piece of AWS's customers. Perhaps redundant VPN's might be suitable as well.</li><li>What happens if/when the AWS endpoints are not available? How if at all can the instances and the workloads on the Outpost be managed? </li><li>How <em>self-service</em> will the offering be? I assume it will only be a node-by-node expansion - or per 1/4 rack. you will not be able to add more disks on your own, more RAM on your own etc. This makes sense. </li></ul><p>In short - since this was announced at re:Invent 10 days ago - and that AWS have already stated this will not be available before H2 2019 - I do not expect that we will see anything before October/November 2019 (but that is just my hunch).</p><p>At the moment - there is a lot more to this announcement than meets the eye.... </p>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-53389220311187273512018-12-05T17:04:00.000+02:002018-12-05T17:04:43.229+02:00My overall impression of re:Invent 2018 #reInvent #aws<p>I am now on a plane on my way back home, on a really long flight from SFO to TLV (13.5 hours) so now is a good time to re-cap and reflect on what happened last week at re:Invent.<br>
I think that this will be a set of posts - because there are a number of topics that I would like to address - and some of them deserve their own dedicated insight.</p>
<p>The first and foremost post I would like to go into - is the overall impression about of the conference.</p><p><a href="https://maishsk.com/blog/images/My-overall-impression-of-reInvent-2018-r_EA8C/IMG_20181125_140329.jpg"><img width="640" height="480" title="IMG_20181125_140329" style="border: 0px currentcolor; border-image: none; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;" alt="IMG_20181125_140329" src="https://maishsk.com/blog/images/My-overall-impression-of-reInvent-2018-r_EA8C/IMG_20181125_140329_thumb.jpg" border="0"></a></p>
<p> <br>
AWS made a significant number of changes as compared to last years event. And overall - I found the event to be amazing!!</p>
<p> <br>
If you ask me - last year’s event was not user friendly, for a number of reasons.</p><ul><li>Tracks were located in a single venue. That meant going between topics was not really possible. </li><li>Transport - the shuttles had a route - along a number of of the venues. The shuttles took a great deal of time.</li><li>Lines in the sessions were bad - they were really bad - people were lining up for hours before, without any real indication if they were going to get into a session.</li><li>The mobile app - was pretty much useless - and was not at all helpful.</li><li>The amount of repeats were not enough, and overflow sessions were also scarce.</li></ul><p>This year AWS fixed all of the above.</p>
<ul><li> Tracks were not restricted to a single venue, you could get ML, serverless, Storage and networking - were not only in one venue - but in multiple venues, that meant you did not need to bounce around between the venues.</li><li>The shuttles were point to point. No more round trips. This was brililant to save time - but on the other hand - there were a number of times where there were 3-5 people on a shuttle at times, not really an efficient way to spend money - it was kind not elastic in any way - and not well utilized from a cost perspective.</li><li>The mobile app - was much better, still slow as hell - but there was more functionality. Such as when will the sessions be repeated, what sessions have open seats right now, how much time it will take to get from one venue to the other - in real-time.</li><li>There were many more overflows… The amount of repeats were by large - more than we had last year - which meant you had an option to choose..</li></ul>
<p>
The lines this year for sessions - were better - much, much better!! No more lines of 500 people wrapping round the whole of the Venetian to get into a session. No more disgruntled attendees - who were not able to get into a session after having waited for an hour in line. </p>
<p>Lines for buses were much shorter - no more “routes” - but point to point - which was very well managed and funneled throughout the event.</p>
<p> You were not allowed to line up for a session more than an hour in advance. Now this solved most of the long line problems, but was not always enforced (take the DeepRacer sessions for example) </p>
<p> For me the overall impression was amazing. I think that I was only turned away from a single session throughout the whole event - and that was a builder session - which I was not registered to. I managed to get into any session I wanted, not only frontal sessions, but also workshops as well. </p>
<p> AWS pride themselves on being fanatical about their customers, they listen to what their customers want, they listen to their feedback and they want to make thing better, they want to solve our problems. The feedback that I heard from attendees from last year was that it was a in plain words - a train wreck - because of all the reasons above. </p>
<p> If you ask me - they addressed all of the feedback points, and fixed almost all of them.</p>
<p> And for that I take my hat off to the event team - and say Bravo, that is a job well done.</p>
<p> Next posts will go into some more details about the announcements and some of the sessions. </p>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-14349124500607559852018-11-12T10:00:00.001+02:002018-11-12T21:35:18.751+02:00How I Get the Most Out of #AWS re:Invent 2018I am not an expert, and I only went to re:Invent for the first time last year, but I have been to a quite a number of conferences over the years.<br />
<br />
So here come my thoughts about making the most of the crazy week in Vegas.<br />
<br />
<a href="https://maishsk.com/blog/images/a419e94c5899_7CE1/image_3.png"><img alt="re:Invent" border="0" height="83" src="https://maishsk.com/blog/images/a419e94c5899_7CE1/image_thumb_3.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="re:Invent" width="640" /></a><br />
<h3>
<br />The (regular) sessions</h3>
<br />Contrary to what you might think, going to sessions where you have a speaker (or speakers) up on stage going through a slide deck, or a panel of speakers talking about a subject - is where you should be, is <strong>not a good use of your time</strong>.<br />
<br />
There are currently 2358 sessions and activities listed on the portal (a good portion of them are repeats - but hell that is a lot of content)<a href="https://maishsk.com/blog/images/a419e94c5899_7CE1/image.png"><img align="right" alt="sessions" border="0" height="97" src="https://maishsk.com/blog/images/a419e94c5899_7CE1/image_thumb.png" style="background-image: none; display: inline; float: right; margin: 7px 0px 9px;" title="sessions" width="240" /></a><br />
<br />
Almost all of the sessions (I will get back this in a few minutes) are recorded and therefore can be consumed after the event - in the car, on the bus or train - or even in the air during your travels. <br />
<br />
Here is a podcast feed (<a href="http://aws-reinvent-audio.s3-website.us-east-2.amazonaws.com/2017/2017.html" title="http://aws-reinvent-audio.s3-website.us-east-2.amazonaws.com/2017/2017.html">http://aws-reinvent-audio.s3-website.us-east-2.amazonaws.com/2017/2017.html</a>) for all 2017 sessions for your listening pleasure.<br />
<br />
That is why you can spend your time better elsewhere. <br />
<h1>
</h1>
<h3>
</h3>
<h3>
<br />The Builder / Chalk Talk / Workshop sessions</h3>
<br />
Here is where I would spend my time. The cost of re:Invent (if you paid the full price) is $1,800 for 4.5 days (Friday is a short day). These are the sessions that will <strong>not be recorded</strong> and where I will probably get the most benefit <br />
(and here are some of my interests). The value I receive is from doing things that I learn from, not by being a passive listener, but by actively participating in a discussion or an activity.<br />
<h4>
</h4>
<h4>
<br /><a href="https://reinvent.awsevents.com/learn/chalk-talk" target="_blank">Chalk talks</a></h4>
<br />
This is similar to getting a design session and time with an AWS expert in their field and diving deep into a specific subject. Most of the sessions are level 300/400 - which meant they are advanced and highly technical. The rooms are small - usually no more than 50-100 people and the participants there are usually people that are looking for a very specific answers about the journey they have embarked on - or are about to.<br />
<br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88688" target="_blank">ARC210-R - SaaS Jumpstart: A Primer for Launching Your SaaS Journey</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88681" target="_blank">ARC213-R - Architecting for the Cloud</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88691" target="_blank">ARC216 - SaaS Operations: The Foundation of SaaS Agility</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22850" target="_blank">ARC301 - Cost Optimization Tooling</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22852" target="_blank">ARC306 - Breaking up the Monolith</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22854" target="_blank">ARC310-R - From One to Many: Diving Deeper into Evolving VPC Design</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22853" target="_blank">ARC317-R - Reliability of the Cloud: How AWS Achieves High Availability</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88690" target="_blank">ARC325 - SaaS Analytics and Metrics: Capturing and Surfacing the Data That's Fundamental to Your Success</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88692" target="_blank">ARC326-R1 - Migrating Single-Tenant Applications to Multi-Tenant SaaS</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22873" target="_blank">ARC408 - Under the Hood of Amazon Route 53</a><br />
<br />
<h4>
<a href="https://reinvent.awsevents.com/learn/builders-sessions" target="_blank">Builder Sessions</a></h4>
<br />
Looking for some personal time with an SA on a specific topic, and even better - you get to build the solution at hand with the guidance from the expert on-hand. Pure learning experience. <br />
<br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=90097" target="_blank">ANT402-R - Securing Your Amazon Elasticsearch Service Domain</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=23100" target="_blank">ARC415-R - Building Multi-Region Persistence with MySQL</a><br />
<br />
<h4>
<a href="https://reinvent.awsevents.com/learn/workshops" target="_blank">Workshops</a></h4>
<br />
Again - a hands-on learning experience - 2-3 hours of sitting down on a specific topic getting my hands dirty...<br />
<br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22871" target="_blank" title="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22871">ARC404 - Designing for Operability: Getting the Last Nines in Five-Nines Availability</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=91124" target="_blank" title="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=91124">ARC315-R1 - Hands-On: Building a Multi-Region Active-Active Solution</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=91495" target="_blank">ARC327-R1 - Hands-on SaaS: Constructing a Multi-Tenant Solution on AWS</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=22870" target="_blank">ARC403 - Resiliency Testing: Verify That Your System Is as Reliable as You Think</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=89675" target="_blank">CMP403-R1 - Running Amazon EKS Workloads on Amazon EC2 Spot Instances</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=91451" target="_blank">DEV303-R2 - Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWatch</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=91439" target="_blank">DEV306-R - Monitoring for Operational Outcomes and Application Insights: Best Practices</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88842" target="_blank">GPSWS402 - Continuous Compliance for Modern Application Pipelines</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88838" target="_blank">GPSWS407 - Automated Solution for Deploying AWS Landing Zone</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=88939" target="_blank">NET410 - Workshop: Deep Dive on Container Networking at Scale on Amazon EKS, Amazon ECS, & Amazon EC2</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=90862" target="_blank">SEC331-R1 - Find All the Threats: AWS Threat Detection and Remediation</a><br />
<a href="https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=90863" target="_blank">SEC337-R - Build a Vulnerability Management Program Using AWS for AWS</a><br />
<br />
<h3>
Hackathons</h3>
<br />
Want to geek out and build something, play a game or solve a whodunnit quest? This is where I will get <a href="https://reinvent.awsevents.com/learn/hackathon" target="_blank">my game on</a>. Some are for fun, some are for fame, and others just for plain doing some good.<br />
<br />
<h3>
Giving back</h3>
<br />
Being at re:Invent is something that is fun, and usually something that can involve consumption of many things. Food, alcohol, entertainment and even your hard earned cash. Me <a href="https://technodrone.blogspot.com/2017/06/giving-just-for-sake-of-giving.html" target="_blank">being me</a> - I prioritize giving back to others as part of my daily life. Spending a week at a conference only receiving is not something I am comfortable with.<br />
So as a result I will be spending some of my time here <a href="https://reinvent.awsevents.com/play/giving-back" title="https://reinvent.awsevents.com/play/giving-back">https://reinvent.awsevents.com/play/giving-back</a><br />
<blockquote>
<em><strong>The BackPack for Kids program provides bags of nutritious, single-serving, ready-to-eat food items each Friday to children who might otherwise go without during weekends and long breaks from school. Come by the Venetian Sands Foyer to get involved and help put together a backpack or two! Learn more about Three Square </strong></em><a href="https://www.threesquare.org/programs/childhood/backpack-for-kids"><em><strong>here</strong></em></a><em><strong>.</strong></em></blockquote>
<br />
<h3>
Keynotes</h3>
<br />
Event though the keynotes can be consumed from a live stream - there is something about sitting in a room (or a huge hall) with a boatload of people - where <a href="https://twitter.com/ajassy" target="_blank">Andy Jassy</a> goes up on stage and bombards you with all the new features that are coming (some that will only be available <a href="http://technodrone.blogspot.com/2018/10/how-long-until-you-get-new-shiny-toys.html" target="_blank">sometime in the future</a>). But still it is quite mesmerizing and if you have not been in one of these keynotes - I would suggest you go. It is quite an experience. <br />
<br />
<h3>
The Certification Lounge</h3>
<br />
As Corey Quinn just wrote a <a href="https://lastweekinaws.com/blog/im-an-aws-certified-cloud-practitioner.html" target="_blank">few days ago</a> <br />
<blockquote>
<em><strong>it's a $100 lounge pass with a very odd entrance questionnaire</strong></em></blockquote>
If you have an AWS certification - go to the lounge - it is a place to get away from the other 49,000 others in the hallways and the constant buzz around you.<br />
<br />
<h3>
The Expo</h3>
<br />
Do not under any circumstances miss going to the <a href="https://reinvent.awsevents.com/partners-sponsors/expo" target="_blank">Expo</a> floor. To really make proper use of the floor - I would say you will need a good 6-8 hours of your schedule (don't do it one shot though). Go to the vendors, especially the smaller ones that don't have the huge booths. Look at your competition, speak to people, make yourself known. Yes you will be bombarded after the show with sales calls - but all it takes is a simple "Sorry not interested anymore" and most vendors will leave you be.<br />
<h3>
<br />Social media</h3>
<br />
I don't think I could get by without following what is going on in Twitter. <br />
I have a search column dedicated for re:Invent (already for the past month) <br />
<br />
<a href="https://maishsk.com/blog/images/a419e94c5899_7CE1/image_4.png"><img alt="image" border="0" height="480" src="https://maishsk.com/blog/images/a419e94c5899_7CE1/image_thumb_4.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="image" width="421" /></a><br />
<br />
I will also be checking the og-aws <a href="https://og-aws-slack.lexikon.io/" target="_blank">Slack channel</a> to co-ordinate snark about the announcements and on-goings at the event and also some face to face meetings with some of the people that I only have met through their avatars.<br />
<br />
(And as always the great set of posts at the <a href="http://conferenceparties.com/reinvent2018/guide-of-guides/" target="_blank">Guide of Guides</a> is invaluable.)<br />
<br />
See you all in 2 weeks! Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-17022089545276806392018-11-08T10:00:00.000+02:002018-11-08T10:00:09.625+02:00Events as a Service (EaaS)Most vendors that perceive themselves as a market leader will have a major annual event (some will even have multiple events in different geographical locations).<br />
<div>
<br /></div>
<div>
Here are few of these major events that come to mind:<br />
<div>
<br /></div>
<div>
<ul>
<li>AWS (<a href="https://reinvent.awsevents.com/" target="_blank">re:invent</a>) </li>
<li>VMware (<a href="https://www.vmworld.com/" target="_blank">VMworld</a>)</li>
<li>Cisco (<a href="https://ciscolive.cisco.com/" target="_blank">Cisco Live!</a>)</li>
<li>Microsoft (<a href="https://www.microsoft.com/en-us/ignite" target="_blank">Ignite</a>)</li>
<li>Google (<a href="https://cloud.withgoogle.com/next18/sf/" target="_blank">Cloud Next</a>) </li>
</ul>
<div>
<br />
And every year we come around to the registration and scheduling of sessions to these events, and they almost always suck... </div>
</div>
</div>
<div>
<br /></div>
<div>
(I am going to use re:invent as the victim here - but I am sure that the experience is probably the same with most conferences) </div>
<div>
<ul>
<li><a href="https://www.reddit.com/r/aws/comments/7f0qd5/ads_on_the_android_reinvent_app/" target="_blank">The Mobile App sucks</a>.</li>
<li>The registration process is annoying. </li>
<li>When sessions open for scheduling - the <a href="https://twitter.com/AWSreInvent/status/1050446278616698881" target="_blank">site crashes</a>.</li>
<li>Scheduling sucks. </li>
<li><a href="https://www.startsmartltd.com/blog/news/2017/12/6/aws-reinvent-2017" target="_blank">The lines are horrible</a>.</li>
<li>Transport sucks.</li>
<li>Food sucks.</li>
</ul>
</div>
<div>
<br /></div>
<div>
There are more than enough things that one could find wrong with the way things go at a conference - and I am not diminishing the problems one little bit.</div>
<div>
<br /></div>
<div>
I would like us all to view it in a different perspective.</div>
<div>
<br /></div>
<div>
The companies that hold these events - are tech companies. They are great at selling technology, great at creating some amazing technology. An of course they also have people that are in charge of events and marketing - but it is not their core business. </div>
<div>
<br /></div>
<div>
I do not underestimate the impact a good event can have on your product - or how a bad event can damage a company's brand - that is why companies like these spend many millions of dollars on events like this. But again that is not what they are trying to sell, they are not trying to sell an event. They are not event planners, this is something we seem to forget from time to time especially when things are not optimal (another polite way of saying that they suck).</div>
<div>
<br /></div>
<div>
They outsource the events to an external company.</div>
<div>
<br /></div>
<div>
The signs, the transport, the advertising, the venue, website, the on-site services, scanners, the food - and yes - even the mobile app. All of these do not belong to any one of these companies they are all provided as part of the service that another company sells to these market leaders.</div>
<div>
<br /></div>
<div>
It does not make sense for any of the large vendors to bring up an event all by themselves. For an event that is sometimes no more than 5 days in a year - they will not maintain all the dedicated resources (physical, human and virtual) for just one event. </div>
<div>
<br /></div>
<div>
So it make sense to outsource it all. And they do.</div>
<div>
<br /></div>
<div>
There are a few vendors out there that are capable of bringing up events on this scale - such as <a href="https://www.cvent.com/" target="_blank">Cvent</a> or <a href="https://www.lanyon.com/" target="_blank">Lanyon</a> and if you ask me - they do a pretty good job.</div>
<div>
<br /></div>
<div>
There are always things that can be improved. The app could be better (this year there are significant improvements in the re:Invent app experience 😃 ) The registration could be better, the directing of human traffic at the conference could better, the list could go on and on.</div>
<div>
<br /></div>
<div>
Is <b>IS</b> the job of the tech vendor marketing teams to demand from these event companies to improve from one event to another and get better from year to year. To make sure the food is better, improve registration, make sure that the (also human) traffic flows. </div>
<div>
<br /></div>
<div>
If I look at this from a technology perspective - it is a classic case of consuming something aaS (As a Service). AWS provides us with infrastructure, and they maintain software. but they do not employ all the people that put the chips on the motherboards of every server in their datacenters. They do have people that provide input into the design of the servers - in order for them to operate more efficiently, and in turn provide a better service to their customers (you and me). </div>
<div>
<br /></div>
<div>
I would not expect them to have chip designers or assembly plants on the payroll to allow them to run their business. They outsource / contract that work from a 3rd party. </div>
<div>
<br /></div>
<div>
They contract / outsource their event management. All the big companies do - it makes perfect financial sense. </div>
<div>
<br /></div>
<div>
Does that mean we should stop bitching about the food, the lines, the app? Hell no! By providing constructive criticism (or complaining) we make things better, because that is what we the customer demand. And these event management companies - will hopefully improve.</div>
<div>
<br /></div>
<div>
Some food (pun intended) for thought - when you are your next conference. </div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-54001121137481192018-11-05T12:00:00.003+02:002018-12-13T23:49:22.935+02:00The #AWS Visio Stencils<br />It seems like only yesterday, but it was actually almost 10 years ago when I gave something awesome to the VMware community - <a href="https://technodrone.blogspot.com/2013/04/vmware-visio.html" target="_blank">the first version of the VMware stencils</a>.<br /><br />
The reason I did this was because at the time - there was no decent set of VMware stencils out there - so I took the initiative and created a set. And I subsequently set out to update them over the years.<br />
I have a small confession. The VMware Visio stencils have been the biggest driver of traffic to my blog over the years. Even till this day - I have a minimum of 5000 monthly views (and this on a post that is more than 5 years old). <br />
<br />And I already hear you say - there is already a number of architectural Visio icon sets available - and even an official one from AWS - you can find them <a href="https://aws.amazon.com/architecture/icons/" target="_blank">here</a>. (I assume that the graphics are going to be updated just before / after re:Invent - with the new design that they have released a few weeks ago, in the meantime - only the current one is available)<br />
<br />There are other tools that have online graphics as well. <a href="https://www.lucidchart.com/pages/aws" target="_blank">LucidChart</a>, <a href="https://cacoo.com/lang/en/examples/aws-software" target="_blank">Cacoo</a>, <a href="http://creately.com/draw-amazon-architecture-diagrams" target="_blank">Creately</a>, <a href="https://www.draw.io/?splash=0&libs=aws3" target="_blank">draw.io</a>, <a href="https://cloudcraft.co/" target="_blank">Cloudcraft</a> (the only vendor who has original graphics - the rest are all the standard AWS icons). <br />
If you are following some of the AWS community work - will probably have heard of Jerry Hargrove (better know as <a href="https://twitter.com/awsgeek" target="_blank">@awsgeek</a>). He actually works at Lucidchart and is famous for his <a href="https://www.awsgeek.com/" target="_blank">unbelievable sketch notes</a> on AWS and their products. Not only are they beautiful, clear and sometimes even really funny, they are also very informative and extremely useful. <br />
<br />Jerry was also recently awarded the honor of <a href="https://aws.amazon.com/developer/community/heroes/jerry-hargrove/" target="_blank">AWS community Hero</a>.<br />
I mean really - <a href="https://www.awsgeek.com/" target="_blank">these</a> are a real work of art!!<br /><br />
<a href="https://maishsk.com/blog/images/The-AWS_13E61/image.png"><img alt="image" border="0" height="176" src="https://maishsk.com/blog/images/The-AWS_13E61/image_thumb.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="image" width="720" /></a><br />
<br />
<br />So without further ado I present to you version <a href="https://maishsk.com/blog/aws-visio-stencils/v1.0/AWS%20Product%20Icons.vssx" target="_blank">1.0</a><strong></strong> of the <a href="https://maishsk.com/blog/aws-visio-stencils/v1.0/AWS%20Product%20Icons.vssx" target="_blank">AWS Community Visio Stencils</a>.<br />
<a href="https://maishsk.com/blog/images/The-AWS_13E61/all_icons.png"><img alt="all_icons" border="0" height="422" src="https://maishsk.com/blog/images/The-AWS_13E61/all_icons_thumb.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="all_icons" width="496" /></a><br />
Jerry was kind enough to allow me to use his graphics and provide the AWS community with a set of graphics - that (in my opinion) are not only more appealing to the eye - but are just plain fun!!<br />
<br />40 icons of AWS services.<br />
<ul>
<li>API Gateway</li>
<li>
AppStream</li>
<li>Athena</li>
<li>Cloudfront</li>
<li>CloudTrail</li>
<li>CloudWatch</li>
<li>Code Build</li>
<li>Code Pipeline</li>
<li>Comprehend</li>
<li>Directory Service</li>
<li>EBS</li>
<li>EC2</li>
<li>EFS</li>
<li>Elastic Beanstalk</li>
<li>ElasticCache</li>
<li>ELB</li>
<li>GuardDuty</li>
<li>IAM</li>
<li>Kinesis</li>
<li>KMS</li>
<li>Lambda</li>
<li>Lambda Edge</li>
<li>Machine Learning</li>
<li>Neptune</li>
<li>RDS</li>
<li>Redshift</li>
<li>Rekognition</li>
<li>Route53</li>
<li>S3</li>
<li>SES</li>
<li>SNS</li>
<li>SQS</li>
<li>Step Functions</li>
<li>Storage Gateway</li>
<li>VMware on AWS</li>
<li>VPC</li>
<li>VPC Endpoint</li>
<li>WAF</li>
<li>WorkSpaces</li>
<li>xRay<br />
</li>
</ul>
All of the graphics are from Jerry's artwork. <br />
Each of the Icons is resizable<br />
<a href="https://maishsk.com/blog/images/The-AWS_13E61/resize.png"><img alt="resize" border="0" height="290" src="https://maishsk.com/blog/images/The-AWS_13E61/resize_thumb.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="resize" width="352" /></a><br />
If you so please, the blue background can be removed.<br />
<a href="https://maishsk.com/blog/images/The-AWS_13E61/remove_background.png"><img alt="remove_background" border="0" height="181" src="https://maishsk.com/blog/images/The-AWS_13E61/remove_background_thumb.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="remove_background" width="348" /></a><br />
You can modify the text on each icon.<br />
<a href="https://maishsk.com/blog/images/The-AWS_13E61/change_text.png"><img alt="change_text" border="0" height="198" src="https://maishsk.com/blog/images/The-AWS_13E61/change_text_thumb.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="change_text" width="348" /></a><br />
Each icon has 9 possible anchor points.<br />
<a href="https://maishsk.com/blog/images/The-AWS_13E61/anchor_points.png"><img alt="anchor_points" border="0" height="367" src="https://maishsk.com/blog/images/The-AWS_13E61/anchor_points_thumb.png" style="background-image: none; border: 0px currentcolor; display: inline;" title="anchor_points" width="348" /></a><br />
All yours to use for free, to modify, and diagram to your hearts content.<br />
<br />And yes - this is only the beginning... There are over another 200 graphics and icons that I will be taking out of these sketches and converting them into usable icons for your diagramming pleasure.<br />
<br />v1.0 is available for download <a href="https://maishsk.com/blog/aws-visio-stencils/v1.0/AWS%20Product%20Icons.vssx" target="_blank">here</a><br />
<br />
I would love to hear your feedback!<br />
<br />
<b>Update Dec. 13, 2018</b><br /><br />Today I have released version 1.1 of the Stencils. Here is what changed.<br /><br />
<ol>
<li>New AWS Product icon - DynamoDB</li>
<li>I decided to add a few additional stencils<br /><br />a. People<br /><br /><br /><a href="https://maishsk.com/blog/images/The-AWS_13E61/people1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="800" data-original-width="569" height="320" src="https://maishsk.com/blog/images/The-AWS_13E61/people1.jpg" width="227" /></a> <a href="https://maishsk.com/blog/images/The-AWS_13E61/people2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="800" data-original-width="576" height="320" src="https://maishsk.com/blog/images/The-AWS_13E61/people2.jpg" width="230" /></a><br /><br /><a href="https://maishsk.com/blog/images/The-AWS_13E61/people3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="576" data-original-width="654" height="281" src="https://maishsk.com/blog/images/The-AWS_13E61/people3.jpg" width="320" /></a><br /><br /><br />b. Icons<br /><br /><br /><a href="https://maishsk.com/blog/images/The-AWS_13E61/icons1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="800" data-original-width="582" height="320" src="https://maishsk.com/blog/images/The-AWS_13E61/icons1.jpg" width="232" /></a><a href="https://maishsk.com/blog/images/The-AWS_13E61/icons2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="138" data-original-width="534" height="82" src="https://maishsk.com/blog/images/The-AWS_13E61/icons2.jpg" width="320" /></a><br /><br /><br />c. Shapes and Banners<br /><br /><br /><a href="https://maishsk.com/blog/images/The-AWS_13E61/shapes1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="800" data-original-width="599" height="320" src="https://maishsk.com/blog/images/The-AWS_13E61/shapes1.jpg" width="239" /></a> <a href="https://maishsk.com/blog/images/The-AWS_13E61/shapes2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="645" data-original-width="672" height="307" src="https://maishsk.com/blog/images/The-AWS_13E61/shapes2.jpg" width="320" /></a></li>
</ol>
<div>
<br /></div>
<div>
Enjoy!! There is still more to come.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
v1.0 is available for download <a href="https://maishsk.com/blog/aws-visio-stencils/v1.0/AWS%20Product%20Icons.vssx" target="_blank">here</a></div>
<div>
v1.1 is available for download here:<br /><br /><ul>
<li><a href="https://maishsk.com/blog/aws-visio-stencils/v1.1/aws-product-icons.vssx" target="_blank">Product Icons</a></li>
<li><a href="https://maishsk.com/blog/aws-visio-stencils/v1.1/icons.vssx" target="_blank">Icons</a></li>
<li><a href="https://maishsk.com/blog/aws-visio-stencils/v1.1/people.vssx" target="_blank">People</a></li>
<li><a href="https://maishsk.com/blog/aws-visio-stencils/v1.1/shapes-banners.vssx" target="_blank">Shapes and Banners</a></li>
</ul>
</div>
<div>
<br /></div>
Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-52507167372257331522018-10-22T10:00:00.000+03:002018-10-22T10:00:00.179+03:00Keeping Kosher at re:Invent 2018<div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><br></div>
<div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr">
So we are a little more than a month away from the yearly ascent to all things AWS - <a href="https://reinvent.awsevents.com/" target="_blank">re:Invent 2018</a>. <br><br>Last year one of my most useful posts was the Kosher perspective on the event <a href="https://technodrone.blogspot.com/2017/11/keeping-kosher-at-reinvent-2017.html" target="_blank">Keeping Kosher at re:Invent 2017</a>. <br><br>So this year - nothing much has changed - there is still no kosher food.. #boo</div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><br></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><a href="https://maishsk.com/blog/images/Keeping-Kosher-at-reInvent-2018_6BD4/image.png"><img width="778" height="145" title="No Kosher Food #boo" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="No Kosher Food #boo" src="https://maishsk.com/blog/images/Keeping-Kosher-at-reInvent-2018_6BD4/image_thumb.png" border="0"></a></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><br></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr">(This is not last years graphic but taken from the current site) <br><br>So again - no Kosher food throughout the day.<br><br>Last year, I went on Sunday to one of the Kosher supermarkets, and did some shopping. Every morning I made myself lunch for every day. Better than standing in lines or going out looking for food. </div><div><br></div>
<div>
Hot and cold drinks throughout the day are available at the various venues, sometimes there are fresh fruit - and some snacks here and there that have a Kosher certification (OU, OK, Star-K, and <a href="https://www.kashrut.com/agencies/" target="_blank">many more</a> - depending on what are comfortable with eating) <br><br>The Supermarkets were great and they had a wonderful selection of Kosher food<br><ul>
<li>Smith's Food and Drug Store (<a href="https://www.google.co.il/maps/place/2211+N+Rampart+Blvd,+Las+Vegas,+NV+89134,+USA/@36.1999906,-115.282494,127a,35y,330.34h,45t/data=!3m1!1e3!4m5!3m4!1s0x80c8955532d85dbb:0xc0a43b43f02621d!8m2!3d36.2017257!4d-115.2835733" target="_blank">2211 N. Rampart Blvd.</a>) 25 Minute drive from the Venetian</li>
<li>Albertson's Market (<a href="https://www.google.co.il/maps/@36.1432332,-115.2973612,28a,49.3y,85.5h,68.93t/data=!3m1!1e3" target="_blank">2550 S. Fort Apache</a>). 25 Minute drive from the Venetian.</li>
</ul>
<br>There is The Jewish Visitor's Guide to Las Vegas guide (downloadable <a href="http://chabadlv.org/media/pdf/508/feob5086257.pdf" target="_blank">here</a>) which has accurate information as of June 2018. <br><br>Here is a list of the Kosher restaurants as of today (please check the sites for up to date information before you go)<br><ul>
<li><a href="http://aceofsteakslv.com/" target="_blank">Ace of Steaks</a> (5825 W Sahara Ave Unit M. - +1 702-899-4223). Open till 23.00 Sun.-Thurs.<br>It is about a 17 minute drive from the Venetian</li>
<li><a href="http://www.anisetapas.com/" target="_blank">Anise Tapas and Grill</a> (3100 S Durango Dr. - +1 702-586-4088). Open till 22.00 Sun-.Thurs.<br>It is about a 20 minute drive from the Venetian<br><i><b><br>Last year a group of us sat down for a late dinner - the restaurant was empty - but the food was good<br><br></b></i></li>
<li><a href="https://www.kingsolomonskosherlv.com/" target="_blank">King Solomon’s Table</a> (4561 W Flamingo Rd. - +1 725-244-4034). Open till 22.00 Sun.-Thurs.<br>It is about a 10 minute drive from the Venetian</li>
<li><a href="http://www.haifarestaurant.com/" target="_blank">Haifa Restaurant</a> (900 E Karen Ave # H102 - +1 702-940-8000). Open till 21.00 Sun.-Thurs.<br>It is about a 11 minute drive from the Venetian<br><br><i><b>Place is in the middle of nowhere - was practically empty - and the food was nothing special<br><br></b></i></li>
<li><a href="https://www.jerusalemgrillvegas.com/" target="_blank">Jerusalem Grill & Bar</a> (4825 W Flamingo Rd. Suite 10 - +1 702-341-5555). <br>Open till 22.30 Sun.-Thurs. It is about a 11 minute drive from the Venetian.<br><br><i><b>I had dinner there (twice) - and the food was really good!<br><br></b></i></li>
<li><a href="http://www.sababavegas.com/" target="_blank">Sababa Grille & Restaurant</a> (3220 South Durango Dr. - +1 702-547-5556).<br>It is about a 20 minute drive from the Venetian</li>
<li><a href="http://shawarmavegas.com/" target="_blank">Shawarma Vegas</a> (2521 S Fort Apache Rd. - +1 702-703-7700). Open till 21.00 Sun.-Thurs.<br>It is about a 25 minute drive from the Venetian<br><br><b><i>Shawarma Place - Fast food - was great for a quick meal<br><br></i></b></li>
<li><a href="https://simonjoestrattoria.eat24hour.com/" target="_blank">Simon & Joe’s</a> (3720 W Tropicana Ave. - +1 702-759-0333). Open till 21.30 Sun.-Thurs.<br>It is about a 10 minute drive from the Venetian.</li>
</ul>
<br>If you are looking for a list of Kosher products - the list from the <a href="http://ahavastorahcenter.org/local-information/henderson-kosher-food-directory-2/" target="_blank">Ahavas Torah Center</a> has a substantial amount of information.<br><h3>
<br></h3>
<h3>
Shabbat</h3>
<div><br></div>
The conference ends on Friday at around 12:00 which means for most of us that are visiting from outside of the States - that you either leave early - to get back home on time, or fly to family / friends somewhere else in America, or you stay in Vegas for Shabbat. <br><br>The Strip is of course not a Shabbat-friendly atmosphere - and there are a number of Jewish Orthodox (I am sure there are other denominations as well - I will only list the ones that I would go to) communities in the area.<br><br>If you so wish - many of them have some option of Shabbat hospitality as well<br><ul>
<li><a href="http://ahavastorahcenter.org/shabbos/" target="_blank">http://ahavastorahcenter.org/shabbos/</a></li>
<li><a href="https://www.chabadofhenderson.org/templates/articlecco_cdo/aid/490952/jewish/Hotels.htm" target="_blank">https://www.chabadofhenderson.org/templates/articlecco_cdo/aid/490952/jewish/Hotels.htm</a></li>
<li><a href="https://www.chabadlv.org/templates/articlecco_cdo/aid/495929/jewish/Accommodations-for-Shabbos.htm" target="_blank">https://www.chabadlv.org/templates/articlecco_cdo/aid/495929/jewish/Accommodations-for-Shabbos.htm</a></li>
<li><a href="http://www.yiaishlv.org/request-shabbos-hospitality/" target="_blank">http://www.yiaishlv.org/request-shabbos-hospitality/</a></li>
</ul>
<br>(I have personally spent a Shabbat in at the Young Israel community a good number of years ago) There is a hotel that is quite Shabbat friendly - literally 200 meters from the shul La Quinta Inn - not the best of hotels - but OK for Shabbat, and the community was very nice to invite me for meals.<br><br>Last but not least - there is also an Eiruv - <a href="http://www.lasvegaskollel.org/las-vegas-west-side-eruv" target="_blank">http://www.lasvegaskollel.org/las-vegas-west-side-eruv</a> <br><br>As we did last year - we have a WhatsApp group with those who are interested in meeting up for meals after a long day, or perhaps organizing a Minyan for Mincha - or just even to say hello.<br><br> <a href="https://chat.whatsapp.com/IR8hAlhj9vKKG3pWN2SWJQ" target="_blank"><img width="183" height="141" title="Kosher_re:Invent" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="Kosher_re:Invent" src="https://maishsk.com/blog/images/Keeping-Kosher-at-reInvent-2018_6BD4/image_3.png" border="0"></a><br><br><a href="https://chat.whatsapp.com/IR8hAlhj9vKKG3pWN2SWJQ" target="_blank">https://chat.whatsapp.com/IR8hAlhj9vKKG3pWN2SWJQ</a><br><br>Currently there are about 20 people (mostly Israelis - open to all!) <br><br><div><br></div>
<div>
Looking forward to see some old faces and new ones as well next month!</div>
</div>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-48890696068225471332018-10-15T12:00:00.000+03:002018-10-15T12:00:02.084+03:00How Long Until you Get the New Shiny Toys from re:Invent?<p>re:Invent is coming - and the frenzy of releases that will build up to the event is just around the corner.</p>
<p>I have always had in the back of my mind that all the products announced at re:Invent are great for the press releases and the small digs at other vendors, but sometimes it takes a while until we actually get what was announced on stage in front of ~20,000 people and the rest of the world.</p>
<p>And I went out to look for some data. It is obvious that not everything that we heard about on stage was baked and ready for production use.<br>
</p><p><span style="color: black; font-family: arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre; background-color: transparent;"></span></p><p><a href="https://maishsk.com/blog/images/How-Long-Until-you-Get-the-New-Shiny-Toy_7C3A/image.png"><img width="640" height="358" title="Andy Jassy - re:invent 2017 keynote" style="border: 0px currentcolor; border-image: none; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;" alt="Andy Jassy - re:invent 2017 keynote" src="https://maishsk.com/blog/images/How-Long-Until-you-Get-the-New-Shiny-Toy_7C3A/image_thumb.png" border="0"></a></p>
<p>Here are some examples from last years re:Invent </p>
<h3><br></h3><h3>re:Invent 2017</h3>
<p>EKS (188 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/amazon-elastic-container-service-for-kubernetes/">https://aws.amazon.com/blogs/aws/amazon-elastic-container-service-for-kubernetes/</a><br><a href="https://aws.amazon.com/blogs/aws/amazon-eks-now-generally-available/">https://aws.amazon.com/blogs/aws/amazon-eks-now-generally-available/</a> (June 5, 2018)</p>
<p> <br>
Bare Metal (170 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/new-amazon-ec2-bare-metal-instances-with-direct-access-to-hardware/https://aws.amazon.com/about-aws/whats-new/2018/05/announcing-general-availability-of-amazon-ec2-bare-metal-instances/ (May 17, 2018) ">https://aws.amazon.com/blogs/aws/new-amazon-ec2-bare-metal-instances-with-direct-access-to-hardware/<br><a href="https://aws.amazon.com/about-aws/whats-new/2018/05/announcing-general-availability-of-amazon-ec2-bare-metal-instances/">https://aws.amazon.com/about-aws/whats-new/2018/05/announcing-general-availability-of-amazon-ec2-bare-metal-instances/</a> (May 17, 2018) </a></p>
<p> <br>
Serverless App repo (83 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/aws-serverless-app-repo/">https://aws.amazon.com/blogs/aws/aws-serverless-app-repo/</a><br><a href="https://aws.amazon.com/blogs/aws/now-available-aws-serverless-application-repository/">https://aws.amazon.com/blogs/aws/now-available-aws-serverless-application-repository/</a> (Feb 21, 2018)</p>
<p> <br>
Neptune (183 days)</p>
<p><a href="https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-neptune-fast-reliable-graph-database-built-for-the-cloud/">https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-neptune-fast-reliable-graph-database-built-for-the-cloud/</a><br><a href="https://aws.amazon.com/blogs/aws/amazon-neptune-generally-available/">https://aws.amazon.com/blogs/aws/amazon-neptune-generally-available/</a> (May 30, 2018)</p>
<p> <br>
Aurora Multi-master (<strong>Still not released</strong>) </p>
<p><a href="https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/">https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/</a><br>Yet to be released (Oct 14, 2018)</p>
<p> <br>
Aurora Serverless (254 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/in-the-works-amazon-aurora-serverless/">https://aws.amazon.com/blogs/aws/in-the-works-amazon-aurora-serverless/</a><br><a href="https://aws.amazon.com/blogs/aws/aurora-serverless-ga/">https://aws.amazon.com/blogs/aws/aurora-serverless-ga/</a> (Aug 9, 2018)</p>
<p> <br>
IOT 1-click (169 days)</p>
<p><a href="https://aws.amazon.com/about-aws/whats-new/2017/11/aws-iot-one-click-now-in-preview/">https://aws.amazon.com/about-aws/whats-new/2017/11/aws-iot-one-click-now-in-preview/</a><br><a href="https://aws.amazon.com/about-aws/whats-new/2018/05/aws-iot-1-click-generally-available/">https://aws.amazon.com/about-aws/whats-new/2018/05/aws-iot-1-click-generally-available/</a> (May 16, 2018)</p>
<p> <br>
Translate (127 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/introducing-amazon-translate-real-time-text-language-translation/">https://aws.amazon.com/blogs/aws/introducing-amazon-translate-real-time-text-language-translation/</a><br><a href="https://aws.amazon.com/blogs/aws/amazon-translate-now-generally-available/">https://aws.amazon.com/blogs/aws/amazon-translate-now-generally-available/</a> (Apr 4, 2018)</p>
<p> <br>
Transcribe (127 days) </p>
<p><a href="https://aws.amazon.com/blogs/aws/amazon-transcribe-scalable-and-accurate-automatic-speech-recognition/">https://aws.amazon.com/blogs/aws/amazon-transcribe-scalable-and-accurate-automatic-speech-recognition/</a><br><a href="https://aws.amazon.com/blogs/aws/amazon-transcribe-now-generally-available/">https://aws.amazon.com/blogs/aws/amazon-transcribe-now-generally-available/</a> (Apr 4, 2018) </p>
<p> <br>
Appsync (137 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/">https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/</a><br><a href="https://aws.amazon.com/about-aws/whats-new/2018/04/aws-appsync-now-ga/">https://aws.amazon.com/about-aws/whats-new/2018/04/aws-appsync-now-ga/</a> (Apr 13, 2018)</p>
<p> <br>
S3 Select (126 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/s3-glacier-select/">https://aws.amazon.com/blogs/aws/s3-glacier-select/</a><br><a href="https://aws.amazon.com/about-aws/whats-new/2018/04/amazon-s3-select-is-now-generally-available/">https://aws.amazon.com/about-aws/whats-new/2018/04/amazon-s3-select-is-now-generally-available/</a> (Apr 3, 2018)</p>
<h3><br>re:Invent 2016</h3>
<p>Lex (141 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/amazon-lex-build-conversational-voice-text-interfaces/">https://aws.amazon.com/blogs/aws/amazon-lex-build-conversational-voice-text-interfaces/</a> <a href="https://aws.amazon.com/blogs/aws/amazon-lex-now-generally-available/">https://aws.amazon.com/blogs/aws/amazon-lex-now-generally-available/</a> (Apr 19, 2017)</p>
<p> <br>
PostgreSQL for Aurora (329 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgresql-compatibility/">https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgresql-compatibility/</a> <br><a href="https://aws.amazon.com/blogs/aws/now-available-amazon-aurora-with-postgresql-compatibility/">https://aws.amazon.com/blogs/aws/now-available-amazon-aurora-with-postgresql-compatibility/</a> (Oct 24, 2017)</p>
<p> <br>
GreenGrass (190 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/aws-greengrass-ubiquitous-real-world-computing/">https://aws.amazon.com/blogs/aws/aws-greengrass-ubiquitous-real-world-computing/</a><br><a href="https://aws.amazon.com/blogs/aws/aws-greengrass-run-aws-lambda-functions-on-connected-devices/">https://aws.amazon.com/blogs/aws/aws-greengrass-run-aws-lambda-functions-on-connected-devices/</a> (Jun 07, 2017)</p>
<p> <br>
X-Ray (140 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/aws-x-ray-see-inside-of-your-distributed-application/">https://aws.amazon.com/blogs/aws/aws-x-ray-see-inside-of-your-distributed-application/</a> <br><a href="https://aws.amazon.com/blogs/aws/aws-x-ray-update-general-availability-including-lambda-integration/">https://aws.amazon.com/blogs/aws/aws-x-ray-update-general-availability-including-lambda-integration/</a> (Apr 19, 2017)</p>
<p> <br>
Batch (36 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/aws-batch-run-batch-computing-jobs-on-aws/">https://aws.amazon.com/blogs/aws/aws-batch-run-batch-computing-jobs-on-aws/</a><br><a href="https://aws.amazon.com/about-aws/whats-new/2017/01/aws-batch-now-generally-available/">https://aws.amazon.com/about-aws/whats-new/2017/01/aws-batch-now-generally-available/</a> (Jan 5, 2017)</p>
<p> <br>
Lambda Edge (229 days)</p>
<p><a href="https://aws.amazon.com/blogs/aws/coming-soon-lambda-at-the-edge/">https://aws.amazon.com/blogs/aws/coming-soon-lambda-at-the-edge/</a><br><a href="https://aws.amazon.com/about-aws/whats-new/2017/07/lambda-at-edge-now-generally-available/">https://aws.amazon.com/about-aws/whats-new/2017/07/lambda-at-edge-now-generally-available/</a> (Jul 17, 2017)</p>
<p> <br>
At a glance it looks like the average amount of time from the list above was about 5 months.</p><p>Now don’t get me wrong. For all of the above items that were not actually available at re:Invent - I would estimate that there were the same number of products (if not more) that were available (at least in a limited number of regions) the same day they were announced. Above and beyond - the problems that AWS is trying solve and <strong>really complex</strong> - and a almost all of them have never been done before - so please AWS take your time in developing the game changing technology that you have been giving to the world.</p><p>
So when Andy Jassy and Werner Vogels get up on stage at the end of November, and announce whatever wonderful stuff they are going to announce - we should all take into account that it could take anything from 1 day to almost a year until we can actually use it in all the AWS regions that we are consuming today.</p><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><span style="color: black; font-family: arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre; background-color: transparent;"></span></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><span style="color: black; font-family: arial; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre; background-color: transparent;"><br></span></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" dir="ltr"><a href="https://maishsk.com/blog/images/How-Long-Until-you-Get-the-New-Shiny-Toy_7C3A/image_3.png"><img width="640" height="359" title="Werner Vogels - re:invent 2017 keynote" style="border: 0px currentcolor; border-image: none; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;" alt="Werner Vogels - re:invent 2017 keynote" src="https://maishsk.com/blog/images/How-Long-Until-you-Get-the-New-Shiny-Toy_7C3A/image_thumb_3.png" border="0"></a></div>
<b style="font-weight: normal;"><br></b>
<p>How can this / does this affect you? I can give an example from the EKS announcement. We were actively looking at a kubernetes deployment on AWS and were contemplating whether we should deploy our own or wait for the managed solution that was announced at re:Invent. </p>
<p>Since we did not have an official release date - we decided to roll our own - and not wait for some some unknown time in the future.</p>
<p> It is nice to know what is coming. You will need to evaluate how long you can wait - are you ready to go with a version one product (that could / will probably have a good number of limitations) or come up with a contingency plan to solve your issues. </p>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-78237008735399805262018-10-08T21:09:00.000+03:002018-10-09T10:36:18.475+03:00#AWS PrivateLink vs. NAT Gateway from a Pricing Perspective<p>A customer came to me with a request. They do not want to use a NAT gateway from their VPC to access the AWS API's. They had a number of security concerns regarding the use of a NAT gateway (no control, logs, auditing - but that is a for a different post) and they asked for a solution.</p><p>
The AWS API's that they needed access to were:<a href="https://maishsk.com/blog/images/AWS.-NAT-Gateway-from-a-Pricing-Perspect_7DED/image.png"><img width="145" height="123" title="Endpoints" align="right" style="margin: 13px 40px 0px 0px; border: 0px currentcolor; border-image: none; float: right; display: inline; background-image: none;" alt="Endpoints" src="https://maishsk.com/blog/images/AWS.-NAT-Gateway-from-a-Pricing-Perspect_7DED/image_thumb.png" border="0"></a><br>
</p><ul>
<li>S3</li>
<li>KMS</li>
<li>SSM</li>
<li>Cloudwatch</li>
<li>Cloudformation</li>
</ul><p>
Last year at <a href="https://aws.amazon.com/blogs/aws/new-aws-privatelink-endpoints-kinesis-ec2-systems-manager-and-elb-apis-in-your-vpc/" target="_blank">re:Invent</a> AWS announced the option to create VPC Interface endpoints using PrivateLink and have steadily been adding more endpoints over the past year.<br>
</p><p>With the use of these endpoints you can actually have a VPC with instances that will not have any internet access (at least not through AWS) and still be able to interact with all the AWS API's.<br>
</p><p>This is technically possible - and can easily be automated, but I wanted to look at the cost perspective.<br>
</p><p>The VPC in us-east-1 has 2 Availability Zones (you should always have at minimum 2).</p>
<p>That would mean deploying 2 NAT gateways in your VPC (<a style="font-size: 1.1em;" href="https://aws.amazon.com/vpc/pricing/" target="_blank">Pricing</a>)</p><blockquote><p><em>
I am going to assume that you have the same amount of data going through both options - so I will not factor this into the price.<br>
</em></p></blockquote><p>Usually you have 730 hours in a month.<br>
</p><p>Each NAT gateway will cost you 0.045*730 = ~$33.<br>
</p><p><strong>Total for 2 NAT Gateways</strong> would be <strong>$66</strong> per month (not including traffic).<br>
<br>
What does this look like for Interface Endpoints? (<a style="font-size: 1.1em;" href="https://aws.amazon.com/privatelink/pricing/" target="_blank">Pricing</a>)</p><p>
Each Endpoint will need to be deployed in both AZ's in pairs.</p><p>
Each Interace Endpoint will cost 0.01*730*2 = ~15</p><p><strong>
Total for all the endpoints above</strong> (4 Interface Endpoints - KMS, SSM, CloudWatch and Cloudformation) would be<strong> $60</strong> per month. <br>The S3 endpoint is a Gateway endpoint - and therefore does not cost you any extra.<br>
<br>
As you can see - it is not that much cheaper.<br>
</p><p>Take into account the following scenario - you need API access to 15 out of the 21 possible interface <a style="font-size: 1.1em;" href="https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html" target="_blank">Endpoints</a></p><p>
This would run you the steep amount of<strong> $225</strong> per month - which is a lot more than just a NAT Gateway.<br>
</p><p>Design decisions always have tradeoffs - sometimes you prefer security and other times it will be cost.
I hope that this will enable you to make an informed decision in your VPC design.</p>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-19951740790268347022018-10-02T13:00:00.000+03:002018-10-02T13:00:00.870+03:00Bastardizing #DevOpsI have come across two separate discussions this past week where it became clear that some people have no idea what DevOps is. <p>The first one was an Israeli company here in Israel - <a style="font-size: 1.1em;" href="https://devopsexperts.co.il/" target="_blank">https://devopsexperts.co.il/</a>. Here is the proposed syllabus:</p><p><a href="https://maishsk.com/blog/images/Abusing-DevOps_8C42/UNSET.png"><img width="480" height="583" title="[UNSET]" style="border: 0px currentcolor; border-image: none; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;" alt="[UNSET]" src="https://maishsk.com/blog/images/Abusing-DevOps_8C42/UNSET_thumb.png" border="0"></a></p><p><br></p><p>They are offering this course - for a fee (of course), selling the hope that if someone would graduate the course - then they would be able to get a position as an DevOps engineer.</p><p>Someone asked on a channel - "Was this course worthwhile?".</p><p>I would like to share with you my answer. </p><blockquote><p><em><strong>I do not want to take away anyone's livelihood but there is no such a thing a "teaching/learning" DevOps. There is no single course that can encompass all the capabilities that one would need to become a successful DevOps professional. Above and beyond that - in each and every organization - the term DevOps will mean something completely different. </strong></em></p><p><em><strong>There are a number of basic topics that one can learn - and with them build up a strong foundation of skills in order help your specific company. If I would evaluate a potential candidate - and their education was based mainly on this course - I <u>would not</u></strong> <strong>hire such a candidate.</strong></em></p></blockquote>The demand for talented professionals is high , everyone wants DevOps engineers - and there are not many people that have enough experience or the know how. Of course with a demand - people identify an opportunity to make money. <p>Looking at syllabus - it has so many flaws. The course was 45 hours (which means about 1 work week) </p><p><ul><li>Scripting - what language are they going to teach you? Python? But who says that the company you might work for - could be using something completely different.</li><li>Version Control - so this is basically git.</li><li>Linux fundamentals - basic Linux course</li><li>Provisioning Resources - with what? Terraform? Ansible? something else? </li><li>Build Automation - Building a pipeline - with which tools? </li><li>Continuous Monitoring - is that even a concept? </li><li>Working with containers - <font face="Courier New">docker run, docker build, docker pull/push</font></li><li>Configuration Management - use which technology - I can name at least 3 CM tools that you might use</li></ul><p>As you can see, this is a 50,000 ft. view of what you might do in your day to day work as a DevOps engineer - but in no way or form can you learn any of these things in a course - and definitely not in 45 hours. </p><p>For me a good candidate would be someone that has the ability to learn, understands the big picture of how software is built, deployed and managed on a regular basis. There is no list of technologies that could be checked off a list that would qualify a candidate. Does someone know Jenkins? That might be great - but if we use something else - CircleCI, Electric Commander? What will the specific Jenkins knowledge help? </p><p>DevOps is not something that you can learn in school, or in a course. It is a collection of technologies that you collect during the years, it is a state of mind that you become accustomed to as you grow, it is a set of organizational practices that you pick up on your journey. </p><p><strong>Not something you can learn in school.</strong></p><p>Next one was Microsoft - who decided to rebrand VSTS into <a style="font-size: 1.1em;" href="https://thenewstack.io/microsoft-rebranding-leads-to-cloud-native-azure-devops/" target="_blank">Azure DevOps</a>. Again a shiny buzzword which Microsoft assumes will attract people to the product and their offering. </p><a name="more"></a><blockquote style="margin: 0px 0px 20px; padding: 0px; outline: 0px; border: 0px currentcolor; border-image: none; color: rgb(51, 51, 51); line-height: 27.2px; font-family: merriweather, serif; font-size: 16px; vertical-align: baseline; box-sizing: border-box; background-color: rgb(253, 253, 253);">“Azure now has a new set of five DevOps services,” <a class="ext-link" style="margin: 0px; padding: 0px; outline: 0px; border: 0px currentcolor; border-image: none; color: rgb(0, 175, 244); font-family: inherit; font-style: inherit; font-weight: inherit; vertical-align: baseline; box-sizing: border-box; text-decoration-line: none;" href="https://azure.microsoft.com/it-it/blog/author/jamiec/" rel="external ">Jamie Cool</a>, Microsoft’s newly retitled director of product management for Azure DevOps, told The New Stack. “They’re going to help developers be able to ship faster, [<em style="margin: 0px; padding: 0px; outline: 0px; border: 0px currentcolor; border-image: none; font-family: inherit; font-weight: inherit; vertical-align: baseline; box-sizing: border-box;">with</em>] higher quality. Oftentimes when I have conversations, ‘DevOps’ can mean different things to different folks. So to us in this context, we really think of DevOps as the people, the process, and <b>the products for delivering constant value to customers</b>.” </blockquote><p>Here in the statement above is the problem (the emphasis is mine). Products do not deliver DevOps, at least not what Azure is offering. I do agree with the part about the people and the process - but not the products. Maybe the tools - but not products. </p><p>If they would have branded the product <strong><em>Azure CI/CD</em></strong> then I would have been all for it - but to me it seems that this is marketing play - trying to catch a goal that today everyone is trying to achieve. </p>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-89590448454053008492018-09-27T13:53:00.000+03:002018-09-27T17:18:53.277+03:00Replacing the AWS ELB - Final ThoughtsThis is the last part in the Replacing the AWS ELB series. <br>
<ol>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-problem.html" target="_blank">Replacing the AWS ELB - The Problem</a></li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-challenges.html" target="_blank">Replacing the AWS ELB - The Challenges</a> </li>
<ol></ol>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-design.html" target="_blank">Replacing the AWS ELB - The Design</a> </li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-network-deep-dive.html" target="_blank">Replacing the AWS ELB - The Network Deep Dive</a></li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-automation.html" target="_blank">Replacing the AWS ELB - Automation</a></li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-final-thoughts.html" target="_blank">Replacing the AWS ELB - Final Thoughts</a> (this post)</li></ol><p>If you haven't already read the previous posts in the series - please take the time to go through them.</p><p>So here are some additional thoughts and ideas about the whole journey.</p><p>First and foremost - none of this would have been possible without group effort of the team that worked on this. <br><a href="https://www.linkedin.com/in/udi-yaffe-66723382/" target="_blank">Udi</a>, <a href="https://www.linkedin.com/in/mark-prager-921542/" target="_blank">Mark</a>, and <a href="https://www.linkedin.com/in/mike-weeks/" target="_blank">Mike</a> - thank you all for your input, help and hard work that went into this. </p><h3>Was it all worth it?</h3><p>Yes, yes and hell yes!! The cost of having to refactor applications to work with the way that the AWS ELB works - was not financially viable and would take far to long . There was no way we could make our delivery dates and have all the applications modify the way they worked. </p><p>So not only was it worth it - it was a <strong>necessity</strong>, without this - the project was a non-starter. </p><h3>What was the hardest part of the solution?</h3><p>Definitely the automation. We had the solution white-boarded out after a an hour or two, brought up a PoC within another hour or two. </p><p> As I said somewhere else in the post - if this was a one-off then it would not have been worth while - but we needed about 10 pairs of haproxy instances in each deployment - and there were 10- 15 deployments - so manual was not going to work here. There was a learning curve that we needed to get over and that took some time. </p><h3>This can't be all you were doing with haproxy.. </h3><p>Of course not.. The configurations in the examples are really basic and simple. The actual haproxy.cfg was a lot more complicated and was generated on the fly using Consul and <a href="https://github.com/hashicorp/consul-template" target="_blank">consul-template</a>. This allows for some very interesting and wonderful things that can be accomplished. The instances were what could be considered as pets, because they were hardly re-provisioned, but the configuration was constantly changing based on the environment. </p><h3>So did you save money? </h3><p>No! This was more expensive than provisioning an ELB from AWS. The constraints dictated that this was the chosen solution - not cost. Well in a way this was wasted resources, because there are instances that are sitting idle most of the time - without actually doing anything. The master-slave model is not a cost effective solution because you are spending money to address a scenario when (and if) you lose a node. </p><h3>Does this scale? How?</h3><p>We played around with this a bit and also created a prototype that provisioned an auto scaling group with that would work active-active-active with multiple haproxy's - but this required some changes in the way we did our service discovery. This happened a good number of months after we went live - as part of the optimization stage. Ideally - this would have been the way we would have chosen if we could do it over again. </p><p>For this example the only way to scale is to scale up the instances sizes - not to scale out. </p><p>So to answer the question above - in the published form - no it does not.</p><h3>Any additional benefits to rolling your own solution?</h3><p>This could be ported to any and every cloud - or deployment you would like. All you need to do it change the modules and the parts that interact directly with AWS with the cloud of your choice - and it would probably work. It is not a simple rip and replace - but the method would work - just would take a bit of extra time and coding. </p><h3>What about external facing load balancers - will this work? </h3><p>Yes, all you will need to do is replace the routes - with an elastic IP, and have the keepalived script switch the EIP from one instance to another. I should really post about that as well. </p><h3>So why did you not use an EIP in the first place? </h3><p>Because the this was internal traffic. If I was to use an external facing load balancer, the traffic would essentially go out to the internet and come back in - for two instances that were in the same subnet in the same AZ. This does not make sense neither from a financial nor a security perspective. </p><h3>Can I contact you if I have any specific questions on the implementation? </h3><p>Please feel free to do so. You can either leave a comment on any of the posts in the series, ping me on Twitter (<a href="https://twitter.com/maishsk" target="_blank">@maishsk</a>), or use the <a href="https://technodrone.blogspot.com/p/contact-me.html" target="_blank">contact me</a> on the top.</p><h3></h3><ol>
</ol>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.comtag:blogger.com,1999:blog-5819640694385843490.post-60941043308954784072018-09-27T13:47:00.000+03:002018-09-27T17:18:21.495+03:00Replacing the AWS ELB - AutomationThis is Part 5 in the Replacing the AWS ELB series. <br>
<ol><!--StartFragment-->
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-problem.html" target="_blank">Replacing the AWS ELB - The Problem</a></li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-challenges.html" target="_blank">Replacing the AWS ELB - The Challenges</a></li>
<ol></ol>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-design.html" target="_blank">Replacing the AWS ELB - The Design</a></li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-network-deep-dive.html" target="_blank">Replacing the AWS ELB - The Network Deep Dive</a></li>
<li><a href="https://technodrone.blogspot.com/2018/08/replacing-aws-elb-automation.html" target="_blank">Replacing the AWS ELB - Automation</a> (this post)</li>
<li><a href="http://technodrone.blogspot.com/2018/09/replacing-aws-elb-final-thoughts.html" target="_blank">Replacing the AWS ELB - Final Thoughts</a></li>
</ol>
It goes without saying that anything that I have described in the previous posts can be accomplished - it is just a really tedious work to go through all the stages when you are doing this manually.<br>
Let's have a look at the stages<br>
<ol>
<li>Create an IAM role with a specific policy that will allow you to execute commands from within the EC2 instances</li>
<li>Create a security group that will allow the traffic to flow between and to your haproxy instances</li>
<li>Deploy 2 EC2 instances - one in each availability zone</li>
<li>Install the haproxy and keepalived on each of the instances</li>
<li>Configure the correct scripts on each of the nodes (one for master and the other for slave) and setup the correct script for transferring ownership on each instance.</li>
</ol><p>
If you were to to all of this manually then this could probably take you a good 2-3 hours to set up a highly-available haproxy pair. And how long does it take to setup an AWS ELB? Less than 2 minutes? This of course is not viable - especially since it should be something that is automated and something that is easy to use.<br>
This one will be a long post - so please bare with me - because I would like to explain in detail how this exactly works. <br>
First and foremost - all the code for this post can be found here on GitHub - <a title="https://github.com/maishsk/replace-aws-elb" href="https://github.com/maishsk/replace-aws-elb">https://github.com/maishsk/replace-aws-elb</a> (please feel free to contribute/raise issues/questions)</p><p>
(Ansible was my tool of choice - because that is what I am currently working with - but this can also be done in any tool that you prefer).</p><p>
The Ansible <a href="https://github.com/maishsk/replace-aws-elb/blob/master/playbooks/main.yml" target="_blank">playbook</a> is relatively simple<br>
<br><script src="https://gist.github.com/maishsk/1bf4670d0618080d5472960bfe4486fc.js"></script>
Part one has 3 roles. <br></p>
<ol>
<li><a href="https://github.com/maishsk/replace-aws-elb/tree/master/roles/iam_role" target="_blank">Create</a> the IAM role</li>
<li><a href="https://github.com/maishsk/replace-aws-elb/tree/master/roles/secgroup" target="_blank">Create</a> the security group</li>
<li><a href="https://github.com/maishsk/replace-aws-elb/tree/master/roles/ec2" target="_blank">Create</a> the instances </li>
</ol><p>
The <a href="https://gist.github.com/maishsk/1bf4670d0618080d5472960bfe4486fc#file-gistfile1-txt-L26" target="_blank">part two</a> - set's up the correct routing that will send the traffic to the correct instance<br>
The <a href="https://gist.github.com/maishsk/1bf4670d0618080d5472960bfe4486fc#file-gistfile1-txt-L55" target="_blank">part three</a> - goes into the instances themselves and sets up all the software.</p><p>
Let's dive into each of these.<br>
</p><h3>
Part One</h3><p>In order to allow the haproxy instances to modify the route they will need access to the AWS API - this is what you should use an <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html" target="_blank">IAM role</a> for. The two policy files you will need are <a href="https://github.com/maishsk/replace-aws-elb/tree/master/playbooks/files" target="_blank">here</a>. Essentially for this - the only permissions that the instance will need are:<br></p>
<ul>
<li><a href="https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateRoute.html" target="_blank">CreateRoute</a></li>
<li><a href="https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ReplaceRoute.html" target="_blank">ReplaceRoute</a></li>
</ul><p>
I chose to create this IAM role as a <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html" target="_blank">managed policy and not as a inline policy</a> for some reasons that will be explained in a future blog post - both of these work - so you can choose whatever works for you.</p><p>
Next was the security group - and the ingress rule I used <a href="https://github.com/maishsk/replace-aws-elb/blob/master/playbooks/vars/vars.yml#L20" target="_blank">here</a> - was far too permissive - it opens the SG to all ports within the VPC - the reason that this was done was because the haproxy here was used to proxy a number of applications - on a significant number of ports - so the decision was to open all the ports on the instances. You should evaluate the correct security posture for your applications.</p><p>
Last but not least - deploying the EC2 instances - pretty straight forward - except for the last part where I preserve a few bits of instance details for future use.<br>
</p><h3>
Part Two</h3><p><a href="https://github.com/maishsk/replace-aws-elb/blob/master/playbooks/main.yml#L26" target="_blank">Here</a> I get some information about all the rout tables in the VPC you are currently using. This is important because you will need to update the route table entries here for each of the entries. The reason that this is done through a shell script and not an Ansible module - was because the <a href="https://docs.ansible.com/ansible/2.6/modules/ec2_vpc_route_table_module.html" target="_blank">module</a> does not support updates - only create or delete - which would made the process of collecting all the existing entries, storing them and them adding a new one to the list - was far too complicated. This is an Ansible limitation - and a simple way to get around it.<br></p>
<h3>
Part Three</h3><p>
So the instances themselves have been provisioned. The whole idea of VRRP presumes that one of the nodes is a master and the other is the slave. The critical question is how did I decide what should be the master and which one would be the slave? </p><p>
This was done <a href="https://github.com/maishsk/replace-aws-elb/blob/master/playbooks/main.yml#L20" target="_blank">here</a>. When the instances are provisioned - they are provisioned in a random order, but they have a sequence in which they were provisioned - and it is possible to access this sequence - from <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/ec2/tasks/main.yml#L72" target="_blank">this</a> fact. I then exposed it in a simpler form <a href="https://github.com/maishsk/replace-aws-elb/blob/master/playbooks/main.yml#L20" target="_blank">here</a> - for easier re-use.</p><p>
<a href="https://maishsk.com/blog/images/7f09ef70e8af_DD8D/image.png"><img width="297" height="107" title="facts" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="facts" src="https://maishsk.com/blog/images/7f09ef70e8af_DD8D/image_thumb.png" border="0"></a></p><p>
Using this <a href="https://github.com/maishsk/replace-aws-elb/blob/master/playbooks/main.yml#L60" target="_blank">fact</a> - I can now run some logic during the software installation based on the identity of the instance. you can see how this was done <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/haproxy/tasks/main.yml#L67" target="_blank">here</a>.</p><p>
<a href="https://maishsk.com/blog/images/7f09ef70e8af_DD8D/image_3.png"><img width="387" height="271" title="identity" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="identity" src="https://maishsk.com/blog/images/7f09ef70e8af_DD8D/image_thumb_3.png" border="0"></a></p><p>
The other part of where the identity of the node is used is in the <a href="https://github.com/maishsk/replace-aws-elb/tree/master/roles/haproxy/templates" target="_blank">jinja templates</a>. the IP address of the node is injected into the <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/haproxy/templates/keepalived-master.j2#L12" target="_blank">file</a> based on the identity.</p><p>
And of course the <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/haproxy/tasks/main.yml#L76" target="_blank">script</a> that the instance uses to update the route table uses facts and variables collected from different places throughout the playbook.</p><p>
<a href="https://maishsk.com/blog/images/7f09ef70e8af_DD8D/image_4.png"><img width="826" height="201" title="bash_script" style="border: 0px currentcolor; border-image: none; display: inline; background-image: none;" alt="bash_script" src="https://maishsk.com/blog/images/7f09ef70e8af_DD8D/image_thumb_4.png" border="0"></a></p><p>
One last thing of course. The instance I used was the Amazon Linux - which means that the AWS cli is pre-installed. If you are using something else - then you will need to install the CLI on your own. The instances of course get their credentials from the IAM role that is <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/ec2/tasks/main.yml#L24" target="_blank">attached</a>, but when running an AWS cli command - you also need to provide an AWS region - otherwise - the command will fail. This is done with jinja (again) <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/haproxy/tasks/main.yml#L40" target="_blank">here</a>.</p><p>
One last thing - in order for haproxy to expose the logs - a few <a href="https://github.com/maishsk/replace-aws-elb/blob/master/roles/haproxy/tasks/main.yml#L85" target="_blank">short commands</a> are necessary.<br>
Here you have a fully provisioned haproxy pair that will serve traffic internally with a single virtual IP.</p><p>Here is asciinema recording of the process - takes just of 3 minutes</p><p><br><script src="https://asciinema.org/a/203340.js" id="asciicast-203340" async></script></p>
<p>In the <a href="http://technodrone.blogspot.com/2018/09/replacing-aws-elb-final-thoughts.html" target="_blank">last post</a> - I will go into some of the thoughts and lessons learned during this whole exercise.</p>Maish Saidel-Keesinghttp://www.blogger.com/profile/04421762433235332489noreply@blogger.com