#BillBI - AWS Billing Business Intelligence - is here to grow…
As we started working with bigger customers, we understood that existing Bill Analytics tools do not provide an adequate solution. We were very surprised by that. We thought that tools like the AWS Cost Explorer, which is constantly improving and other common 3rd party tools provide ample clarity into the AWS bill. Turns out that ain't so… While trying to figure out why, we have come to the understanding that very much like in operation, in the financial management different organizations behave very differently and have different requirements. This led us to build the infrastructure for three different tools targeting three different functions and skill set and organization. We have built dashboards so you can quickly gerd your situation and easily search for anomalies. Here are a few images.
We have also integrated two different BI tools into the platform. One is designed for people that tend towards Excel and simple graphs. It provides the ability to analyze the data in a simple intuitive way. Applying filters, drilling down, displaying graphs, Exporting data and saving your own reports is very simple. It's a very cool tool for Finance people. Here are a few images.
The second is a full-blown BI tool that allows you direct access into an Athena table that contains the CUR. this allows you to do anything good Bri to it would let you do including writing your custom SQL queries and storing them as dashboards for future use. As you can see, we went dark mode on this one.
We use both Cost Explorer and the CUR as data sources to allow visibility even when access to the CUR is not available. We would very much like to get your feedback on using the Various bi tools. Please let us know what you think.
OTP - One Time Password is here.
Well, you know you have to have security. So, as our customers are granting more employees access our product, we thought it is important to increase our security. We have implemented a simple OTP mechanism. All you need to do is go to the settings menu and enable OTP. As the last step of the login process, you will get an email with a code. that's it.
#SnapOnShutDown - Don't pay for storage when your servers are down!
What prompted us to create this tag was seeing customers use huge machines (p3+) with huge disks using maximum IOPS for training AI models. These machines are indeed expensive and the cloud has a huge advantage for customers as they can run very large amounts of machines to quickly train a new model and then shut them down and not pay for them. What we found out is that the "not pay for them part" is not working very well for them. Why? Well mostly because the disks they are using, often multiple of them per server, are not "shutting down" and keep accumulating costs. We have seen this cost amount to 40% of the monthly EC2 cost. So, we built SnapOnSpot to "shutdown" the disks when the machines are down.
How does SnapOnShutDown work? - As usual it's very simple. Tag all the volumes you want to shutdown when the machine is down with SnapOnShutDown and that's it. CloudHawk will now identify the machine is down and automatically do the following: 1. Create a snap of the volume. 2. Detach & Delete the volume. 3. tag the instance with a CHK-SnappedVolume-[VolumeName] tag that will preserve all the information specific to the snapped volume. So what about starting up the instance now? When CloudHawk starts the instance it will automatically create a new volume from the snapshot, re-attach it to the original instance preserving all original settings like volume type, IOPS and encryption, and then start the instance. If you want to start the instance from the AWS console, you can do these steps manually by following the tags on the instance. To start it from the CloudHawk console, just click the start button next in the instance line on the List page.
This tag works excellently with our down & up tags to schedule automatic shutdown and startup of machines at certain times. It is not a good fit for instances that go down for less than 30 minutes at a time. Encryption is preserved throughout the process. If the volume is encrypted, the snapshot will be encrypted & the new volume created will also be encrypted. The same idea applies where relevant to IOPS and other volume properties.
CloudHawk on AWS Marketplace
We are happy to announce that as of today Aug 15th, 2019 CloudHawk is listed on the AWS marketplace. You can subscribe to it here.
More options and offerings coming soon.
#RIUtil - Reserved Instances Utilization Monitor is live!
We are happy to announce our new RI utilization monitor. This feature is designed for our customers that are relying heavily on RIs to optimize their cloud costs. This rule will let you know when your RIs are not utilized. It monitors the last two days and triggers on alert when utilization is < 95%.
You don't have to do anything to activate it. The rule is automatically active on your account. You will receive the notifications to your usual email. You can opt-out from the email or by going to the rules section in the console.
A bit of background. RIs are a way to commit to a certain usage and get a substantial discount. There are many kinds of RIs. We will not cover all of them here. There are 1 year RIs & 3 years RIs. Usually, on Linux servers 1 year will give you an about 40% discount, while 3 years will give you roughly a 60% discount. There are three payment options: All upfront, partial upfront and no upfront. The different payment options will give you different discounts around the average numbers mentioned above. RIs can be convertible to other RIs, or fixed. There are multiple strategies for implementing RIs. Our #RIUtil rule gives you the alerts you need to apply your strategy. This allows you to choose strategies that require more management but give you a significantly higher discount. Implementing your strategy is obviously part of our full service package.
Once you receive the notification you have three options: 1. Convert the RIs, assuming they are convertible. 2. Migrate some machines that are not covered by RIs to the type that is covered by your RI, even if this means you need to scale up your machine. 3. Sell your RIs on the marketplace.
We are happy to announce the availability of SetMDM. This new Auto Scaling Group (ASG) tag is designed to allow you to apply the same operations that you can apply to a single EC2 Instance to an entire ASG. For example this will allow you to "shutdown" the ASG by using a tag like SetMDM-0-0-0-Day-20:00. You can start it up again or simply change capacity. All the operations that work on a single EC2 Instance will work on an ASG except teminate, of course as an ASG has no cost as it's set to 0-0-0.
We are happy to announce the availability of SpotASG. This new Auto Scaling Group (ASG) tag is designed to save you 75% on your EC2 instance cost by taking advantage of spot pricing and the AWS spot auto scaling group feature.
To activate this simply add the SpotASG tag to your on-demand ASG (key=SpotASG, value=SpotASG). CloudHawk will do the rest. Here is how it works:
CloudHawk will create a matching ASG with the same values of minimum, desired and maximum values as your original on-demand ASG. The max price for the spot is set high enough so that the spot instances are terminated only when there is zero availability of these instances. Once the number of the spot instances in your ASG reaches the desired number, CloudHawk will shut down the original ASG. Now you are saving approximately 80% on the on-demand price. If the spot instances are terminated, CloudHawk will automatically restart your on-demand ASG to insure availability of your service.
If your ASG resides behind a load-balancer, you can add both the original and the newly created AWS to the same target group. This will enable load balancing over instances from both Auto Scaling Groups no matter which one of them is currently active.