Unit metrics, a stepping stone to sustained cloud cost efficiency

What is a cloud cost unit metric? Why is it important?

Unit metrics represent incremental costs relative to a technical or business cost driver. They help teams and organizations understand the value they’re getting out of the cloud. 

If you’re a growing software company, you want absolute cloud costs to be increasing. Your software is being used more and (presumably) providing more value to your customers. This said, you typically want your unit metrics (e.g. $/customer) to be steady or even decreasing. Thinking about cloud cost optimization in terms of absolute numbers or setting KPIs with absolute numbers can be counterproductive and can create unhealthy incentives for engineering and product teams. You want your cloud costs to double and triple as you grow. You don’t want your unit metrics to double and triple as you grow, or soon you’ll be losing money for each customer you serve!

For a deeper financial description of what a unit metric is, see AWS’s description here. For a great case study of unit metrics in action, check out Maciej Zbierski’s insightful article on Sumo Logic’s approach here.‍

Brief semantic note: when it comes to discussions of cloud costs, “unit metrics”, “unit costs”, and “unit economics” are typically used interchangeably.

Types of unit metrics:

Universal engineering unit metrics

Examples

“$ / Lambda invocation”, “$ / EC2 instance”, “$ / GB stored in S3”

Definition

A unit metric for each AWS service that combines cost and application/infrastructure monitoring data and exists universally without requiring environment specific information (e.g. tags).

Pros‍‍

• ‍Turnkey - with access to cost and relevant application/infrastructure monitoring data, they can be created instantly without user input‍
• Benchmarking - their universality creates opportunities for benchmarking applications/teams/accounts within organizations and across companies‍
• Engineering specific - can be created for any engineering use case regardless of what part of the business that team serves

Cons

• Superficial - they give a good quick gut check but don’t necessarily help troubleshoot where cost changes are coming from. It’s unlikely that universal engineering unit metrics can be used to forecast costs reliably‍
• Engineering specific - while these can be a powerful tool for FinOps teams to create KPIs and surface issues, these aren’t metrics that financial/business teams can easily relate to their work
• Apples to oranges - Some workloads have different architectural and operational requirements which create natural benchmarking differences. $ / EC2 Instance for the g5.xlarge you're using for an Ethereum mining operation is of course going to be higher than your smaller instance's compute-light use case.

Custom engineering unit metrics

Examples

“How much does Team X spend for each ELB request they make?”, “How much is cost center X spending for each GB stored in Account Y?”

Definition

A unit metric that combines cost and application/infrastructure monitoring data and uses environment specific information (e.g. tags).

Pros

• Customized - they can be tailored to how your organization thinks about cost centers and application/infrastructure metrics. You can likely use these to forecast and these will be forecasts that engineering owners are empowered to make (because they use engineering metrics as forecast input)‍
• Engineering specific - they can be created for any engineering use case regardless of what part of the business that team serves

Cons

• Engineering specific - these are an even better tool for FinOps teams to create KPIs and surface issues but they still aren’t metrics that finance/business teams can easily relate to their work 

Business unit metrics

Examples

“$ / order”, “$ / ride” (Lyft use case, built by our very own Ilia!), “$ / game” (Wildlife use case)

Definition

A unit metric that combines cost and business data, typically a business metric that’s correlated to a company’s 1) revenue model and 2) infrastructure cost drivers.

Pros

• Business specific - they typically are related to revenue drivers and intuitive for business and finance teams. This makes them valuable for financial reporting, profit analysis, pricing products, and forecasting 

Cons

• Business specific - although easily relatable for finance/business teams, engineering teams not working on something related to the business metric can feel like the business unit metrics doesn’t apply to their work 
• More complex - requires combining cost data and business/financial data which often lives in different places, requires different access rights, and needs business context to combine accurately

How unit metrics are a stepping stone to engineering cost ownership

So why should your organization care about unit metrics?? At Cloudthread, we believe that in order for you to achieve sustained cloud cost efficiency, you need to create a culture of engineering team cost ownership - this is exactly where unit metrics, specifically engineering unit metrics, can help!

Foundation for KPIs - Alerts, budgets, and anomaly detection that are inline with the value you’re getting from the cloud

Alerts and budgets on absolute costs can be misleading or create unhealthy incentives. If you’re going through a seasonal growth in usage/costs, absolute number alerts will be triggered, application owners will shrug and blame the increase in usage, and your organization will have zero idea whether the value you’re getting from the cloud has been consistent. An alert on a custom engineering unit metric will make changes in value you’re getting from the cloud instantly apparent and can be used to monitor whether cost efficiency is going up or down over time regardless of changes in absolute numbers.

Benchmarking - Easily understand how a team is performing relative to other teams and organizations. 

Universal engineering unit metrics can be easily created, used to benchmark, and provide a quick gut check on where there might be easily fixable issues. Team A has a “$ / GB stored” that’s 10x the organization average? Check and see if this is a known result of their operations or if they can easily implement Amazon S3 Intelligent-Tiering and save.

Customizable and relatable - Can flexibly be applied to different teams and use cases creating cost driver signal amidst cost data noise

While a top down business unit metric can be very useful for financial reporting needs (e.g. $/ride at Lyft), it can be unrelated to what an engineering team is working on and feel useless for understanding value and forecasting. Custom engineering unit metrics can be tailored to different engineering use cases to understand cost spikes or forecast costs.

From his experience at Lyft, Ilia explains:

“having $/ride was a powerful tool for finance to report on infrastructure costs and forecast at a high level, but this metric was often senseless for engineering leaders. They were manually creating unit metrics with the initial cost data we provided so they could report cost abnormalities accurately.”

How can Cloudthread help?

Our mission at Cloudthread is to empower engineering teams to build cost efficiently in the cloud. We think unit metrics set the foundation for making all teams fluent in cloud cost efficiency and our platform provides cloud unit metrics as a service. 

Universal engineering unit metrics: These are automatically generated for all services we support - we’re building a library of engineering unit metrics for each AWS service so that anybody can use them instantly.

Custom engineering unit metrics: We provide a Unit Metric Constructor so that your team can create engineering unit metrics that are meaningful to your business. 

Business unit metrics: We’re building connectors to business intelligence data warehouses to make creating business unit metrics something easy and intuitive - coming soon publicly!

Get in touch!

We’re always looking for new, creative ways that people segment their costs in ways that are meaningful for them. If you have an interesting unit metric use case you’ve found valuable, or if there’s a unit metric you’ve been trying to create for your team that you think we could help with, don’t hesitate to drop us a line at hey@cloudthread.io.