Kubernetes has become the default choice for container orchestration. It allows organizations to deploy, manage, and scale their containerized applications, providing many benefits including scalability, availability, reliability, and agility.

However, while Kubernetes has become a key component of the technology stack for building and deploying modern applications, keeping Kubernetes-related costs under control has become a significant challenge. The cost of running Kubernetes includes two primary components:

1. Actual expenditure of running Kubernetes clusters that include compute, storage, networking, and other infrastructure costs
2. Operational costs of managing clusters

In this blog post, we’ll explore the various factors that can impact the cost of using Kubernetes and provide tips and best practices for Kubernetes cost management to keep your cloud bills under control.

Kubernetes cost management challenges

Kubernetes infrastructure brings unique challenges to cost management. Most of these challenges are related to the complexity of Kubernetes and its usage. For example, containerized applications deployed in Kubernetes use various resources such as pods, deployments, ingress, persistent volumes, and namespaces. Calculating the cost for applications involves looking at the usage metrics of all of these resources at a granular level.

In addition, Kubernetes applications are often spread across multiple clusters, business units, data center and cloud environments, and application teams, or the clusters themselves may be shared among multiple business units or application teams. This level of additional complexity makes it hard to track (and assign) costs.

Though many organizations already have a cost management solution, it’s essential to have a solution that natively supports Kubernetes cost management, as Kubernetes infrastructure adds additional challenges to cost control.

To operate Kubernetes infrastructure most cost-effectively, organizations need to employ various techniques and practices. Some of these include:

• Right-sizing cloud instances and application resources
• Using Kubernetes multi-tenancy wherever possible
• Implementing granular Kubernetes cost visibility and monitoring
• Cost optimization policies
• Reducing operational overhead in managing Kubernetes infrastructure
• Adopting a Kubernetes cost management solution

The following sections will delve into these six best practices.

Right-size your cloud instances

One of the first and foremost steps when setting up your Kubernetes infrastructure is to understand the resource requirements of each application. To avoid costly overprovisioning, as well as the adverse impacts of underprovisioning, it’s essential to profile the resource needs of each application. Then you can choose the resources that best fit your application requirements.

Public cloud instances are optimized for different workloads (e.g., compute, memory, or GPU). Hence, choosing the right instance type for your application based on application characteristics is critical.

You can explore spot instances for batch processing, continuous integration, testing environments, and other bursty or ad hoc workloads. Leveraging spot instances can provide significant cost savings, but you must thoroughly analyze the ideal workloads to run on spot instances.

It’s equally important to profile applications to understand minimum and peak CPU and memory requirements of all of the services that run in Kubernetes infrastructure. Based on the profiling data, you can configure the correct requests (minimum) and limits (peak).

Similarly, you should adopt Kubernetes Horizontal Pod AutoScaling (HPA) and Vertical Pod AutoScaling (VPA) to scale your application resources, starting with minimum resources and increasing them as usage grows. You might also explore the advanced cluster autoscaler, Karpenter, for scaling your Kubernets clusters. Karpenter can scale out the cluster when the load increases and scale in the cluster as the load decreases, reducing costs.

Take advantage of Kubernetes multi-tenancy

Clusters are a fundamental resource in the Kubernetes infrastructure. You can deploy clusters in two ways, dedicated and shared. Dedicated clusters are typically deployed for a single application, environment, or team. Shared clusters are distributed across applications, teams, business units, etc. Deciding when to deploy clusters in a dedicated or shared model is critical for managing costs, as dedicated clusters incur significantly higher costs than shared clusters.

Here are a few scenarios in which dedicated clusters are deployed:

• Application has low latency requirements (i.e., target SLA/SLO is significantly higher than others), thus any potential noisy neighbor problems must be avoided
• Application has unique needs (e.g., CNI plugin, GPU worker nodes)
• Based on the type of environment (i.e., dedicated clusters in the production environment and shared clusters in stage and test environments)

Except for these specific use cases where dedicated clusters are needed, it’s a good idea to standardize on a shared cluster model. Kubernetes natively supports multi-tenancy by way of namespaces. However, you must do additional hardening from a security and governance perspective to prepare clusters for multi-tenant deployment.

Additional cluster hardening steps include:

• Deploying and managing cluster-wide services that are used by all applications running the cluster
• Application and namespace-level quota management
• Network policies for namespace isolation
• Security and governance policies leveraging tools such as Open Policy Agent
• SSO and RBAC for secure controlled access to the shared clusters

Provide cost visibility to stakeholders

“You can’t effectively manage what you can’t measure” holds true for Kubernetes cost management. Regularly monitoring the resource usage of your services and applications can help identify the components that are consuming more resources and help you optimize them to reduce costs. You can use Kubernetes dashboards and monitoring tools to track resource usage and identify areas for improvement.

You can gain insights from the usage metrics to optimize the resource limits and usage quotas of your applications. This optimization will ensure that your applications consume resources based on their needs and allocations, preventing overspending on resources.

Similarly, you can configure budget thresholds to provide early warnings when costs exceed certain limits. These thresholds can act as guardrails to provide the necessary financial discipline to the Kubernetes infrastructure teams. It’s also critical to provide cost visibility to individual business units, development teams, application owners, and other teams. Cost transparency helps create financial discipline and accountability among stakeholder teams, and provides them with both the insights and the motivation to find additional ways to reduce costs.

Implement cost optimization policies

Implementing cost optimization policies to delete unused and under-utilized resources can greatly reduce Kubernetes infrastructure costs. For example, public cloud-managed Kubernetes distributions such as Amazon EKS support shutting down all of the worker node groups while running the control plane. For long-running clusters used for UAT (user acceptance testing) or preview deployments, you can build automation to bring down worker node groups during weekends and other off hours, and quickly bring them back when needed, while keeping all of the configuration and data intact. 

Similarly, sandbox and developer environments can be brought down automatically during off hours to clean up unused resources. Tags and labels can be employed to affix owners, environments, and expiration times to the resources that can be utilized in the cleanup policies. Tags can also be used to exclude certain resources if needed.

It’s also a good practice to limit the allowed regions to a select few as the cost could vary by region. Similarly, you should enable only the instance types needed for your applications and restrict all other types. You can create standardized templates that use optimal resources and share them with your users to create self-serve environments.

In addition, you can set up automated policies to clean up any unused and dangling resources.

Don’t overlook indirect management and maintenance costs

Management and maintenance costs associated with the Kubernetes infrastructure are often overlooked. This indirect expense can become a major chunk of your total Kubernetes expenditure, especially if you manage and operate a reasonably large-scale Kubernetes infrastructure. Kubernetes management and maintenance tasks include:

• Creating new Kubernetes clusters for production and non-production environments
• Deploying additional add-ons required at the cluster level
• Configuring required security policies for the clusters
• Setting up logging and monitoring
• Setting up Kubernetes RBAC for end users
• Deploying applications
• Performing Kubernetes version upgrades
• Performing add-on version upgrades
• Setting up backup and restore for disaster recovery
• Troubleshooting and resolving infrastructure issues

Besides the above tasks, Kubernetes SRE, operations, and platform teams perform many other activities to manage and maintain infrastructure. Performing these tasks manually may result in huge operational costs for the organization. Automating these tasks will not only substantially reduce the costs but also improve the developer experience and accelerate product delivery times.

A number of Kubernetes operations platforms provide turnkey automation for managing Kubernetes infrastructure. It’s worth exploring these platforms to manage and maintain your Kubernetes infrastructure, as building the automation in-house can also be very expensive.

Use purpose-built Kubernetes cost management tools

Given the complexity and nuances, leveraging a third-party open-source or commercial tool built specifically for Kubernetes cost management is essential. Such a tool should provide the following features:

• Consolidated view of all Kubernetes costs across clusters, teams, business units, applications, and environments
• Granular visibility into cost metrics by namespace, pod, label, etc.
• Chargebacks and cost allocations for the FinOps team to distribute costs across teams
• Long-term retention of metrics to predict future costs
• Integrated RBAC to provide respective cost insights to individual teams

You can use a native Kubernetes cost management tool to configure appropriate budget thresholds, chargeback groups, and other cost control policies.

Cost management is a critical factor for successful Kubernetes deployments. Organizations must invest significant time and care into developing a cost management strategy. Due to the inherent complexity of Kubernetes, a Kubernetes-specific cost management solution must be used to handle the use cases specific to Kubernetes.

From selecting right-sized instances to monitoring Kubernetes resource usage and costs at a granular level, following the best practices outlined in this article will help you ensure that costs stay under control. A dedicated Kubernetes cost management tool can help you provide the visibility into costs and establish the necessary financial governance, enabling FinOps teams to implement adequate cost controls across the organization.

Hemanth Kavuluru is co-founder and SVP of engineering at Rafay Systems, a leading platform provider for Kubernetes operations.

—

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.