Modern Apps and Trusted Storage
During the German .next on Tour in November I gave a talk called “Nutanix Unified Storage – Data Services for the modern age”. The talk was about how NUS (but also Nutanix in general) gives us all the data storage services we need for even the newest kind of applications.
The talk walks through a journey of a cluster running VMs on AHV + AOS, to running an AI chatbot application on the same cluster, just by leveraging K8s and NUS. While the talk was well received, 45 minutes are not enough to cover all of persistent storage for modern applications. However, multiple blogs posts can 😊
Modern Applications, Cloud Native, K8s
What do we mean with modern apps, cloud native, etc.? And more importantly, why do they matter? In our world where software powers everything from streaming our favorite shows to running AI applications, these terms represent the blueprint for building and running applications that can keep up with the pace of modern life.
Modern applications are all about being adaptable, scalable, and efficient. Gone are the days of only running big, bulky monolithic applications and servers (remember Windows Small Business Server anyone?). Today's applications are often built using microservices, small independent pieces that work together. This approach makes development oftentimes faster and lets apps grow and shrink to handle demand without breaking a sweat.
And then there’s “cloud native”, a term that often gets mistaken for “built in / for the public cloud”. In my opinion it’s not that. Being cloud-native is a way of designing apps to be flexible, automated and portable across any environment – whether it is a private data center, a public cloud (aka someone else's server), or a mix of both. It emphasizes using modern infrastructure without “being married to one”.
This is also where Kubernetes (“K8s”) comes in. K8s is the enabler behind these cloud-native apps. Think of it as the conductor of an orchestra, making sure every container (the building blog of modern apps) works together in harmony. Though it is the leading container orchestrator today and supports many different types of applications, it started out focused on running stateless apps.
The Past – Overcoming Early Kubernetes Storage Hurdles
K8s is all about managing containers. From the beginning, it provided a straightforward way to deploy and scale stateless apps, where containers could spin up, do their job, and disappear without worrying about saving data. For developers focused on apps like web servers, this worked perfectly. However, enterprises require data to persist for other application services like databases.
In those early days, K8s did not have a native way to handle storage. If your app needed to store data, you had to provision storage manually. This meant creating and configuring storage systems yourself, then finding ways to connect them to your containers. I have been told it worked, but wasn’t smooth, and certainly not automated.
The introduction of Persistent Volumes (PVs) and Persistent Volume Claims (PVCs) helped by separating storage from workloads, but the process was still static – storage had to be pre-allocated. Dynamic provisioning improved things further, allowing Kubernetes to create storage on demand, but integration remained an issue. Each storage vendor needed custom plugins, leading to complexity and maintenance headaches.
kubelet talking to external storage with vendor plugins in the k8s core
The turning point came with the Container Storage Interface (CSI), a standardized API that allowed vendors to build plugins compatible with almost any Kubernetes environment. CSI simplified integration, making it easier to use dynamic, scalable storage across systems.
With these advancements, Kubernetes grew from supporting only stateless apps to becoming a platform capable of handling stateful workloads. These incremental improvements transformed Kubernetes into the foundation for modern applications.
The Present – Kubernetes Storage All Grown Up
Since its humble beginnings, K8s has grown into a platform that handles stateful workloads with ease. What once needed duct tape and manual effort is now part of its core functionality, thanks to advancements like the Container Storage Interface (CSI). CSI has simplified how K8s integrates with storage systems by moving volume provisioner plugins out of the K8s core. In the past, these “in-tree” plugins were tied to K8s versions, forcing storage vendors to keep up with K8s rapid release cadence. This slowed innovation and was messy to maintain. CSI solves this by creating a standardized interface that allows vendors to build and maintain plugins with reduced dependency on K8s releases.
K8s using csi drivers to communicate with storage
This evolution made K8s a reliable choice for stateful workloads. Tools like “StatefulSets” now give developers a native way to manage applications that need persistent data, such as databases or analytics apps.
K8s also shines in hybrid and multi-cloud setups. The storage abstraction allows workloads to use on-premises storage alongside cloud-based solutions. This flexibility means organizations can choose the best storage for each workload, while keeping everything under a single K8s umbrella.
Security and automation have stepped up, too. Features like encryption and replication (either built into CSI backed storage, or via 3rd party) ensure data is both safe and resilient. Tools like operators and Helm charts make deploying stateful apps with K8s easier than ever.
Today, K8s is not just a platform for containers – it is a full-fledged system for running modern applications, regardless of storage requirements.
The Future - A Glance Ahead
As K8s continues to evolve, the Special Interest Group Storage (SIG Storage, https://github.com/kubernetes/community/blob/master/sig-storage/README.md) is actively working on several initiatives to enhance storage (and surrounding) capabilities. Here is a look at the key project I am most excited about:
Change Block Tracking (CBT)
Well known for backup and virtualization admins, this crucial capability is still missing for Kubernetes. While some third-party solutions have implemented their own CBT mechanisms for specific use cases (e.g. VEEAM K10 and VMware CBT), these are platform specific and not natively K8s integrated. In general, CBT allows systems to process only the modified portions of data rather than entire volumes. By implementing CBT, K8s will improve data protection workflows, making backups faster and more efficient. It is worth noting that CBT is first planned as an alpha feature in v1.33, so it may take a few release cycles before it is production-ready.COSI – Container Object Storage Interface
COSI was introduced as an alpha feature in v1.25 in 2022 and is a standard for provisioning and consuming object storage in K8s. As far as I know, it still is an alpha feature, but some vendors already have plugins available. For example, Nutanix has a COSI driver for NUS Objects on GitHub: https://github.com/nutanix-cloud-native/cosi-driver-nutanix. The decision for a new storage interface was probably due to the big differences between file- and block storage vs object storage. While file/block storage returns volumes, mount paths, etc., object storage provides access and secret keys alongside a bucket path. This separation from CSI simplifies object storage integration in my opinion.
Persistent Storage as a Foundation for the Future
The journey of K8s from a platform for stateless workloads to a platform for modern applications has been remarkable (and one I should have followed closer sooner). Persistent storage has played an important role in this transformation, enabling K8s to support the evolving needs of stateful workloads like databases and our beloved AI Driven applications.
Looking ahead, technologies like Nutanix Kubernetes Platform and Nutanix Unified Storage are set to take these capabilities even further, making it even easier to handle modern workloads seamlessly. In a future post, we will look into how these technologies build on Kubernetes’ foundation to deliver trusted, scalable, and efficient solutions for the most demanding applications.
Stay tuned!