Skip to main content

Your submission was sent successfully! Close

Thank you for signing up for our newsletter!
In these regular emails you will find the latest updates from Canonical and upcoming events where you can meet our team.Close

Thank you for contacting us. A member of our team will be in touch shortly. Close

  1. Blog
  2. Article

Alex Chalkias
on 4 April 2022

Zero-ops scaling Kubernetes storage with MicroK8s and OpenEBS Mayastor


This is a guest post, originally posted on Angelos Kolaitispersonal blog, reproduced here with his permission. Angelos is a MicroK8s software engineer at Canonical.

How we brought a universal, cluster-wide storage solution to MicroK8s.

Why is storage in Kubernetes hard?

Kubernetes is well-known as an open-source system for automating deployment, scaling and management of containerised applications. As such, it uses concepts such as Pods and Deployments to abstract away details regarding the underlying compute, networking and storage infrastructure.

Storage is abstracted as PersistentVolumes (a volume that is provisioned in the underlying infrastructure) and PersistentVolumeClaims (a claim for a pod that uses a persistent volume) resources. Typically, running a Kubernetes cluster also involves running a CSI provisioner, which watches for PersistentVolumeClaims and automatically provisions the requested volumes.

There are a large number of CSI drivers available. For example, if running Kubernetes on top of AWS, one would most likely use the aws-ebs-csi-driver. If running Kubernetes on-premises or in an OpenStack cloud, cinder-csi-driver would be more appropriate. If running on-premises and a Ceph cluster is available, ceph-csi could also be used.

While this solves the storage problems for Kubernetes users, it moves the hurdle of managing everything to the administrators of the Kubernetes cluster, as well as the Kubernetes distribution.

So, the question is, how can storage be made simpler for the administrator of the Kubernetes cluster?

Introducing Mayastor

Mayastor is currently under-development as a sub project of the Open Source CNCF project OpenEBS.

Mayastor can be deployed on any Kubernetes cluster and provide Kubernetes-native storage. It can also use existing block devices on the Kubernetes cluster nodes (if available), as well as image files.

Further, the Mayastor data plane supports replication. You can replicate your volume data across multiple pools, to ensure redundancy in case of a single node failure.

In general, Mayastor is a very interesting solution for Kubernetes-native, fast, redundant storage that works universally across any Kubernetes cluster.

Sounds interesting! How do I try this out?

Deploy Mayastor

NOTE: For the latest instructions, refer to the mayastor addon page.

The mayastor addon will be available with MicroK8s 1.24. You can already try it out by installing MicroK8s from latest/edge:

  1. Enable nvme_tcp and HugePages:sudo apt-get install linux-modules-extra-$(uname -r) sudo modprobe nvme_tcp sudo sysctl vm.nr_hugepages=1024
  2. Install MicroK8s from latest/edge:sudo snap install microk8s --classic --channel latest/edge
  3. Enable Mayastor:sudo microk8s enable core/mayastor

Deploy a test workload

Now that mayastor is deployed, it is time to test it using an example pod:

# pod-with-pvc.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  storageClassName: mayastor
  accessModes: [ReadWriteOnce]
  resources: { requests: { storage: 5Gi } }
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  volumes:
    - name: pvc
      persistentVolumeClaim:
        claimName: my-pvc
  containers:
    - name: nginx
      image: nginx
      ports:
        - containerPort: 80
      volumeMounts:
        - name: pvc
          mountPath: /usr/share/nginx/html

Create the pod with:

sudo microk8s.kubectl create -f pod-with-pvc.yaml

Then, we can check to see that Mayastor has created our persistent volume, and the pod has come up:

sudo microk8s.kubectl get pod,pvc

The output should look like this:

NAME        READY   STATUS    RESTARTS   AGE
pod/nginx   1/1     Running   0          4m

NAME                           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/my-pvc   Bound    pvc-e280b734-3224-4af3-af0b-e7ad3c4e6d79   5Gi        RWO            mayastor       4m

Conclusion and future steps

The new mayastor addon in MicroK8s is a very simple way to provision fast, replicated and redundant (though not highly-available) persistent volumes in Kubernetes.

The mayastor addon is considered beta as of MicroK8s 1.24 release. You can already try it by installing from the latest/edge track.

Future steps include:

  • Scale and stress testing.
  • Graduate addon from beta to stable status.
  • Refine the mayastor addon. Right now it automatically provisions image-file backed pools. but we want to make the addon easier to configure for MicroK8s clusters with designated block devices for storage.
  • Compare Mayastor with Rook (Ceph) and other Kubernetes native storage solutions.
  • Document and test backup and restore of mayastor volumes.
  • Support arm64 architectures. This is pending support in mayastor

Photo by Darrin Moore on Unsplash

Related posts


Marcin "Perk" Stożek
14 March 2024

How should a great K8s distro feel? Try the new Canonical Kubernetes, now in beta

Kubernetes Article

Try the new Canonical Kubernetes beta, our new distribution that combines ZeroOps for small clusters and intelligent automation for larger production environments that also want to benefit from the latest community innovations ...


Marcin "Perk" Stożek
14 December 2023

Canonical Kubernetes 1.29 is now generally available

Kubernetes Article

A new upstream Kubernetes release, 1.29, is generally available, with significant new features and bugfixes. Canonical closely follows upstream development, harmonising our releases to deliver timely and up-to-date enhancements backed by our commitment to security and support – which means that MicroK8s 1.29 is now generally available as ...


Michael C. Jaeger
9 November 2023

Turbocharge your API and microservice delivery on MicroK8s with Microcks

Ubuntu Article

Give Microcks on MicroK8s a try and experience the benefits of accelerated development cycles and robust testing. ...