Skip to main content

Your submission was sent successfully! Close

Thank you for signing up for our newsletter!
In these regular emails you will find the latest updates from Canonical and upcoming events where you can meet our team.Close

Thank you for contacting us. A member of our team will be in touch shortly. Close

  1. Blog
  2. Article

robgibbon
on 3 July 2023

Charmed Spark beta release is out – try it today


Charmed Spark 3 beta – out now

The Canonical Data Fabric team is pleased to announce the first beta release of Charmed Spark, our solution for Apache Spark.

Apache Spark is a free, open source software framework for developing distributed, parallel processing jobs. It’s popular with data engineers and data scientists alike when building data pipelines for both batch and continuous data processing at scale. Engineers can write Python or Scala code to develop Spark jobs for ETL (extract-transform-load), analytics and machine learning.

Canonical is building a supported, packaged solution for running Spark jobs on Kubernetes. The preview release is the first milestone towards building a comprehensive solution for Spark users. 

The beta release includes features for:

  • Submitting jobs to the cluster
  • Managing job configuration
  • Security maintained container images
  • A software operator to deploy and operate the Spark History Server

Charmed Spark is a part of Canonical Data Fabric, a set of solutions for data processing, with additional solutions to be announced.

Charmed Spack reference architecture

Users can deploy Charmed Spark to MicroK8s, Charmed Kubernetes and AWS Elastic Kubernetes Service (EKS). Read the reference architecture guide:

Charmed Spark 3 release 1 reference architecture guide

Share your feedback

At Canonical, we always value the community’s feedback about our products. We would like to ask you to try out Canonical’s Charmed Spark and send us your comments, bug reports and general feedback so we can include them in our future releases.

To get started, head over to the Charmed Spark documentation pages and install the spark-client snap.

Chat with us at https://chat.charmhub.io/charmhub/channels/data-platform or file bug reports and feature requests in Github.


Related posts


robgibbon
3 May 2023

Big data security foundations in five steps

Data Platform Article

We’ve all read the headlines about spectacular data breaches and other security incidents, and the impact that they have had on the victim organisations. And in some ways there’s no place more vulnerable to attack than a big data environment like a data lake. ...


robgibbon
23 May 2024

Can it play Doom? Running an AI LAN party on a Spark cluster with ViZDoom

AI Article

It’s all about AI these days, so I decided to try and answer the important question: can you make a Spark cluster run AI agents that play a game of Doom, in a multiplayer LAN party? Although I’m no data scientist, I was able to get this to work and I’ll show you how so ...


robgibbon
17 October 2023

Why we built a Spark solution for Kubernetes

Data Platform Article

We’re super excited to announce that we have shipped the first release of our solution for big data – Charmed Spark. Charmed Spark packages a supported distribution of Apache Spark and optimises it for deployment to Kubernetes, which is where most of the industry is moving these days. Reimagining how to work with big data ...