Rob Gibbon

Rob Gibbon

28 posts

Migrating from Apache Spark 3 to Spark 4

The purpose of this guide is to highlight the key differences between Apache Spark 3 and Spark 4, and provide advice on how to plan a migration. Let’s get...

Understanding disaggregated GenAI model serving with llm-d

What is llm-d? llm-d is an open source solution for managing high-scale, high-performance Large Language Model (LLM) deployments. LLMs are at the heart of...

Hybrid search and reranking: a deeper look at RAG

Many of us are familiar with the retrieval augmented generative AI (RAG) pattern for building agentic AI applications – like digital concierges, frontline...

Apache Spark 4.0 beta release – try it now

Apache Spark is a popular framework for developing distributed, parallel data processing applications. Our solution for Apache Spark on Kubernetes has made...

Deploying and scaling Apache Spark on Amazon AWS EKS

Move over Hadoop, it’s time for Spark on Kubernetes Apache Spark, a framework for parallel distributed data processing, has become a popular choice for...

Can it play Doom? Running an AI LAN party on a Spark cluster with ViZDoom

It’s all about AI these days, so I decided to try and answer the important question: can you make a Spark cluster run AI agents that play a game of Doom, in a...

Deploy an on-premise data hub with Canonical MAAS, Spark, Kubernetes and Ceph

Download the Spark reference architecture guide In this post we’ll explore deploying a fully operational, on-premise data hub using Canonical’s data centre...

Migrating from Cloudera to a modern data hub architecture

In the early 2010s, Apache Hadoop captured the imagination of the tech community. A free and powerful open source platform, it gave users a way to process...

Announcing the Charmed Kafka beta

Charmed Kafka is a complete solution to manage the full lifecycle of Apache Kafka. The Canonical Data Fabric team is pleased to announce the first beta...

Why we built a Spark solution for Kubernetes

We’re super excited to announce that we have shipped the first release of our solution for big data – Charmed Spark. Charmed Spark packages a supported...

Write a Spark big data job with ChatGPT

I’ve read and watched more than a few articles about ChatGPT in the last couple of months. It seems the large language model AI hype machine just can’t stop. ...

Charmed Spark beta release is out – try it today

The Canonical Data Fabric team is pleased to announce the first beta release of Charmed Spark, our solution for Apache Spark. Apache Spark is a free, open...

Big data security foundations in five steps

We’ve all read the headlines about spectacular data breaches and other security incidents, and the impact that they have had on the victim organisations. And...

Apache Kafka service design for low latency and no data loss

Designing a production service environment around Apache Kafka that delivers low latency and zero-data loss at scale is non-trivial. Indeed, it’s the holy...

Kubernetes operators – the top 5 things to watch for

Software operators are steadily revolutionising how we deploy and run complex distributed systems. They offer the promise of low-intervention, self-driving...

Canonical Data Platform 2021 winter roundup

Canonical Data Platform: that was 2021 It’s that time of the year again: many folks are panic buying cans of windscreen de-icer spray and thermal underwear,...

  1. Previous page
  2. 1
  3. 2
  4. Next page