Your submission was sent successfully! Close

You have successfully unsubscribed! Close

Thank you for signing up for our newsletter!
In these regular emails you will find the latest updates about Ubuntu and upcoming events where you can meet our team.Close

Ceph storage on Ubuntu: An overview

Alex Chalkias

on 18 February 2020

This article was last updated 1 year ago.

Ceph is a compelling open-source alternative to proprietary software-defined storage solutions from traditional vendors, with a vibrant community collaborating on the technology. Ubuntu was an early supporter of Ceph and its community. That support continues today as Canonical maintains premier member status and serves on the governing board of the Ceph Foundation.

With many global enterprises and telco operators running Ceph on Ubuntu, organisations are able to combine block and object storage at scale while tapping into the economic and upstream benefits of open source.

Why use Ceph?

Ceph is unique because it makes data available in multiple ways: as a POSIX compliant filesystem through CephFS, as block storage volumes via the RBD driver and for object store, compatible with both S3 and Swift protocols, using the RADOS gateway.

A common use case for Ceph is to provide block and object store to OpenStack clouds via Cinder and as a Swift replacement. Kubernetes has similarly adopted Ceph as a popular way for physical volumes (PV) as a Container Storage Interface (CSI) plugin.

Even as a stand-alone, Ceph is a compelling open-source storage alternative to closed-source, proprietary solutions as it reduces OpEx costs organisations commonly accrue with storage from licensing, upgrades and potential vendor lock-in fees.

How Ceph works

Ceph stores data in pools, which are accessible from users or other services to provide block, file or object storage. A Ceph pool backs each of these mechanisms. Replication, access rights and other internal characteristics (such as data placement, ownership, access, etc.) are expressed on a per pool basis.

The Ceph Monitors (MONs) are responsible to maintain the Cluster state. They manage the location of data using the CRUSH map. These operate in a cluster with quorum-based HA, with data stored and retrieved via Object Storage Devices (OSDs).

There is a 1:1 mapping of a storage device and a running OSD daemon process. OSDs utilise the CPU and RAM of the participating cluster member host aggressively. This is why it is important to carefully balance the number of OSDs with the number of CPU cores and memory when architecting a Ceph cluster. This is especially true when aiming for a hyper-converged architecture (such as for OpenStack or Kubernetes for example).

Using LXD as a container hypervisor helps to properly enforce resource limitations on most running processes on a given node. LXD is used extensively to provide the best economics in Canonical’s Charmed OpenStack distribution by isolating the Ceph MONs. Containerising the Ceph OSDs is currently not recommended.

Ceph storage mechanisms

Accessing each data pool equates choosing the mechanism to do so. For example, one pool may be used to store block volumes, while another provides the storage backend for object store or filesystems. In the case of volumes, the host that is seeking to mount the volume needs to load the RBD kernel module after which Ceph volumes can be mounted, just as local volumes would be.

Object buckets are not mounted generally – client-side applications can use overlay filesystems to simulate a ‘drive’, but it is not an actual volume being mounted. Instead, the RADOS gateway enables access to Object buckets. RADOSGW provides a REST API to access objects using the S3 or Swift protocols. Filesystems are created and formatted using CephFS after which they are exported in a similar fashion as NFS mounts, and available to local networks.

Volume and object store use cases have been in production at scale for quite some time. Using Ceph to combine volume and object store provides many benefits to operators. Aside from the obvious support for multiple storage use cases, it also allows for the best density when properly architected and scaled.

Ceph storage support with Canonical

Canonical provides Ceph support as part of Ubuntu Advantage for Infrastructure with Standard and Advanced SLAs, corresponding to business hours and 24×7 support, respectively. Each covered node includes support for up to 48TB of raw storage of a Ceph cluster.

This coverage derives from our reference hardware recommendation for OpenStack and Kubernetes in a hyper-converged architecture at an optimal price per TB range while preserving best performance across compute and network in an on-premise cloud. In case the number of nodes to TB ratio does not coincide with our recommendation and exceeding this limit, Canonical offers per TB pricing beyond the included accommodating our scale-out storage customers.

Ceph is available on Ubuntu in the main repository, and as such, users receive free security updates for up to five years on an LTS version. An additional five years of paid, commercial support beyond the standard support cycle is available through UA Infrastructure.

Discover how organisations benefit from Canonical’s Ceph support with these case studies:

Learn more about Canonical’s Ceph storage solution and get more details on our Ceph support offering.

ceph logo

What is Ceph?

Ceph is a software-defined storage (SDS) solution designed to address the object, block, and file storage needs of both small and large data centres.

It's an optimised and easy-to-integrate solution for companies adopting open source as the new norm for high-growth block storage, object stores and data lakes.

Learn more about Ceph ›

ceph logo

How to optimise your cloud storage costs

Cloud storage is amazing, it's on demand, click click ready to go, but is it the most cost effective approach for large, predictable data sets?

In our white paper learn how to understand the true costs of storing data in a public cloud, and how open source Ceph can provide a cost effective alternative!

Access the whitepaper ›

Interested in running Ubuntu in your organisation? Talk to us today

ceph logo

A guide to software-defined storage for enterprises

Ceph is a software-defined storage (SDS) solution designed to address the object, block, and file storage needs of both small and large data centres.

In our whitepaper explore how Ceph can replace proprietary storage systems in the enterprise.

Access the whitepaper ›

Interested in running Ubuntu in your organisation? Talk to us today

Newsletter signup

Get the latest Ubuntu news and updates in your inbox.

By submitting this form, I confirm that I have read and agree to Canonical's Privacy Policy.

Related posts

The role of secure data storage in fueling AI innovation

There is no AI without data Artificial intelligence is the most exciting technology revolution of recent years. Nvidia, Intel, AMD and others continue to...

CentOS EOL – What does it mean for Ceph storage?

Out of the darkness and into the light, a new path forward Back in 2020, the CentOS Project announced that they would focus only on CentOS Stream, meaning...

Ceph Storage for AI

Use open source Ceph storage to fuel your AI vision The use of AI is a hot topic for any organisation right now. The allure of operational insights, profit,...