The Ceph File System, or CephFS, is a POSIX-compliant file system built on top of Ceph’s distributed object store, RADOS.
The ceph-fs charm deploys the metadata server daemon (MDS), which is the underlying management software for CephFS. The charm is deployed within the context of an existing Charmed Ceph cluster.
Highly available CephFS is achieved by deploying multiple MDS servers (i.e. multiple ceph-fs application units).
To deploy a three-node MDS cluster in a pre-existing Ceph cluster:
juju deploy -n 3 --to lxd:0,lxd:1,lxd:2 ceph-fs juju add-relation ceph-fs:ceph-mds ceph-mon:mds
Here the three ceph-fs units are containerised, where new containers are placed on existing machines 0, 1, and 2.
CephFS is now fully set up.
CephFS client usage
This section will provide optional instructions for verifying the CephFS service by setting up a simple client environment. Deploy the client using the steps provided in the Client setup appendix.
These instructions are based upon the native capabilities of the Linux kernel (v.4.x).
A kernel driver will allow the client to mount CephFS as a regular file system.
An example deployment will have a
juju status output similar to the following:
Model Controller Cloud/Region Version SLA Timestamp ceph my-controller my-maas/default 2.8.1 unsupported 19:34:16Z App Version Status Scale Charm Store Rev OS Notes ceph-fs 15.2.3 active 3 ceph-fs jujucharms 24 ubuntu ceph-mon 15.2.3 active 3 ceph-mon jujucharms 49 ubuntu ceph-osd 15.2.3 active 3 ceph-osd jujucharms 304 ubuntu ceph-client 20.04 active 1 ubuntu jujucharms 15 ubuntu Unit Workload Agent Machine Public address Ports Message ceph-client/0* active idle 3 10.0.0.240 ready ceph-fs/0 active idle 0/lxd/0 10.0.0.245 Unit is ready ceph-fs/1 active idle 1/lxd/0 10.0.0.246 Unit is ready ceph-fs/2* active idle 2/lxd/0 10.0.0.241 Unit is ready ceph-mon/0 active idle 0/lxd/1 10.0.0.247 Unit is ready and clustered ceph-mon/1 active idle 1/lxd/1 10.0.0.242 Unit is ready and clustered ceph-mon/2* active idle 2/lxd/1 10.0.0.249 Unit is ready and clustered ceph-osd/0 active idle 0 10.0.0.229 Unit is ready (2 OSD) ceph-osd/1* active idle 1 10.0.0.230 Unit is ready (2 OSD) ceph-osd/2 active idle 2 10.0.0.252 Unit is ready (2 OSD)
The client host is represented by the
Verify that the filesystem name set up by the ceph-fs charm is ‘ceph-fs’:
juju ssh ceph-mon/0 "sudo ceph fs ls"
name: ceph-fs, metadata pool: ceph-fs_metadata, data pools: [ceph-fs_data ]
Create a CephFS user (‘test’) with read/write permissions at the root of the ‘ceph-fs’ filesystem, collect the user’s keyring file, and transfer it to the client:
juju ssh ceph-mon/0 "sudo ceph fs authorize ceph-fs client.test / rw" \ | tee ceph.client.test.keyring juju scp ceph.client.test.keyring ceph-client/0:
Connect to the client:
juju ssh ceph-client/0
From the CephFS client,
Configure the client using the keyring file and set up the correct permissions:
sudo mv ~ubuntu/ceph.client.test.keyring /etc/ceph sudo chmod 600 /etc/ceph/ceph.client.test.keyring sudo chown root: /etc/ceph/ceph.client.test.keyring
Note: The key installed on a client host authorises access to the CephFS filesystem for the host itself, and not to a particular user.
Mount the CephFS filesystem and create a test file:
sudo mkdir /mnt/cephfs sudo mount -t ceph :/ /mnt/cephfs -o name=test sudo mkdir /mnt/cephfs/work sudo chown ubuntu: /mnt/cephfs/work touch /mnt/cephfs/work/test