CephFS

User Guide to Provisioning and Deploying CephFS

Introduction

CephFS provides a common filesystem that can be shared amongst multiple openstack vm hosts. Access to the service is granted via requests to cloud@computecanada.ca.

This is a fairly technical procedure that assumes basic linux skills for creating/editing files, setting permissions and creating mount points. Contact your technical resource for your project for assistance in setting up this service.

Technical Procedure

Request Access to Shares

• If you do not already have a quota for the service you will need to request this through cloud@computecanada.ca.
  • In your request provide the following:
    • Project name
    • Expected file system share name.
      • This should be the name of the project followed by a unique name for the share (e.g. def-smith-WebServerShare).
    • Approximate size of share in GB.

Create Share

1. Create a share in "Shares" under the "Share" menu:
   • Give this a name that identifies your project: project-name-shareName
     • e.g. def-smith-WebServerShare
   • Share Protocol = cephfs
   • Size = size you need for this share
   • Share Type = cephfs
2. Provide the name of your share to the cloud team so they can generate a key for you. Do so by replying to the initial request ticket you opened in the first steps.
   • You will need to provide the following in your request:
     • Share name (e.g. def-smith-WebServerShare).

Configure Host

1. Retrieve Key:

   • View the "Share Details" by clicking on the share name in the "Shares" pane.

     • Key is located under the "Access Rules" section labeled "Access Key".

   • Create two files each containing the key provided.

   • File 1: /etc/ceph/client.fullkey.shareName (e.g. client.fullkey.WebServerShare)

     • contents:

       [client.shareName]
           key = KeyProvidedByAdmin
       

   • File 2: /etc/ceph/client.keyonly.shareName (e.g client.keyonly.WebServerShare)

     • contents:

       KeyProvidedByAdmin
       

     • This file only contains the key provided by the admin

   • Own these files correctly to protect the key information:

     • Each file should be own to root (sudo chown root.root filename).
     • Each file should be only readable by root (sudo chmod 600 filename).

2. Create /etc/ceph/ceph.conf with contents:

   [client]
       client quota = true
       mon host = 10.30.201.3:6789,10.30.202.3:6789,10.30.203.3:6789
   

   • Note: these are the monitors for the Arbutus cluster - if connecting to a different cluster you will need the monitor information specific to that cluster.
     • You can find the monitor information in the Share Details for your share in the "Path" field.

3. Install required packages

• Red Hat Family (RHEL, CentOS, Fedora, Scientific Linux, SUSE, etc.): 1. Install relevant repos for access to ceph client packages: ceph-stable (nautilus is current as of this writting) epel 1. Install packages to enable the ceph client on all the VMs you plan on mounting the share: libcephfs2 python-cephfs ceph-common python-ceph-argparse ceph-fuse (only if you intend a fuse mount)

• Debian Family (Debian, Ubuntu, Mint, etc.):

  sudo apt install ceph-common
  

1. Retrieve the connection information from the share page for your connection:
   • Open up the share details by clicking the name of the share in the Shares page.
   • Copy the entire path of the share for mounting the filesystem.
2. Mount the filesystem
   • Create mount point directory somewhere in your host (likely under /mnt/ - e.g. /mnt/ShareName)
   • Via kernel mount using the ceph driver:
     • Syntax: sudo mount -t ceph < path information> < mountPoint > -o name=< shareKeyName >, secretfile=< /path/to/keyringfileOnlyFile >
     • sudo mount -t ceph mon1:6789,mon2:6789,mon3:6789:/volumes/_nogroup/share_instance_id
       • e.g sudo mount -t ceph 192.168.17.13:6789,192.168.17.14:6789,192.168.17.15:6789:/volumes/_nogroup/a87b5ef3-b266-4664-a5ed-026cddfdcdb7 /mnt/WebServerShare -o name=def-smith,secretfile=/etc/ceph/client.keyonly.WebServerShare
   • Via ceph-fuse
     • Need to install ceph-fuse
     • Syntax: sudo ceph-fuse < mountPoint > --id=< projectName > --conf=< pathtoCeph.conf > --keyring=< fullKeyringLocation > --client-mountpoint=pathFromShareDetails
       • e.g. sudo ceph-fuse /mnt/WebServerShare --id=def-smith --conf=/etc/ceph/ceph.conf --keyring=/etc/ceph/client.fullkey.WebServerShare --client-mountpoint=/volumes/_nogroup/a87b5ef3-b266-4664-a5ed-026cddfdcdb7

Notes

• A particular share can have more than one user key provisioned for it.
  • This allows a more granular access to the filesystem.
  • For example, if you needed some hosts to only access the filesystem in a read only capacity.
  • If you have multiple keys for a share you can add the extra keys to your host and modify the above mounting procedure.
• This service is not available to hosts outside of the Openstack cluster.