OpenStack Manila using CephFS Storage Backend

1015767

Overview

In this article we will configure OpenStack Manila using CephFS as a storage backend. OpenStack Manila is an OpenStack project providing file services. Manila is storage backend agnostic and you can have many different kinds of storage backends, similar to Cinder. CephFS is a POSIX-Compliant file system that uses the Ceph storage cluster to store data. This guide should serve as a starting point for getting things going.

Prerequisites

The following are required to configure OpenStack Manila with CephFS:
  • Already configured Ceph cluster (Jewel or higher). See here to setup Ceph cluster.
  • Already configured OpenStack (Mitaka or higher). See here to setup OpenStack.

Configure CephFS

The following commands should be run on the Ceph ansible node.Add repository for ceph tools.
# subscription-manager repos --enable=rhel-7-server-rhceph-2-tools-rpms
Update Ansible inventory file and add metadata server.
#vi /etc/host/ansible
...
[mdss]
ceph2
...
Note: For non-lab environment you definitely wan’t to configure multiple MDS servers. One is active and additional servers are passive.Run Ansible.
# su - ansible
$ cd /usr/share/ceph-ansible
$ ansible-playbook site.yml -vvvv
PLAY RECAP ******************************************************************** 
ceph1 : ok=369 changed=1 unreachable=0 failed=0 
ceph2 : ok=364 changed=11 unreachable=0 failed=0 
ceph3 : ok=369 changed=1 unreachable=0 failed=0
Check CephFS metadata server.
# ceph mds stat
e24: 1/1/1 up {0=ceph2=up:active}
Create keyring and cephx authentication key for Manila service.
# read -d '' MON_CAPS << EOF
allow r,
allow command "auth del",
allow command "auth caps",
allow command "auth get",
allow command "auth get-or-create"
EOF

# ceph auth get-or-create client.manila -o manila.keyring \
mds 'allow *' \
osd 'allow rw' \
mon "$MON_CAPS"
Enable CephFS snapshots.
# ceph mds set allow_new_snaps true --yes-i-really-mean-it
Copy Ceph configuration and manila keyring to OpenStack controller running Manila share service.
# scp /etc/ceph/ceph.conf root@192.168.122.80:/etc/ceph
# scp manila.keyring root@192.168.122.80:/etc/ceph
The following commands should be run on OpenStack controller.Install Ceph tools and python-cephs
# subscription-manager repos --enable=rhel-7-server-rhceph-2-tools-rpms
# yum install -y ceph-common
# yum install python-cephfs
Change permissions on Ceph configuration and keyring for manila.
# chown manila /etc/ceph/manila.keyring
# chown manila /etc/ceph/ceph.conf
Update Ceph configuration.
# vi /etc/ceph/ceph.conf
...
[client.manila]
client mount uid = 0
client mount gid = 0
log file = /var/log/manila/ceph-client.manila.log
admin socket = /var/run/ceph/ceph-$name.$pid.asok
keyring = /etc/ceph/manila.keyring

...
Update Manila configuration.vi /etc/manila/manila.confenabled_share_protocols = NFS,CIFS,CEPHFS
# vi /etc/manila/manila.conf
...
enabled_share_protocols = NFS,CIFS,CEPHFS
enabled_share_backends = generic,cephfs

[cephfs]
driver_handles_share_servers = False
share_backend_name = cephfs
share_driver = manila.share.drivers.cephfs.cephfs_native.CephFSNativeDriver
cephfs_conf_path = /etc/ceph/ceph.conf
cephfs_auth_id = manila
cephfs_cluster_name = ceph
cephfs_enable_snapshots = True
...
Restart Manila services.
# systemctl restart openstack-manila-scheduler
# systemctl restart openstack-manila-api
# systemctl restart openstack-manila-share
Set share type for CephFS.
# manila type-create cephfstype false
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| required_extra_specs | driver_handles_share_servers : False |
| Name | cephfstype |
| Visibility | public |
| is_default | - |
| ID | ae7cc121-d8b6-47e5-86ba-36d607df19b0 |
| optional_extra_specs | snapshot_support : True |
+----------------------+--------------------------------------+
Create Manila share.
# manila create --share-type cephfstype --name cephshare1 cephfs 1
+-----------------------------+--------------------------------------+
| Property | Value |
+-----------------------------+--------------------------------------+
| status | creating |
| share_type_name | cephfstype |
| description | None |
| availability_zone | None |
| share_network_id | None |
| share_server_id | None |
| host | |
| access_rules_status | active |
| snapshot_id | None |
| is_public | False |
| task_state | None |
| snapshot_support | True |
| id | c72318fd-3cb2-4c0a-855e-b75c5bd43c6d |
| size | 1 |
| user_id | 9d592f8a49654e8592de4e69fd15e603 |
| name | cephshare1 |
| share_type | ae7cc121-d8b6-47e5-86ba-36d607df19b0 |
| has_replicas | False |
| replication_type | None |
| created_at | 2017-03-28T12:33:30.000000 |
| share_proto | CEPHFS |
| consistency_group_id | None |
| source_cgsnapshot_member_id | None |
| project_id | 29f6ba825bf5418395919c85874db4a5 |
| metadata | {} |
+-----------------------------+--------------------------------------+
Show Manila export location.
# manila share-export-location-list cephshare1
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------+-----------+
| ID | Path | Preferred |
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------+-----------+
| a85a1653-ce6e-4c01-ad05-17b9c41ad241 | 192.168.122.81:6789,192.168.122.82:6789,192.168.122.83:6789:/volumes/_nogroup/b1464433-f10e-458f-b120-b9b41d3f0083 | False |
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------+-----------+

Summary

In this article we configured OpenStack Manila to use the CephFS storage backend. Ceph is the perfect fit for OpenStack storage as it is a unified distributed software-defined storage system that scales with OpenStack. Ceph provides all storage access methods such as block (Cinder, Nova, Glance), file (Manila) and object (S3/Swift/Glance). As such Ceph can satisfy all OpenStack storage needs in a single unified, easy to manage system. Hopefully you found this article of use. Please let me know any and all feedback.Happy Manilaing!(c) 2017 Keith Tenzer                       
, , ,
One comment on “OpenStack Manila using CephFS Storage Backend

Leave a Reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: