Csi-driver-manila-operator#

Csi-driver-manila-operator – это оператор для развертывания csi-driver-manila в DropApp PLus.

Csi-driver-manila-operator управляет экземпляром объекта ClusterCSIDriver с именем manila.csi.openstack.org и параллельно запускает следующие контроллеры:

• manilaController. Обращается к OpenStack API и проверяет предоставлена ли Manila:

  • Если служба Manila найдена:

    • manilaControllerSet запускает csidriverset.Controller, который устанавливает csi-driver-manila.

    • nfsController запускает csidriverset.Controller, который устанавливает csi-driver-nfs.

    • Создается StorageClass для каждого типа общего ресурса, который указан Manila. По умолчанию раз в минуту осуществляется синхронизация. При добавлении нового типа общего ресурса ему будет создан StorageClass. Если для ранее созданного StorageClass был не найден соответствующий ему тип общего ресурса, то удаления StorageClass не происходит. Отсутствие типа общего ресурса может быть вызвано временной ошибкой при изменениях в настройке OpenStack или Manila.

  • Если служба Manila не найдена, то manilaController отмечает экземпляр объекта ClusterCSIDriver флагом ManilaControllerDisabled: True. Драйверы CSI в этом случае продолжат работу, но pod сможет размонтировать их тома CSI.

• secretSyncController. Синхронизирует секрет, который предоставляет cloud-credentials-operator, с секретом, который используется драйверами CSI.

Последовательность выполнения#

Развертывание и управление csi-driver-manila-operator осуществляется с помощью оператора cluster-storage-operator (оператор кластерного хранилища – подробнее в разделе «Cluster-storage-operator»).

Настройка осуществляется с использованием YAML-манифестов:

1. Создайте пространство имен для csi-driver-manila в cluster-storage-operator:

   apiVersion: v1
   kind: Namespace
   metadata:
     name: openshift-manila-csi-driver
     annotations:
       include.release.openshift.io/self-managed-high-availability: "true"
       openshift.io/node-selector: ""
       workload.openshift.io/allowed: "management"
     labels:
       pod-security.kubernetes.io/enforce: privileged
       pod-security.kubernetes.io/audit: privileged
       pod-security.kubernetes.io/warn: privileged
   

2. Создайте ServiceAccount для csi-driver-manila-operator:

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: manila-csi-driver-operator
     namespace: openshift-cluster-csi-drivers
   

3. Создайте роль manila-csi-driver-operator-role:

   manila_role.yaml

   apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: manila-csi-driver-operator-role
        namespace: openshift-cluster-csi-drivers
      rules:
      - apiGroups:
        - ''
        resources:
        - pods
        - services
        - endpoints
        - persistentvolumeclaims
        - events
        - configmaps
        - secrets
        verbs:
        - get
        - list
        - watch
        - create
        - update
        - patch
        - delete
      - apiGroups:
        - ''
        resources:
        - namespaces
        verbs:
        - get
      - apiGroups:
        - apps
        resources:
        - deployments
        - daemonsets
        - replicasets
        - statefulsets
        verbs:
        - get
        - list
        - watch
        - create
        - update
        - patch
        - delete
   

4. Создайте RoleBinding для связи роли:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: manila-csi-driver-operator-rolebinding
     namespace: openshift-cluster-csi-drivers
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: manila-csi-driver-operator-role
   subjects:
   - kind: ServiceAccount
     name: manila-csi-driver-operator
     namespace: openshift-cluster-csi-drivers
   

5. Создайте ClusterRole:

   manila_clusterrole.yaml

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: manila-csi-driver-operator-clusterrole
   rules:
   - apiGroups:
     - security.openshift.io
     resourceNames:
     - privileged
     resources:
     - securitycontextconstraints
     verbs:
     - use
   
   # The operator needs these config maps:
   # - read/write openshift-manila-csi-driver/cloud-provider-config
   # - read-only kube-system/extension-apiserver-authentication
   # - read/write manila-csi-driver-operator-lock
   - apiGroups:
     - ''
     resources:
     - configmaps
     verbs:
     - watch
     - list
     - get
     - create
     - delete
     - patch
     - update
   - apiGroups:
     - rbac.authorization.k8s.io
     resources:
     - clusterroles
     - clusterrolebindings
     - roles
     - rolebindings
     verbs:
     - watch
     - list
     - get
     - create
     - delete
     - patch
     - update
   - apiGroups:
     - ''
     resources:
     - serviceaccounts
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   - apiGroups:
     - apiextensions.k8s.io
     resources:
     - customresourcedefinitions
     verbs:
     - get
     - list
     - create
     - watch
     - delete
   - apiGroups:
     - coordination.k8s.io
     resources:
     - leases
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   - apiGroups:
     - ''
     resources:
     - nodes
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   - apiGroups:
     - ''
     resources:
     - secrets
     verbs:
     - get
     - list
     - watch
     # For CA certificate sync
     - create
     - patch
     - update
   - apiGroups:
     - ''
     resources:
     - namespaces
     verbs:
     - get
     - list
     - watch
     - create
     - patch
     - delete
     - update
   - apiGroups:
     - ''
     resources:
     - persistentvolumes
     verbs:
     - create
     - delete
     - list
     - get
     - watch
     - update
     - patch
   - apiGroups:
     - ''
     resources:
     - persistentvolumeclaims
     verbs:
     - get
     - list
     - watch
     - update
   - apiGroups:
     - ''
     resources:
     - persistentvolumeclaims/status
     verbs:
     - patch
     - update
   - apiGroups:
     - apps
     resources:
     - deployments
     - daemonsets
     - replicasets
     - statefulsets
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   - apiGroups:
     - storage.k8s.io
     resources:
     - volumeattachments
     verbs:
     - get
     - list
     - watch
     - update
     - delete
     - create
     - patch
   - apiGroups:
     - snapshot.storage.k8s.io
     resources:
     - volumesnapshotcontents/status
     verbs:
     - update
     - patch
   - apiGroups:
     - storage.k8s.io
     resources:
     - storageclasses
     - csinodes
     verbs:
     - create
     - get
     - list
     - watch
     - update
     - delete
   - apiGroups:
     - '*'
     resources:
     - events
     verbs:
     - get
     - patch
     - create
     - list
     - watch
     - update
     - delete
   - apiGroups:
     - snapshot.storage.k8s.io
     resources:
     - volumesnapshotclasses
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - delete
   - apiGroups:
     - snapshot.storage.k8s.io
     resources:
     - volumesnapshotcontents
     verbs:
     - create
     - get
     - list
     - watch
     - update
     - delete
     - patch
   - apiGroups:
     - snapshot.storage.k8s.io
     resources:
     - volumesnapshots
     verbs:
     - get
     - list
     - watch
     - update
     - patch
   - apiGroups:
     - snapshot.storage.k8s.io
     resources:
     - volumesnapshots/status
     verbs:
     - patch
   - apiGroups:
     - storage.k8s.io
     resources:
     - csidrivers
     verbs:
     - create
     - get
     - list
     - watch
     - update
     - delete
   - apiGroups:
     - csi.openshift.io
     resources:
     - '*'
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   - apiGroups:
     - config.openshift.io
     resources:
     - infrastructures
     - proxies
     - apiservers
     verbs:
     - get
     - list
     - watch
   - apiGroups:
     - operator.openshift.io
     resources:
     - 'clustercsidrivers'
     - 'clustercsidrivers/status'
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   # Allow kube-rbac-proxy to create TokenReview to be able to authenticate Prometheus when collecting metrics
   - apiGroups:
     - "authentication.k8s.io"
     resources:
     - "tokenreviews"
     verbs:
     - "create"
   # Allow the operator to create ServiceMonitor in the driver namespace
   - apiGroups:
     - monitoring.coreos.com
     resources:
     - servicemonitors
     verbs:
     - get
     - create
     - update
     - patch
     - delete
   # Allow the operator to create Service in the driver namespace
   - apiGroups:
     - ''
     resources:
     - services
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   # Grant these permissions in the driver namespace to Prometheus
   - apiGroups:
     - ''
     resources:
     - pods
     - endpoints
     verbs:
     - 'get'
     - 'list'
     - 'watch'
   - apiGroups:
     - policy
     resources:
     - poddisruptionbudgets
     verbs:
     - get
     - list
     - watch
     - create
     - update
     - patch
     - delete
   

6. Создайте ClusterRoleBinding для связи ClusterRole:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: manila-csi-driver-operator-clusterrolebinding
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: manila-csi-driver-operator-clusterrole
   subjects:
   - kind: ServiceAccount
     name: manila-csi-driver-operator
     namespace: openshift-cluster-csi-drivers
   

7. Настройте Deployment:

   manila_deployment.yaml

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: manila-csi-driver-operator
     namespace: openshift-cluster-csi-drivers
     annotations:
       config.openshift.io/inject-proxy: manila-csi-driver-operator
   spec:
     replicas: 1
     selector:
       matchLabels:
         name: manila-csi-driver-operator
     strategy: {}
     template:
       metadata:
         annotations:
           target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
         labels:
           name: manila-csi-driver-operator
       spec:
         containers:
         - args:
           - start
           - -v=${LOG_LEVEL}
           env:
           - name: DRIVER_IMAGE
             value: ${DRIVER_IMAGE}
           - name: NFS_DRIVER_IMAGE
             value: ${NFS_DRIVER_IMAGE}
           - name: PROVISIONER_IMAGE
             value: ${PROVISIONER_IMAGE}
           - name: ATTACHER_IMAGE
             value: ${ATTACHER_IMAGE}
           - name: RESIZER_IMAGE
             value: ${RESIZER_IMAGE}
           - name: SNAPSHOTTER_IMAGE
             value: ${SNAPSHOTTER_IMAGE}
           - name: NODE_DRIVER_REGISTRAR_IMAGE
             value: ${NODE_DRIVER_REGISTRAR_IMAGE}
           - name: LIVENESS_PROBE_IMAGE
             value: ${LIVENESS_PROBE_IMAGE}
           - name: KUBE_RBAC_PROXY_IMAGE
             value: ${KUBE_RBAC_PROXY_IMAGE}
           - name: POD_NAME
             valueFrom:
               fieldRef:
                 fieldPath: metadata.name
           image: ${OPERATOR_IMAGE}
           imagePullPolicy: IfNotPresent
           name: manila-csi-driver-operator
           volumeMounts:
           - name: cacert
             mountPath: /etc/openstack-ca/
           - name: cloud-credentials
             # Create /etc/openstack/clouds.yaml
             mountPath: /etc/openstack/
           resources:
             requests:
               memory: 50Mi
               cpu: 10m
         priorityClassName: system-cluster-critical
         serviceAccountName: manila-csi-driver-operator
         nodeSelector:
           node-role.kubernetes.io/master: ""
         tolerations:
         - key: CriticalAddonsOnly
           operator: Exists
         - key: node-role.kubernetes.io/master
           operator: Exists
           effect: "NoSchedule"
         volumes:
         - name: cacert
           # Extract ca-bundle.pem to /usr/share/pki/ca-trust-source if present.
           # Let the pod start when the ConfigMap does not exist or the certificate
           # is not preset there. The certificate file will be created once the
           # ConfigMap is created / the cerificate is added to it.
           configMap:
             name: cloud-provider-config
             items:
               - key: ca-bundle.pem
                 path: ca-bundle.pem
             optional: true
         - name: cloud-credentials
           secret:
             secretName: manila-cloud-credentials
             optional: false
     securityContext:
       allowPrivilegeEscalation: false
     capabilities.drop: ALL
       privileged: false
   

8. Создайте ClusterCSIDriver:

   apiVersion: operator.openshift.io/v1
   kind: ClusterCSIDriver
   metadata:
     name: manila.csi.openstack.org
   spec:
     managementState: Managed
     logLevel: Normal
     operatorLogLevel: Normal
   

Результат#

Csi-driver-manila-operator настроен и готов к использованию.