Container storage interface is an industry standard that enables storage vendors to develop a plugin once and have it work across a number of container orchestration systems.
SeaweedFS is a simple and highly scalable distributed file system, to store and serve billions of files fast!
- Already have a working Kubernetes cluster (includes
kubectl) - Already have a working SeaweedFS cluster
-
Git clone this repository and adjust your SeaweedFS Filer address via variable SEAWEEDFS_FILER in
deploy/kubernetes/seaweedfs-csi.yaml(2 places) -
Apply the container storage interface for SeaweedFS for your cluster. Use the '-pre-1.17' version for any cluster pre kubernetes version 1.17. To generate an up to date manifest from the helm chart, do:
$ helm template seaweedfs ./deploy/helm/seaweedfs-csi-driver > deploy/kubernetes/seaweedfs-csi.yaml
Then apply the manifest.
$ kubectl apply -f deploy/kubernetes/seaweedfs-csi.yaml
- Ensure all the containers are ready and running
$ kubectl get po -n kube-system
- Testing: Create a persistant volume claim for 5GiB with name
seaweedfs-csi-pvcwith storage classseaweedfs-storage. The value, 5Gib does not have any significance as for SeaweedFS the whole filesystem is mounted into the container.
$ kubectl apply -f deploy/kubernetes/sample-seaweedfs-pvc.yaml
- Verify if the persistant volume claim exists and wait until its the STATUS is
Bound
$ kubectl get pvc
- After its in
Boundstate, create a sample workload mounting that volume
$ kubectl apply -f deploy/kubernetes/sample-busybox-pod.yaml
- Verify the storage mount of the busybox pod
$ kubectl exec my-csi-app -- df -h
- Clean up
$ kubectl delete -f deploy/kubernetes/sample-busybox-pod.yaml
$ kubectl delete -f deploy/kubernetes/sample-seaweedfs-pvc.yaml
$ kubectl delete -f deploy/kubernetes/seaweedfs-csi.yaml
- Clone project
git clone https://github.com/seaweedfs/seaweedfs-csi-driver.git- Install
helm install --set seaweedfsFiler=<filerHost:port> seaweedfs-csi-driver ./seaweedfs-csi-driver/deploy/helm/seaweedfs-csi-driver- Clean up
helm uninstall seaweedfs-csi-driverWhen update DaemonSet ( DS ) break processes who implements fuse mount. And now new pod not remount net device
For better safe update use node.updateStrategy.type: OnDelete in this need manual update. Steps:
- delete DS pods on node where no exists seaweedfs PV
- cordon or taint node
- evict or delete pods with seaweedfs PV
- delete DS pod on node
- uncordon or remove taint on node
- repeat all steps on [all nodes
By default, driver will create separate folder (/buckets/<volume-id>) and will use separate collection (volume-id)
for each request. Sometimes we need to use exact collection name or change replication options.
It can be done via creating separate storage class with options:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: seaweedfs-special
provisioner: seaweedfs-csi-driver
parameters:
collection: mycollection
replication: "011"
diskType: "ssd"
There is another use case when we need to access one folder from different pods with ro/rw access. In this case we do not need additional StorageClass. We need to create PersistentVolume:
apiVersion: v1
kind: PersistentVolume
metadata:
name: seaweedfs-static
spec:
accessModes:
- ReadWriteMany
capacity:
storage: 1Gi
csi:
driver: seaweedfs-csi-driver
volumeHandle: dfs-test
volumeAttributes:
collection: default
replication: "011"
path: /path/to/files
diskType: "ssd"
readOnly: true
persistentVolumeReclaimPolicy: Retain
volumeMode: Filesystem
and bind PersistentVolumeClaim(s) to it:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: seaweedfs-static
spec:
storageClassName: ""
volumeName: seaweedfs-static
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
Participation in this project is governed by Kubernetes/CNCF code of conduct