正文
- 存储类的好处之一便是支持PV的动态供给,它甚至可以直接被视作为PV的创建模版,用户用到持久性存储时,需要通过创建PVC来绑定匹配的PV,此类操作需求较大,或者当管理员手动创建的PV无法满足PVC的所有需求时,系统按PVC的需求标准动态创建适配的PV会为存储管理带来极大的灵活性,不过仅那些属于StorageClass的PVC和PV才能产生绑定关系,即没有指定StorageClass的PVC只能绑定同类的PV。
- 存储类对象的名称至关重要,它是用户调用的标识,创建存储类对象时,除了名称之外,还需要为其定义三个关键字段。provisioner、parameter和reclaimPolicy。
- 所以kubernetes提供了一种可以动态分配的工作机制,可用自动创建PV,该机制依赖于StorageClass的API,将某个存储节点划分1T给kubernetes使用,当用户申请5Gi的PVC时,会自动从这1T的存储空间去创建一个5Gi的PV,而后自动与之进行关联绑定。
- 动态PV供给的启用需要事先创建一个存储类,不同的Provisoner的创建方法各有不同,并非所有的存储卷插件都由Kubernetes内建支持PV动态供给。
基于NFS实现动态供应
由于kubernetes内部不包含NFS驱动,所以需要使用外部驱动nfs-subdir-external-provisioner是一个自动供应器,它使用NFS服务端来支持动态供应。
NFS-subdir-external- provisioner实例负责监视PersistentVolumeClaims请求StorageClass,并自动为它们创建NFS所支持的PresistentVolumes。
GitHub地址: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner
准备NFS服务端的共享目录
这里的意思是要把哪个目录给kubernetes来使用。把目录共享出来。
1
2
3
4
5
|
[root@kn-server-node02-15 ~] # ll /data/ 总用量 0 [root@kn-server-node02-15 ~] # showmount -e 10.0.0.15 Export list for 10.0.0.15: /data 10.0.0.0 /24 |
安装NFS-Server驱动。
首先创建RBAC权限。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
|
[root@kn-server-master01-13 nfs-provisioner] # cat nfs-rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io /v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups: [ "" ] resources: [ "nodes" ] verbs: [ "get" , "list" , "watch" ] - apiGroups: [ "" ] resources: [ "persistentvolumes" ] verbs: [ "get" , "list" , "watch" , "create" , "delete" ] - apiGroups: [ "" ] resources: [ "persistentvolumeclaims" ] verbs: [ "get" , "list" , "watch" , "update" ] - apiGroups: [ "storage.k8s.io" ] resources: [ "storageclasses" ] verbs: [ "get" , "list" , "watch" ] - apiGroups: [ "" ] resources: [ "events" ] verbs: [ "create" , "update" , "patch" ] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io /v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io --- kind: Role apiVersion: rbac.authorization.k8s.io /v1 metadata: name: leader-locking-nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default rules: - apiGroups: [ "" ] resources: [ "endpoints" ] verbs: [ "get" , "list" , "watch" , "create" , "update" , "patch" ] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io /v1 metadata: name: leader-locking-nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default roleRef: kind: Role name: leader-locking-nfs-client-provisioner apiGroup: rbac.authorization.k8s.io [root@kn-server-master01-13 nfs-provisioner] # kubectl apply -f nfs-rbac.yaml serviceaccount /nfs-client-provisioner created clusterrole.rbac.authorization.k8s.io /nfs-client-provisioner-runner created clusterrolebinding.rbac.authorization.k8s.io /run-nfs-client-provisioner created role.rbac.authorization.k8s.io /leader-locking-nfs-client-provisioner created rolebinding.rbac.authorization.k8s.io /leader-locking-nfs-client-provisioner created |
部署NFS-Provisioner
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
|
[root@kn-server-master01-13 nfs-provisioner] # cat nfs-provisioner-deploy.yaml apiVersion: apps /v1 kind: Deployment metadata: name: nfs-client-provisioner labels: app: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default spec: replicas: 1 strategy: type : Recreate selector: matchLabels: app: nfs-client-provisioner template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: k8s.gcr.io /sig-storage/nfs-subdir-external-provisioner :v4.0.2 镜像在国内是拉取不到的,因此为下载下来了放在我的docker hub。 替换为lihuahaitang /nfs-subdir-external-provisioner :v4.0.2 volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env : - name: PROVISIONER_NAME value: k8s-sigs.io /nfs-subdir-external-provisioner NFS-Provisioner的名称,后续StorageClassName要与该名称保持一致 - name: NFS_SERVER NFS服务器的地址 value: 10.0.0.15 - name: NFS_PATH value: /data volumes: - name: nfs-client-root nfs: server: 10.0.0.15 path: /data [root@kn-server-master01-13 nfs-provisioner] # kubectl apply -f nfs-provisioner-deploy.yaml deployment.apps /nfs-client-provisioner created Pod正常运行。 [root@kn-server-master01-13 nfs-provisioner] # kubectl get pods NAME READY STATUS RESTARTS AGE nfs-client-provisioner-57d6d9d5f6-dcxgq 1 /1 Running 0 2m25s describe查看Pod详细信息; [root@kn-server-master01-13 nfs-provisioner] # kubectl describe pods nfs-client-provisioner-57d6d9d5f6-dcxgq Name: nfs-client-provisioner-57d6d9d5f6-dcxgq Namespace: default Priority: 0 Node: kn-server-node02-15 /10 .0.0.15 Start Time: Mon, 28 Nov 2022 11:19:33 +0800 Labels: app=nfs-client-provisioner pod-template- hash =57d6d9d5f6 Annotations: <none> Status: Running IP: 192.168.2.82 IPs: IP: 192.168.2.82 Controlled By: ReplicaSet /nfs-client-provisioner-57d6d9d5f6 Containers: nfs-client-provisioner: Container ID: docker: //b5ea240a8693185be681714747f8e0a9f347492a24920dd68e629effb3a7400f Image: k8s.gcr.io /sig-storage/nfs-subdir-external-provisioner :v4.0.2 镜像来自k8s.gcr.io Image ID: docker-pullable: //k8s .gcr.io /sig-storage/nfs-subdir-external-provisioner @sha256:63d5e04551ec8b5aae83b6f35938ca5ddc50a88d85492d9731810c31591fa4c9 Port: <none> Host Port: <none> State: Running Started: Mon, 28 Nov 2022 11:20:12 +0800 Ready: True Restart Count: 0 Environment: PROVISIONER_NAME: k8s-sigs.io /nfs-subdir-external-provisioner NFS_SERVER: 10.0.0.15 NFS_PATH: /data Mounts: /persistentvolumes from nfs-client-root (rw) /var/run/secrets/kubernetes .io /serviceaccount from kube-api-access-q2z8w (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: nfs-client-root: Type: NFS (an NFS mount that lasts the lifetime of a pod) Server: 10.0.0.15 Path: /data ReadOnly: false kube-api-access-q2z8w: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io /not-ready :NoExecute op =Exists for 300s node.kubernetes.io /unreachable :NoExecute op =Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 3m11s default-scheduler Successfully assigned default /nfs-client-provisioner-57d6d9d5f6-dcxgq to kn-server-node02-15 Normal Pulling 3m11s kubelet Pulling image "k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2" Normal Pulled 2m32s kubelet Successfully pulled image "k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2" in 38.965869132s Normal Created 2m32s kubelet Created container nfs-client-provisioner Normal Started 2m32s kubelet Started container nfs-client-provisioner |
创建StorageClass
创建NFS StorageClass动态供应商。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
[root@kn-server-master01-13 nfs-provisioner] # cat storageclass.yaml apiVersion: storage.k8s.io /v1 kind: StorageClass 类型为storageclass metadata: name: nfs-provisioner-storage PVC申请时需明确指定的storageclass名称 annotations: storageclass.kubernetes.io /is-default-class : "true" provisioner: k8s-sigs.io /nfs-subdir-external-provisioner 供应商名称,必须和上面创建的 "PROVISIONER_NAME" 保持一致 parameters: archiveOnDelete: "false" 如果值为 false ,删除pvc后也会删除目录内容, "true" 则会对数据进行保留 pathPattern: "${.PVC.namespace}/${.PVC.name}" 创建目录路径的模板,默认为随机命名。 [root@kn-server-master01-13 nfs-provisioner] # kubectl apply -f storageclass.yaml storageclass.storage.k8s.io /nfs-provisioner-storage created storage简写sc [root@kn-server-master01-13 nfs-provisioner] # kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE nfs-provisioner-storage k8s-sigs.io /nfs-subdir-external-provisioner Delete Immediate false 3s describe查看配详细信息。 [root@kn-server-master01-13 nfs-provisioner] # kubectl describe sc Name: nfs-provisioner-storage IsDefaultClass: Yes Annotations: kubectl.kubernetes.io /last-applied-configuration ={ "apiVersion" : "storage.k8s.io/v1" , "kind" : "StorageClass" , "metadata" :{ "annotations" :{ "storageclass.kubernetes.io/is-default-class" : "true" }, "name" : "nfs-provisioner-storage" }, "parameters" :{ "archiveOnDelete" : "false" , "pathPattern" : "${.PVC.namespace}/${.PVC.name}" }, "provisioner" : "k8s-sigs.io/nfs-subdir-external-provisioner" } ,storageclass.kubernetes.io /is-default-class = true Provisioner: k8s-sigs.io /nfs-subdir-external-provisioner Parameters: archiveOnDelete= false ,pathPattern=${.PVC.namespace}/${.PVC.name} AllowVolumeExpansion: < unset > MountOptions: <none> ReclaimPolicy: Delete VolumeBindingMode: Immediate Events: <none> |
创建PVC,自动关联PV
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
|
[root@kn-server-master01-13 nfs-provisioner] # cat nfs-pvc-test.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: nfs-pvc- test spec: storageClassName: "nfs-provisioner-storage" accessModes: - ReadWriteMany resources: requests: storage: 0.5Gi 这里的PV的名字是随机的,数据的存储路径是根据pathPattern来定义的。 [root@kn-server-node02-15 data] # ls default [root@kn-server-node02-15 data] # ll default/ 总用量 0 drwxrwxrwx 2 root root 6 11月 28 13:56 nfs-pvc- test [root@kn-server-master01-13 pv] # kubectl get pv pvc-8ed67f7d-d829-4d87-8c66-d8a85f50772f 512Mi RWX Delete Bound default /nfs-pvc-test nfs-provisioner-storage 5m19s [root@kn-server-master01-13 nfs-provisioner] # kubectl describe pv pvc-8ed67f7d-d829-4d87-8c66-d8a85f50772f Name: pvc-8ed67f7d-d829-4d87-8c66-d8a85f50772f Labels: <none> Annotations: pv.kubernetes.io /provisioned-by : k8s-sigs.io /nfs-subdir-external-provisioner Finalizers: [kubernetes.io /pv-protection ] StorageClass: nfs-provisioner-storage Status: Bound Claim: default /nfs-pvc-test Reclaim Policy: Delete Access Modes: RWX VolumeMode: Filesystem Capacity: 512Mi Node Affinity: <none> Message: Source: Type: NFS (an NFS mount that lasts the lifetime of a pod) Server: 10.0.0.15 Path: /data/default/nfs-pvc-test ReadOnly: false Events: <none> describe可用看到更详细的信息 root@kn-server-master01-13 nfs-provisioner] # kubectl describe pvc Name: nfs-pvc- test Namespace: default StorageClass: nfs-provisioner-storage Status: Bound Volume: pvc-8ed67f7d-d829-4d87-8c66-d8a85f50772f Labels: <none> Annotations: pv.kubernetes.io /bind-completed : yes pv.kubernetes.io /bound-by-controller : yes volume.beta.kubernetes.io /storage-provisioner : k8s-sigs.io /nfs-subdir-external-provisioner Finalizers: [kubernetes.io /pvc-protection ] Capacity: 512Mi 定义的存储大小 Access Modes: RWX 卷的读写 VolumeMode: Filesystem Used By: <none> Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ExternalProvisioning 13m persistentvolume-controller waiting for a volume to be created, either by external provisioner "k8s-sigs.io/nfs-subdir-external-provisioner" or manually created by system administrator Normal Provisioning 13m k8s-sigs.io /nfs-subdir-external-provisioner_nfs-client-provisioner-57d6d9d5f6-dcxgq_259532a3-4dba-4183-be6d-8e8b320fc778 External provisioner is provisioning volume for claim "default/nfs-pvc-test" Normal ProvisioningSucceeded 13m k8s-sigs.io /nfs-subdir-external-provisioner_nfs-client-provisioner-57d6d9d5f6-dcxgq_259532a3-4dba-4183-be6d-8e8b320fc778 Successfully provisioned volume pvc-8ed67f7d-d829-4d87-8c66-d8a85f50772f |
创建Pod,测试数据是否持久。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
|
[root@kn-server-master01-13 nfs-provisioner] # cat nginx-pvc-test.yaml apiVersion: v1 kind: Pod metadata: name: nginx-sc spec: containers: - name: nginx image: nginx volumeMounts: - name: nginx-page mountPath: /usr/share/nginx/html volumes: - name: nginx-page persistentVolumeClaim: claimName: nfs-pvc- test [root@kn-server-master01-13 nfs-provisioner] # kubectl apply -f nginx-pvc-test.yaml pod /nginx-sc created [root@kn-server-master01-13 nfs-provisioner] # kubectl describe pvc Name: nfs-pvc- test Namespace: default StorageClass: nfs-provisioner-storage Status: Bound Volume: pvc-8ed67f7d-d829-4d87-8c66-d8a85f50772f Labels: <none> Annotations: pv.kubernetes.io /bind-completed : yes pv.kubernetes.io /bound-by-controller : yes volume.beta.kubernetes.io /storage-provisioner : k8s-sigs.io /nfs-subdir-external-provisioner Finalizers: [kubernetes.io /pvc-protection ] Capacity: 512Mi Access Modes: RWX VolumeMode: Filesystem Used By: nginx-sc 可以看到的是nginx-sc这个Pod在使用这个PVC。 和上面名称是一致的。 [root@kn-server-master01-13 nfs-provisioner] # kubectl get pods nginx-sc NAME READY STATUS RESTARTS AGE nginx-sc 1 /1 Running 0 2m43s 尝试写入数据 [root@kn-server-node02-15 data] # echo "haitang" > /data/default/nfs-pvc-test/index.html 访问测试。 [root@kn-server-master01-13 nfs-provisioner] # curl 192.168.2.83 haitang |
以上就是kubernetes数据持久化StorageClass动态供给实现详解的详细内容,更多关于kubernetes数据持久化StorageClass的资料请关注服务器之家其它相关文章!
原文链接:https://www.cnblogs.com/xunweidezui/p/16930975.html