K8S - 理解volumeMounts 中的subpath

2024-09-01 07:20

本文主要是介绍K8S - 理解volumeMounts 中的subpath,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

在上一篇文章中
springboot service如何动态读取外部配置文件

介绍了springboot 中如何实时读取外部配置文件的内容



部署在K8S

接下来我把它部署在k8s



首先, 我们把配置文件放入项目某个目录

这步部是必须的, 毕竟我们要引入是项目外部的文件, 这一步只是方便在不同CICD 环境下的docker 构建
我们就放在 src 外面的configs folder
在这里插入图片描述


修改docker file

# use the basic openjdk image
FROM openjdk:17# setup working directory
WORKDIR /app# create /app/config
RUN mkdir -p /app/config/config2# copy configs files from project folder to /app/config
# When using COPY with more than one source file, the destination must be a directory and end with a /
COPY configs/external-config.properties /app/config/
COPY configs/external-config2.properties /app/config/config2/# copy and rename jar file into container
COPY target/*.jar app.jar# expose port
EXPOSE 8080# define environment variable, and provide a default value
ENV APP_ENVIRONMENT=dev# define the default startup command, it could be overridden in k8s
# CMD java -jar -Dserver.port=8080 -Dspring.config.name=application-${APP_ENVIRONMENT} app.jar
CMD java -jar -Dserver.port=8080 app.jar --spring.profiles.active=${APP_ENVIRONMENT}

需要创建/app/configs 并把配置文件复制到这里, 因为springboot service 会从这个path读取
注意有两个config 文件, 他们并不在同1个folder
external-config2.properties 在subfolder config2里面



构建镜像和部署镜像到GAR

这一步我们用google cloudbuild 完成
至于cloudbuild 的介绍请参考
初探 Google 云原生的CICD - CloudBuild

cloudbuild-gcr.yaml

# just to update the docker image to GAR with the pom.xml versionsteps:- id: run maven installname: maven:3.9.6-sapmachine-17 # https://hub.docker.com/_/mavenentrypoint: bashargs:- '-c'- |whoamiset -xpwdmvn installcat pom.xml | grep -m 1 "<version>" | sed -e 's/.*<version>\([^<]*\)<\/version>.*/\1/' > /workspace/version.txtecho "Version: $(cat /workspace/version.txt)"- id: build and push docker imagename: 'gcr.io/cloud-builders/docker'entrypoint: bashargs:- '-c'- |set -xecho "Building docker image with tag: $(cat /workspace/version.txt)"docker build -t $_GAR_BASE/$PROJECT_ID/$_DOCKER_REPO_NAME/${_APP_NAME}:$(cat /workspace/version.txt) .docker push $_GAR_BASE/$PROJECT_ID/$_DOCKER_REPO_NAME/${_APP_NAME}:$(cat /workspace/version.txt)logsBucket: gs://jason-hsbc_cloudbuild/logs/
options: # https://cloud.google.com/cloud-build/docs/build-config#optionslogging: GCS_ONLY # or CLOUD_LOGGING_ONLY https://cloud.google.com/cloud-build/docs/build-config#loggingsubstitutions:_DOCKER_REPO_NAME: my-docker-repo_APP_NAME: cloud-order_GAR_BASE: europe-west2-docker.pkg.dev

部署完成后, 我们得到1个gar的对应image的url
europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/cloud-order:1.1.0



编写k8s yaml 和部署到k8s

apiVersion: apps/v1
kind: Deployment
metadata:labels: # label of this deploymentapp: cloud-order # custom definedauthor: nvd11name: deployment-cloud-order # name of this deploymentnamespace: default
spec:replicas: 3            # desired replica count, Please note that the replica Pods in a Deployment are typically distributed across multiple nodes.revisionHistoryLimit: 10 # The number of old ReplicaSets to retain to allow rollbackselector: # label of the Pod that the Deployment is managing,, it's mandatory, without it , we will get this error # error: error validating data: ValidationError(Deployment.spec.selector): missing required field "matchLabels" in io.k8s.apimachinery.pkg.apis.meta.v1.LabelSelector ..matchLabels:app: cloud-orderstrategy: # Strategy of upodatetype: RollingUpdate # RollingUpdate or RecreaterollingUpdate:maxSurge: 25% # The maximum number of Pods that can be created over the desired number of Pods during the updatemaxUnavailable: 25% # The maximum number of Pods that can be unavailable during the updatetemplate: # Pod templatemetadata:labels:app: cloud-order # label of the Pod that the Deployment is managing. must match the selector, otherwise, will get the error Invalid value: map[string]string{"app":"bq-api-xxx"}: `selector` does not match template `labels`spec: # specification of the Podcontainers:- image: europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/cloud-order:1.1.0 # image of the containerimagePullPolicy: Alwaysname: container-cloud-ordercommand: ["bash"]args: - "-c"- |java -jar -Dserver.port=8080 app.jar --spring.profiles.active=$APP_ENVIRONMENTenv: # set env varaibles- name: APP_ENVIRONMENT # name of the environment variablevalue: prod # value of the environment variablerestartPolicy: Always # Restart policy for all containers within the PodterminationGracePeriodSeconds: 10 # The period of time in seconds given to the Pod to terminate gracefully

没什么特别

部署:

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl apply -f deployment-cloud-order-with-subpath.yaml 
deployment.apps/deployment-cloud-order created
gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl get pods
NAME                                         READY   STATUS      RESTARTS       AGE
deployment-bq-api-service-6f6ffc7866-8djx9   1/1     Running     3 (12d ago)    17d
deployment-bq-api-service-6f6ffc7866-g4854   1/1     Running     12 (12d ago)   61d
deployment-bq-api-service-6f6ffc7866-lwxt7   1/1     Running     14 (12d ago)   63d
deployment-bq-api-service-6f6ffc7866-mxwcq   1/1     Running     11 (12d ago)   61d
deployment-cloud-order-58ddcf894d-8pjsz      1/1     Running     0              5s
deployment-cloud-order-58ddcf894d-mp4js      1/1     Running     0              5s
deployment-cloud-order-58ddcf894d-sszjd      1/1     Running     0              5s

检查容器内的配置文件:

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl exec -it deployment-cloud-order-58ddcf894d-8pjsz -- /bin/bash
bash-4.4# pwd
/app
bash-4.4# ls config
config2  external-config.properties
bash-4.4# ls config/config2/
external-config2.properties
bash-4.4# 



测试api

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ curl http://www.jp-gcp-vms.cloud:8085/cloud-order/actuator/info | jq .% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100  1842    0  1842    0     0   1827      0 --:--:--  0:00:01 --:--:--  1829
{"app": "Cloud Order Service","appEnvProfile": "prod","version": "1.1.0","hostname": "deployment-cloud-order-58ddcf894d-8pjsz","dbUrl": "jdbc:mysql://192.168.0.42:3306/demo_cloud_order?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description": "This is a simple Spring Boot application to for cloud order...","customConfig1": "value of config1","customConfig2": "value of config2",}
}

注意 customConfig1 和 customConfig2 的值, 部署是成功的



For customConfig2 使用configMap

由于customConfig2 是实时更新的
我们尝试用configmap 来取代 上面external-config2.properties 的配置
注意这里的值改成 value of config2 - from k8s configmap 方便区分



构建1个external-config2 的configmap 资源对象

configmap-cloud-order-external-config2.yaml

apiVersion: v1
kind: ConfigMap
metadata:name: configmap-cloud-order-external-config2
data:external.custom.config2: external.custom.config2=value of config2 - from k8s configmap

部署:

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl apply -f configmap-cloud-order-external-config2.yaml 
configmap/configmap-cloud-order-external-config2 created
gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl describe cm configmap-cloud-order-external-config2
Name:         configmap-cloud-order-external-config2
Namespace:    default
Labels:       <none>
Annotations:  <none>Data
====
external.custom.config2:
----
external.custom.config2=value of config2 - from k8s configmapBinaryData
====Events:  <none>



修改deployment yaml 引入configmap

apiVersion: apps/v1
kind: Deployment
metadata:labels: # label of this deploymentapp: cloud-order # custom definedauthor: nvd11name: deployment-cloud-order # name of this deploymentnamespace: default
spec:replicas: 3            # desired replica count, Please note that the replica Pods in a Deployment are typically distributed across multiple nodes.revisionHistoryLimit: 10 # The number of old ReplicaSets to retain to allow rollbackselector: # label of the Pod that the Deployment is managing,, it's mandatory, without it , we will get this error # error: error validating data: ValidationError(Deployment.spec.selector): missing required field "matchLabels" in io.k8s.apimachinery.pkg.apis.meta.v1.LabelSelector ..matchLabels:app: cloud-orderstrategy: # Strategy of upodatetype: RollingUpdate # RollingUpdate or RecreaterollingUpdate:maxSurge: 25% # The maximum number of Pods that can be created over the desired number of Pods during the updatemaxUnavailable: 25% # The maximum number of Pods that can be unavailable during the updatetemplate: # Pod templatemetadata:labels:app: cloud-order # label of the Pod that the Deployment is managing. must match the selector, otherwise, will get the error Invalid value: map[string]string{"app":"bq-api-xxx"}: `selector` does not match template `labels`spec: # specification of the Podcontainers:- image: europe-west2-docker.pkg.dev/jason-hsbc/my-docker-repo/cloud-order:1.1.0 # image of the containerimagePullPolicy: Alwaysname: container-cloud-ordercommand: ["bash"]args: - "-c"- |java -jar -Dserver.port=8080 app.jar --spring.profiles.active=$APP_ENVIRONMENTenv: # set env varaibles- name: APP_ENVIRONMENT # name of the environment variablevalue: prod # value of the environment variablevolumeMounts:- name: volume-external-config2mountPath: /app/config/config2/volumes:- name: volume-external-config2configMap:name: configmap-cloud-order-external-config2items:- key: external.custom.config2path: external-config2.properties # name of the file to be mountedrestartPolicy: Always # Restart policy for all containers within the PodterminationGracePeriodSeconds: 10 # The period of time in seconds given to the Pod to terminate gracefully

注意这里使用了 volume 和 volumemount



重新部署 cloud-order service

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl delete deploy deployment-cloud-order
deployment.apps "deployment-cloud-order" deleted
gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl apply -f deployment-cloud-order-with-subpath.yaml 
deployment.apps/deployment-cloud-order created



测试api

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ curl http://www.jp-gcp-vms.cloud:8085/cloud-order/actuator/info | jq .% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100  1863    0  1863    0     0   3888      0 --:--:-- --:--:-- --:--:--  3889
{"app": "Cloud Order Service","appEnvProfile": "prod","version": "1.1.0","hostname": "deployment-cloud-order-69d4cd76d6-hwtfj","dbUrl": "jdbc:mysql://192.168.0.42:3306/demo_cloud_order?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description": "This is a simple Spring Boot application to for cloud order...","customConfig1": "value of config1","customConfig2": "value of config2 - from k8s configmap",

并没什么大问题, 证明configmap 的配置是可以覆盖docker里面定义的配置文件的



实时修改configmap 的配置

这时我们修改一下 configmap configmap-cloud-order-external-config2 里的值
由 external.custom.config2: external.custom.config2=value of config2 - from k8s configmap
改成 external.custom.config2: external.custom.config2=value of config2 - from k8s configmap updated!

apiVersion: v1
kind: ConfigMap
metadata:name: configmap-cloud-order-external-config2
data:external.custom.config2: external.custom.config2=value of config2 - from k8s configmap updated!

更新:

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl apply -f configmap-cloud-order-external-config2.yaml 
configmap/configmap-cloud-order-external-config2 configured

等半分钟后 (1是 k8s 需要时间把 configmap的值刷新到 pods里的volumes, 2时是springboot本身需要定时器去重新获取值)
再测试api

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ curl http://www.jp-gcp-vms.cloud:8085/cloud-order/actuator/info | jq .% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100  1872    0  1872    0     0   3900      0 --:--:-- --:--:-- --:--:--  3900
{"app": "Cloud Order Service","appEnvProfile": "prod","version": "1.1.0","hostname": "deployment-cloud-order-69d4cd76d6-qj98f","dbUrl": "jdbc:mysql://192.168.0.42:3306/demo_cloud_order?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description": "This is a simple Spring Boot application to for cloud order...","customConfig1": "value of config1","customConfig2": "value of config2 - from k8s configmap updated!",

果然customConfig2的值自动更新了, 正是我们想要的
而且容器里的文件也的确更新了

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl exec -it deployment-cloud-order-69d4cd76d6-hwtfj -- /bin/bash
bash-4.4# pwd
/app
bash-4.4# ls config
config2  external-config.properties
bash-4.4# cat config/config2/external-config2.properties 
external.custom.config2=value of config2 - from k8s configmap updated!



volumes mount 会覆盖整个文件夹(其他文件被删除)

下面来点整活
我们修改一下deployment 的yaml 配置, 把configmap的值 mark成另1个文件名

  volumeMounts:- name: volume-external-config2mountPath: /app/config/config2/volumes:- name: volume-external-config2configMap:name: configmap-cloud-order-external-config2items:- key: external.custom.config2path: external-config2-1.properties # name of the file to be mounted

按照设想, 容器内的configmap里的
/app/config/config2/ 文件内将会有两个文件

dockerfile 定义的external-config2.properties
和 k8s 定义的 external-config2-1.properties

实际部署测试:

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ curl http://www.jp-gcp-vms.cloud:8085/cloud-order/actuator/info | jq .% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100  1837    0  1837    0     0    926      0 --:--:--  0:00:01 --:--:--   926
{"app": "Cloud Order Service","appEnvProfile": "prod","version": "1.1.0","hostname": "deployment-cloud-order-6878b85d44-cdvlc","dbUrl": "jdbc:mysql://192.168.0.42:3306/demo_cloud_order?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true","description": "This is a simple Spring Boot application to for cloud order...","customConfig1": "value of config1","customConfig2": "not defined",

customConfig2 居然是not defined
检查容器
发现 external-config2.properties 没了

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl exec -it deployment-cloud-order-6878b85d44-cdvlc -- /bin/bash
bash-4.4# ls
app.jar  config
bash-4.4# ls config
config2  external-config.properties
bash-4.4# ls config/config2/
external-config2-1.properties
bash-4.4# 

原因是 当k8s 把 volume volume-external-config2 mount在 /app/config/config2 中的时候, 原来的文件就会消失



尝试把新文件mount到1个subfolder

解决方法1:
把 volume volume-external-config2 mount在 /app/config/config2/config-sub
应该可以解决

我们修改deployment yaml

  volumeMounts:- name: volume-external-config2mountPath: /app/config/config2/config-subvolumes:- name: volume-external-config2configMap:name: configmap-cloud-order-external-config2items:- key: external.custom.config2path: external-config2-1.properties # name of the file to be mounted

mountPath 改成了/app/config/config2/config-sub

这种方法是可行的

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl exec -it deployment-cloud-order-654974f855-cmdz7 -- /bin/bash
bash-4.4# ls
app.jar  config
bash-4.4# cd config
bash-4.4# ls
config2  external-config.properties
bash-4.4# cd config2
bash-4.4# ls
config-sub  external-config2.properties
bash-4.4# cd config-sub
bash-4.4# ls
external-config2-1.properties
bash-4.4# 

而且证明了, mountpath中的路径不存在的话, k8s会尝试创建。



利用subpath 去把文件mount在同样的folder

但是上面的做法, 能把新文件mount在1个字母中, 的确不会另旧文件消失,但是并直接真正解决问题

K8S 提供了subpath 的方法:
我们修改deployment yaml

        volumeMounts:- name: volume-external-config2mountPath: /app/config/config2/external-config2-1.properties # if we need to use subpath, need to provide the filename as wellsubPath: external-config2-1.properties # name of the file to be mounted, if we use subpath, the other files in that same folder will not dispearvolumes:- name: volume-external-config2configMap:name: configmap-cloud-order-external-config2items:- key: external.custom.config2path: external-config2-1.properties # name of the file to be mounted                                                               

注意这里有两个改动:

  1. mountpath 上加上了文件名
  2. 添加subpath 配置, 其值还是文件名

这次的确能令两个配置文件同时存在, 原来的文件external-config2.properties 并没有被抹除

gateman@MoreFine-S500:~/projects/coding/k8s-s/service-case/cloud-order$ kubectl exec -it deployment-cloud-order-7775b8c7cd-jnv7v -- /bin/bash
bash-4.4# pwd
/app
bash-4.4# ls
app.jar  config
bash-4.4# ls config
config2  external-config.properties
bash-4.4# ls config/config2/
external-config2-1.properties  external-config2.properties
bash-4.4# 



subpath的一些限制

k8s 设计的 subpath 其实并不是那么直接
参考:
https://hackmd.io/@maelvls/kubernetes-subpath?utm_source=preview-mode&utm_medium=rec

第1个限制就是 subpath 只能1个文件1个文件地mount, 不支持mount整个folder

参考:
https://kubernetes.io/docs/concepts/configuration/configmap/

第2个限制就是, 用subpath mount的configmap , 即使configmap的值被外部修改, 也不会同步到容器…

这篇关于K8S - 理解volumeMounts 中的subpath的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1126304

相关文章

认识、理解、分类——acm之搜索

普通搜索方法有两种:1、广度优先搜索;2、深度优先搜索; 更多搜索方法: 3、双向广度优先搜索; 4、启发式搜索(包括A*算法等); 搜索通常会用到的知识点:状态压缩(位压缩,利用hash思想压缩)。

【生成模型系列(初级)】嵌入(Embedding)方程——自然语言处理的数学灵魂【通俗理解】

【通俗理解】嵌入(Embedding)方程——自然语言处理的数学灵魂 关键词提炼 #嵌入方程 #自然语言处理 #词向量 #机器学习 #神经网络 #向量空间模型 #Siri #Google翻译 #AlexNet 第一节:嵌入方程的类比与核心概念【尽可能通俗】 嵌入方程可以被看作是自然语言处理中的“翻译机”,它将文本中的单词或短语转换成计算机能够理解的数学形式,即向量。 正如翻译机将一种语言

90、k8s之secret+configMap

一、secret配置管理 配置管理: 加密配置:保存密码,token,其他敏感信息的k8s资源 应用配置:我们需要定制化的给应用进行配置,我们需要把定制好的配置文件同步到pod当中容器 1.1、加密配置: secret: [root@master01 ~]# kubectl get secrets ##查看加密配置[root@master01 ~]# kubectl get se

【C++高阶】C++类型转换全攻略:深入理解并高效应用

📝个人主页🌹:Eternity._ ⏩收录专栏⏪:C++ “ 登神长阶 ” 🤡往期回顾🤡:C++ 智能指针 🌹🌹期待您的关注 🌹🌹 ❀C++的类型转换 📒1. C语言中的类型转换📚2. C++强制类型转换⛰️static_cast🌞reinterpret_cast⭐const_cast🍁dynamic_cast 📜3. C++强制类型转换的原因📝

K8S(Kubernetes)开源的容器编排平台安装步骤详解

K8S(Kubernetes)是一个开源的容器编排平台,用于自动化部署、扩展和管理容器化应用程序。以下是K8S容器编排平台的安装步骤、使用方式及特点的概述: 安装步骤: 安装Docker:K8S需要基于Docker来运行容器化应用程序。首先要在所有节点上安装Docker引擎。 安装Kubernetes Master:在集群中选择一台主机作为Master节点,安装K8S的控制平面组件,如AP

深入理解RxJava:响应式编程的现代方式

在当今的软件开发世界中,异步编程和事件驱动的架构变得越来越重要。RxJava,作为响应式编程(Reactive Programming)的一个流行库,为Java和Android开发者提供了一种强大的方式来处理异步任务和事件流。本文将深入探讨RxJava的核心概念、优势以及如何在实际项目中应用它。 文章目录 💯 什么是RxJava?💯 响应式编程的优势💯 RxJava的核心概念

如何通俗理解注意力机制?

1、注意力机制(Attention Mechanism)是机器学习和深度学习中一种模拟人类注意力的方法,用于提高模型在处理大量信息时的效率和效果。通俗地理解,它就像是在一堆信息中找到最重要的部分,把注意力集中在这些关键点上,从而更好地完成任务。以下是几个简单的比喻来帮助理解注意力机制: 2、寻找重点:想象一下,你在阅读一篇文章的时候,有些段落特别重要,你会特别注意这些段落,反复阅读,而对其他部分

深入理解数据库的 4NF:多值依赖与消除数据异常

在数据库设计中, "范式" 是一个常常被提到的重要概念。许多初学者在学习数据库设计时,经常听到第一范式(1NF)、第二范式(2NF)、第三范式(3NF)以及 BCNF(Boyce-Codd范式)。这些范式都旨在通过消除数据冗余和异常来优化数据库结构。然而,当我们谈到 4NF(第四范式)时,事情变得更加复杂。本文将带你深入了解 多值依赖 和 4NF,帮助你在数据库设计中消除更高级别的异常。 什么是

【Kubernetes】K8s 的安全框架和用户认证

K8s 的安全框架和用户认证 1.Kubernetes 的安全框架1.1 认证:Authentication1.2 鉴权:Authorization1.3 准入控制:Admission Control 2.Kubernetes 的用户认证2.1 Kubernetes 的用户认证方式2.2 配置 Kubernetes 集群使用密码认证 Kubernetes 作为一个分布式的虚拟

分布式系统的个人理解小结

分布式系统:分的微小服务,以小而独立的业务为单位,形成子系统。 然后分布式系统中需要有统一的调用,形成大的聚合服务。 同时,微服务群,需要有交流(通讯,注册中心,同步,异步),有管理(监控,调度)。 对外服务,需要有控制的对外开发,安全网关。