k8s学习(二十九) RocketMq集群(双主双从-同步)

2024-06-20 08:18
文章标签 云原生 集群 学习 k8s rocketmq 同步 二十九 双主双

本文主要是介绍k8s学习(二十九) RocketMq集群(双主双从-同步)，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

确认安装前已经安装好StorageClass，可参考：https://blog.csdn.net/u011943534/article/details/100887530

1、准备jdk镜像，并拷贝到服务器

docker pull java:8-alpine

2、制作rocketmq镜像
下载地址：https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/
将”rocketmq-all-4.8.0-bin-release.zip”拷贝至服务器，解压
修改bin/runbroker.sh,将内存调整为适当（比如调小）

JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"

编写Dockerfile-rocketmq:

FROM java:8-alpineRUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezoneENV LANG en_US.UTF-8ADD rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-all-4.8.0-bin-release
RUN mv /usr/local/rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-4.8.0 \
&& mkdir -p /data/rocketmq/storeCMD ["/bin/bash"]

docker build -f Dockerfile-rocketmq -t 172.16.10.190:80/library/rocketmq:4.8.0 .
docker push 172.16.10.190:80/library/rocketmq:4.8.0

3、制作nameserver镜像
使用第三步的rocketmq-all-4.8.0-bin-release
在conf/下添加namesrv.properties
内容为：

listenPort=20901

编写Dokerfile-namesrv:

FROM java:8-alpineRUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezoneENV LANG en_US.UTF-8ADD rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-all-4.8.0-bin-release
RUN mv /usr/local/rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-4.8.0WORKDIR /usr/local/rocketmq-4.8.0
CMD ["/usr/local/rocketmq-4.8.0/bin/mqnamesrv","-c","/usr/local/rocketmq-4.8.0/conf/namesrv.properties"]
docker build -f Dockerfile-nameserver -t 172.16.10.190:80/library/rocketmq-namesrv:4.8.0 .
docker push 172.16.10.190:80/library/rocketmq-namesrv:4.8.0

4、制作rocketmq-ui镜像
下载源码：
git clone https://github.com/apache/rocketmq-externals.git
下载maven:
wget http://mirrors.hust.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

tar -zvxf rocketmq-externals.tar.gz

将本目录下的”application.properties“中内容替换到rocketmq-externals/rocketmq-console/src/main/resources/application.properties中
编写Dockerfile-ui

FROM java:8-alpineRUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezoneENV LANG en_US.UTF-8
ENV MAVEN_HOME /usr/local/maven
ENV PATH $PATH:$MAVEN_HOME/binADD rocketmq-externals /usr/local/rocketmq-externals
ADD apache-maven-3.3.9 /usr/local/apache-maven-3.3.9
RUN mv /usr/local/apache-maven-3.3.9 /usr/local/mavenWORKDIR /usr/local/rocketmq-externals/rocketmq-console/
RUN mvn clean package -Dmaven.test.skip=trueWORKDIR /usr/local/rocketmq-externals/rocketmq-console/target
EXPOSE 8080
CMD ["nohup","java","-jar","rocketmq-console-ng-1.0.0.jar","&"]

5、准备configmap
broker-a.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=MASTER

broker-a-s.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH

broker-b.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=MASTER
flushDiskType=SYNC_FLUSH

broker-b-s.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH

根据以上文件生成configmap:

kubectl create configmap rocketmq-config --from-file=broker-a.properties --from-file=broker-b.properties --from-file=broker-a-s.properties --from-file=broker-b-s.properties  -n default

6、准备服务文件
broker-a-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-aname: broker-anamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30911selector:app: broker-a
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-anamespace: default
spec:serviceName: broker-areplicas: 1selector:matchLabels:app: broker-atemplate:metadata:labels:app: broker-aspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-atopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-aimage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-a.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-a.propertiessubPath: broker-a.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-config- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

broker-a-s-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-a-sname: broker-a-snamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30912selector:app: broker-a-s
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-a-snamespace: default
spec:serviceName: broker-a-sreplicas: 1selector:matchLabels:app: broker-a-stemplate:metadata:labels:app: broker-a-sspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-a-stopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-a-simage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-a-s.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-a-s.propertiessubPath: broker-a-s.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-configitems:- key: broker-a-s.propertiespath: broker-a-s.properties- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

broker-b-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-bname: broker-bnamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30913selector:app: broker-b
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-bnamespace: default
spec:serviceName: broker-breplicas: 1selector:matchLabels:app: broker-btemplate:metadata:labels:app: broker-bspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-btopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-bimage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-b.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-b.propertiessubPath: broker-b.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-config- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

broker-b-s-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-b-sname: broker-b-snamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30914selector:app: broker-b-s
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-b-snamespace: default
spec:serviceName: broker-b-sreplicas: 1selector:matchLabels:app: broker-b-stemplate:metadata:labels:app: broker-b-sspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-b-stopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-b-simage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-b-s.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-b-s.propertiessubPath: broker-b-s.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-configitems:- key: broker-b-s.propertiespath: broker-b-s.properties- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

namesrv-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: mq-namesrvname: mq-namesrvnamespace: default
spec:type: NodePortports:- port: 20901targetPort: 20901name: namesrv-portnodePort: 30915selector:app: mq-namesrv
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: mq-namesrvnamespace: default
spec:serviceName: mq-namesrvreplicas: 1selector:matchLabels:app: mq-namesrvtemplate:metadata:labels:app: mq-namesrvspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- mq-namesrvtopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: mq-namesrvimage: 172.16.10.190:80/library/rocketmq-namesrv:4.8.0imagePullPolicy: AlwaysvolumeMounts:- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20901initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: timehostPath:path: /usr/share/zoneinfo/Asia/Shanghai

rocketmq-externals-deployment.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: mq-externalsname: mq-externalsnamespace: default
spec:type: NodePortports:- port: 8080targetPort: 8080name: console-portnodePort: 30916selector:app: mq-externals
---
apiVersion: apps/v1
kind: Deployment
metadata:name: mq-externalsnamespace: default
spec:replicas: 1selector:matchLabels:app: mq-externalstemplate:metadata:labels:app: mq-externalsspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- mq-externalstopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: mq-externalsimage: 172.16.10.190:80/library/rocketmq-externals:v3imagePullPolicy: AlwaysvolumeMounts:- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 8080initialDelaySeconds: 15volumes:- name: timehostPath:path: /usr/share/zoneinfo/Asia/Shanghai

7、部署服务
将6步的文件拷贝至K8s

kubectl apply -f .

8、查看
http://172.16.10.160:30916
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

这篇关于k8s学习(二十九) RocketMq集群(双主双从-同步)的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！


http://www.chinasem.cn/article/1077590。

相关文章

HarmonyOS学习(七)——UI（五）常用布局总结

HarmonyOS学习(七)——UI（五）常用布局总结

自适应布局 1.1、线性布局（LinearLayout） 通过线性容器Row和Column实现线性布局。Column容器内的子组件按照垂直方向排列，Row组件中的子组件按照水平方向排列。 属性说明space通过space参数设置主轴上子组件的间距，达到各子组件在排列上的等间距效果alignItems设置子组件在交叉轴上的对齐方式，且在各类尺寸屏幕上表现一致，其中交叉轴为垂直时，取值为Vert
阅读更多...
Ilya-AI分享的他在OpenAI学习到的15个提示工程技巧

Ilya-AI分享的他在OpenAI学习到的15个提示工程技巧

Ilya（不是本人，claude AI）在社交媒体上分享了他在OpenAI学习到的15个Prompt撰写技巧。 以下是详细的内容： 提示精确化:在编写提示时,力求表达清晰准确。清楚地阐述任务需求和概念定义至关重要。例:不用"分析文本",而用"判断这段话的情感倾向:积极、消极还是中性"。 快速迭代:善于快速连续调整提示。熟练的提示工程师能够灵活地进行多轮优化。例:从"总结文章"到"用
阅读更多...
基于MySQL Binlog的Elasticsearch数据同步实践

基于MySQL Binlog的Elasticsearch数据同步实践

一、为什么要做 随着马蜂窝的逐渐发展，我们的业务数据越来越多，单纯使用 MySQL 已经不能满足我们的数据查询需求，例如对于商品、订单等数据的多维度检索。 使用 Elasticsearch 存储业务数据可以很好的解决我们业务中的搜索需求。而数据进行异构存储后，随之而来的就是数据同步的问题。 二、现有方法及问题 对于数据同步，我们目前的解决方案是建立数据中间表。把需要检索的业务数据，统一放到一张M
阅读更多...
服务器集群同步时间手记

服务器集群同步时间手记

1.时间服务器配置（必须root用户） （1）检查ntp是否安装 [root@node1 桌面]# rpm -qa|grep ntpntp-4.2.6p5-10.el6.centos.x86_64fontpackages-filesystem-1.41-1.1.el6.noarchntpdate-4.2.6p5-10.el6.centos.x86_64 （2）修改ntp配置文件 [r
阅读更多...
HDFS—集群扩容及缩容

HDFS—集群扩容及缩容

白名单：表示在白名单的主机IP地址可以，用来存储数据。 配置白名单步骤如下： 1）在NameNode节点的/opt/module/hadoop-3.1.4/etc/hadoop目录下分别创建whitelist 和blacklist文件 （1）创建白名单 [lytfly@hadoop102 hadoop]$ vim whitelist 在whitelist中添加如下主机名称，假如集群正常工作的节
阅读更多...
Hadoop集群数据均衡之磁盘间数据均衡

Hadoop集群数据均衡之磁盘间数据均衡

生产环境，由于硬盘空间不足，往往需要增加一块硬盘。刚加载的硬盘没有数据时，可以执行磁盘数据均衡命令。（Hadoop3.x新特性） plan后面带的节点的名字必须是已经存在的，并且是需要均衡的节点。 如果节点不存在，会报如下错误： 如果节点只有一个硬盘的话，不会创建均衡计划： （1）生成均衡计划 hdfs diskbalancer -plan hadoop102 （2）执行均衡计划 hd
阅读更多...
【前端学习】AntV G6-08 深入图形与图形分组、自定义节点、节点动画（下）

【前端学习】AntV G6-08 深入图形与图形分组、自定义节点、节点动画（下）

【课程链接】 AntV G6：深入图形与图形分组、自定义节点、节点动画（下）_哔哩哔哩_bilibili 本章十吾老师讲解了一个复杂的自定义节点中，应该怎样去计算和绘制图形，如何给一个图形制作不间断的动画，以及在鼠标事件之后产生动画。（有点难，需要好好理解） <!DOCTYPE html><html><head><meta charset="UTF-8"><title>06
阅读更多...
字节面试 | 如何测试RocketMQ、RocketMQ？

字节面试 | 如何测试RocketMQ、RocketMQ？

字节面试：RocketMQ是怎么测试的呢？ 答： 首先保证消息的消费正确、设计逆向用例，在验证消息内容为空等情况时的消费正确性； 推送大批量MQ，通过Admin控制台查看MQ消费的情况，是否出现消费假死、TPS是否正常等等问题。（上述都是临场发挥，但是RocketMQ真正的测试点，还真的需要探讨） 01 先了解RocketMQ 作为测试也是要简单了解RocketMQ。简单来说，就是一个分
阅读更多...
学习hash总结

学习hash总结

2014/1/29/   最近刚开始学hash，名字很陌生，但是hash的思想却很熟悉，以前早就做过此类的题，但是不知道这就是hash思想而已，说白了hash就是一个映射，往往灵活利用数组的下标来实现算法，hash的作用：1、判重；2、统计次数；
阅读更多...
零基础学习Redis(10) -- zset类型命令使用

零基础学习Redis(10) -- zset类型命令使用

zset是有序集合，内部除了存储元素外，还会存储一个score，存储在zset中的元素会按照score的大小升序排列，不同元素的score可以重复，score相同的元素会按照元素的字典序排列。 1. zset常用命令 1.1 zadd  zadd key [NX | XX] [GT | LT]   [CH] [INCR] score member [score member ...]
阅读更多...

Copyright China编程 23002807@qq.com

沪ICP备2023033072号-2