本文主要是介绍SeaTunnel 2.3.4 Cluster in K8S,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
参考:seatunnel k8s运行zeta引擎(cluster-mode模式)_apache seatunnel zeta 启动-CSDN博客
以上参考使用的是2.3.3版本
下载2.3.4版本, 上dlcdn.apache.org下载 ,官网下载有问题
wget https://dlcdn.apache.org/seatunnel/2.3.4/apache-seatunnel-2.3.4-bin.tar.gz
解压
configmap配置,可考虑指定namespace
kubectl create configmap hazelcast-client --from-file= config/hazelcast-client.yaml
kubectl create configmap hazelcast --from-file=config/hazelcast.yaml
kubectl create configmap seatunnelmap --from-file=config/seatunnel.yaml
使用Reloader实现更新configmap后自动重启pod
wget https://raw.githubusercontent.com/stakater/Reloader/master/deployments/kubernetes/reloader.yaml
修改namespace
%s/default/seatunnel/g
kubectl apply -f reloader.yaml
修改maven配置,使用私服下载
进入 .mvn/wrapper/maven-wrapper.properties
可以看到需要使用apache-maven-3.8.4-bin.zip
进入~/.m2/wrapper/dists/apache-maven-3.8.4-bin/52ccbt68d252mdldqsfsn03jlf/apache-maven-3.8.4/conf
修改setting.xml配置即可
下载connector,直接使用私服下载
bin/install-plugin.sh
connectors目录会下载相关的connectors jar包
连接mysql需要一个驱动,mysql 8.0需要对应的jar
plugins/jdbc/lib/mysql-connector-java-8.0.30.jar
把下载好的完成的目录打包,并准备上传至镜像文件
tar -cvf apache-seatunnel-2.3.4.tar apache-seatunnel-2.3.4
编辑 dockerfile
FROM flink:1.18
ENV SEATUNNEL_VERSION="2.3.4"
COPY ./apache-seatunnel-${SEATUNNEL_VERSION}.tar /opt/apache-seatunnel-${SEATUNNEL_VERSION}.tar
WORKDIR /opt
RUN tar -xvf apache-seatunnel-${SEATUNNEL_VERSION}.tar
RUN mv apache-seatunnel-${SEATUNNEL_VERSION} seatunnel
RUN rm -f /opt/apache-seatunnel-${SEATUNNEL_VERSION}.tar
WORKDIR /opt/seatunnel
docker build -t yourharbo/yourlib/seatunnel:2.3.4 -f dockerfile-seatunnel-2.3.4 .
docker push yourharbo/yourlib/seatunnel:2.3.4
提前下载镜像前上传至私服
docker pull ghcr.io/stakater/reloader:v1.0.69
创建seatunnel-cluster.yml并执行
按以上方参考链接文章中的步骤执行即可
这篇关于SeaTunnel 2.3.4 Cluster in K8S的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!