helm安装harbor + nerdctl 制作push 镜像

参考 文章：Helm部署Harbor_helm harbor_风向决定发型丶的博客-CSDN博客

安装好后使用 nerd

containerd对接harbor_containerd 容器 insecure-registries 配置_柠是柠檬的檬的博客-CSDN博客

推送镜像  Containerd 对接私有镜像仓库 Harbor - 知乎 

接下来我们来测试下如何在 containerd 中使用 Harbor 镜像仓库。

首先我们需要将私有镜像仓库配置到 containerd 中去，修改 containerd 的配置文件 /etc/containerd/config.toml：

      [plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.configs."core.harbor.domain".tls]insecure_skip_verify = true[plugins."io.containerd.grpc.v1.cri".registry.configs."core.harbor.domain".auth]username = "admin"password = "Harbor12345"[plugins."io.containerd.grpc.v1.cri".registry.headers][plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".registry.mirrors."core.harbor.domain"]endpoint = ["https://core.harbor.domain"]

在 plugins."io.containerd.grpc.v1.cri".registry.configs 下面添加对应 core.harbor.domain 的配置信息，insecure_skip_verify = true 表示跳过安全校验，然后通过 plugins."io.containerd.grpc.v1.cri".registry.configs."core.harbor.domain".auth 配置 Harbor 镜像仓库的用户名和密码。

配置完成后重启 containerd：

systemctl restart containerd

现在我们使用 nerdctl 来进行登录：

$ nerdctl login -u admin core.harbor.domain
Enter Password:
ERRO[0004] failed to call tryLoginWithRegHost            error="failed to call rh.Client.Do: Get \"https://harbor.k8s.local/v2/\": x509: certificate signed by unknown authority" i=0
FATA[0004] failed to call rh.Client.Do: Get "https://harbor.k8s.local/v2/": x509: certificate signed by unknown authority
[root@master1 ~]#

可以看到还是会报证书相关的错误，只需要添加一个 --insecure-registry 参数即可解决该问题：

nerdctl login -u admin --insecure-registry core.harbor.domain
Enter Password:
WARN[0004] skipping verifying HTTPS certs for "harbor.k8s.local"
WARNING: Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded

制作一个es镜像：

在 Releases · medcl/elasticsearch-analysis-ik · GitHub 下载 es 的ik插件

unzip xx -d ik

FROM docker.elastic.co/elasticsearch/elasticsearch:8.8.2
# ik 下载地址 https://github.com/medcl/elasticsearch-analysis-ik/releases
COPY ik /usr/share/elasticsearch/plugins/ik
# COPY pinyin /usr/share/elasticsearch/plugins/pinyin

然后将该镜像重新 tag 成 Harbor 上的镜像地址：

harbor新增一个项目 es

nerdctl tag elasticsearch:latest core.harbor.domain/elasticsearch:latest

再执行 push 命令即可将镜像推送到 Harbor 上：

nerdctl -n k8s.io  push --insecure-registry core.harbor.domain/es/elasticsearch:latest

$ nerdctl push --insecure-registry harbor.k8s.local/library/busybox:1.35.0
INFO[0000] pushing as a reduced-platform image (application/vnd.docker.distribution.manifest.list.v2+json, sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726)
WARN[0000] skipping verifying HTTPS certs for "harbor.k8s.local"
index-sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:8cde9b8065696b65d7b7ffaefbab0262d47a5a9852bfd849799559d296d2e0cd: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:d8c0f97fc6a6ac400e43342e67d06222b27cecdb076cbf8a87f3a2a25effe81c:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 6.9 s                                                                    total:  2.2 Ki (333.0 B/s)

推送完成后，我们就可以在 Portal 页面上看到这个镜像的信息了：

镜像 push 成功，同样可以测试下 pull：

$ nerdctl rmi harbor.k8s.local/library/busybox:1.35.0
Untagged: harbor.k8s.local/library/busybox:1.35.0@sha256:8c40df61d40166f5791f44b3d90b77b4c7f59ed39a992fd9046886d3126ffa68
Deleted: sha256:cf4ac4fc01444f1324571ceb0d4f175604a8341119d9bb42bc4b2cb431a7f3a5
$ nerdctl rmi busybox:1.35.0
Untagged: docker.io/library/busybox:1.35.0@sha256:8c40df61d40166f5791f44b3d90b77b4c7f59ed39a992fd9046886d3126ffa68
Deleted: sha256:cf4ac4fc01444f1324571ceb0d4f175604a8341119d9bb42bc4b2cb431a7f3a5
$ nerdctl pull --insecure-registry  harbor.k8s.local/library/busybox:1.35.0
WARN[0000] skipping verifying HTTPS certs for "harbor.k8s.local"
harbor.k8s.local/library/busybox:1.35.0:                                          resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:8cde9b8065696b65d7b7ffaefbab0262d47a5a9852bfd849799559d296d2e0cd: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:d8c0f97fc6a6ac400e43342e67d06222b27cecdb076cbf8a87f3a2a25effe81c:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:fc0cda0e09ab32c72c61d272bb409da4e2f73165c7bf584226880c9b85438e63:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.7 s                                                                    total:  2.2 Ki (3.2 KiB/s)$ nerdctl images
REPOSITORY                          TAG       IMAGE ID        CREATED           PLATFORM       SIZE         BLOB SIZE
harbor.k8s.local/library/busybox    1.35.0    29fe0126b13c    17 seconds ago    linux/amd64    1.2 MiB      757.7 KiB

但是上面我们也可以看到单独使用 containerd 比如通过 nerdctl 或者 ctr 命令访问 Harbor 镜像仓库的时候即使跳过证书校验或者配置上 CA 证书也是会出现证书错误的，这个时候我们需要去跳过证书校验或者指定证书路径才行。

# 解决办法1.指定 -k 参数跳过证书校验。
$ ctr images pull --user admin:Harbor12345 -k harbor.k8s.local/library/busybox:1.35.0# 解决办法2.指定CA证书、Harbor 相关证书文件路径。
$ ctr images pull --user admin:Harbor12345 --tlscacert ca.crt harbor.k8s.local/library/busybox:1.35.0

但是如果直接使用 ctrctl 则是有效的：

$ crictl pull harbor.k8s.local/library/busybox@sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726
Image is up to date for sha256:d8c0f97fc6a6ac400e43342e67d06222b27cecdb076cbf8a87f3a2a25effe81c

如果想要在 Kubernetes 中去使用那么就需要将 Harbor 的认证信息以 Secret 的形式添加到集群中去：

$ kubectl create secret docker-registry harbor-auth --docker-server=https://harbor.k8s.local --docker-username=admin --docker-password=Harbor12345 --docker-email=info@ydzs.io -n default

然后我们使用上面的私有镜像仓库来创建一个 Pod：

# test-harbor.yaml
apiVersion: v1
kind: Pod
metadata:name: harbor-registry-test
spec:containers:- name: testimage: harbor.k8s.local/library/busybox:1.35.0args:- sleep- "3600"imagePullSecrets:- name: harbor-auth

创建后可以查看该 Pod 是否能正常获取镜像：

$ kubectl describe pod harbor-registry-test
Name:         harbor-registry-test
Namespace:    default
Priority:     0
Node:         node1/192.168.0.107
Start Time:   Thu, 07 Jul 2022 18:52:39 +0800
# ......
Events:Type    Reason     Age   From               Message----    ------     ----  ----               -------Normal  Scheduled  10s   default-scheduler  Successfully assigned default/harbor-registry-test to node1Normal  Pulling    10s   kubelet            Pulling image "harbor.k8s.local/library/busybox:1.35.0"Normal  Pulled     5s    kubelet            Successfully pulled image "harbor.k8s.local/library/busybox:1.35.0" in 4.670528883sNormal  Created    5s    kubelet            Created container testNormal  Started    5s    kubelet            Started container test

到这里证明上面我们的私有镜像仓库搭建成功了，大家可以尝试去创建一个私有的项目，然后创建一个新的用户，使用这个用户来进行 pull/push 镜像，Harbor 还具有其他的一些功能，比如镜像复制，Helm Chart 包托管等等，大家可以自行测试，感受下 Harbor 和官方自带的 registry 仓库的差别。