Clickhouse集群化(四)使用clickhouse-operator部署clickhouse集群

2024-08-27 07:28

本文主要是介绍Clickhouse集群化(四)使用clickhouse-operator部署clickhouse集群,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

clickhouse operator实际下就是帮助我们自动化的生产一些clickhouse配置文件信息,在目录/etc/clickhouse-server/的config.d conf.d users.d

1.1. 部署clickhouse operateor

下载clickhouse-operator.yaml文件

wget https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator/clickhouse-operator-install-bundle.yaml

文件下载失败可以使用下面的文件 部分镜像可能需要提前下载

提前docker pull下载相关镜像

altinity/clickhouse-operator:0.23.6

clickhouse/clickhouse-server:24.1.2.5

altinity/metrics-exporter:0.23.6

镜像

导出镜像

docker save -o xxx.tar xxx:tag

传输
scp xxx.tar root@192.168.157.xx:/home

导入镜像

docker load --input xxx.tar

# Template Parameters:
#
# KIND=ClickHouseInstallation
# SINGULAR=clickhouseinstallation
# PLURAL=clickhouseinstallations
# SHORT=chi
# OPERATOR_VERSION=0.23.6
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: clickhouseinstallations.clickhouse.altinity.comlabels:clickhouse.altinity.com/chop: 0.23.6
spec:group: clickhouse.altinity.comscope: Namespacednames:kind: ClickHouseInstallationsingular: clickhouseinstallationplural: clickhouseinstallationsshortNames:- chiversions:- name: v1served: truestorage: trueadditionalPrinterColumns:- name: versiontype: stringdescription: Operator versionpriority: 1 # show in wide viewjsonPath: .status.chop-version- name: clusterstype: integerdescription: Clusters countjsonPath: .status.clusters- name: shardstype: integerdescription: Shards countpriority: 1 # show in wide viewjsonPath: .status.shards- name: hoststype: integerdescription: Hosts countjsonPath: .status.hosts- name: taskIDtype: stringdescription: TaskIDpriority: 1 # show in wide viewjsonPath: .status.taskID- name: statustype: stringdescription: CHI statusjsonPath: .status.status- name: hosts-unchangedtype: integerdescription: Unchanged hosts countpriority: 1 # show in wide viewjsonPath: .status.hostsUnchanged- name: hosts-updatedtype: integerdescription: Updated hosts countpriority: 1 # show in wide viewjsonPath: .status.hostsUpdated- name: hosts-addedtype: integerdescription: Added hosts countpriority: 1 # show in wide viewjsonPath: .status.hostsAdded- name: hosts-completedtype: integerdescription: Completed hosts countjsonPath: .status.hostsCompleted- name: hosts-deletedtype: integerdescription: Hosts deleted countpriority: 1 # show in wide viewjsonPath: .status.hostsDeleted- name: hosts-deletetype: integerdescription: Hosts to be deleted countpriority: 1 # show in wide viewjsonPath: .status.hostsDelete- name: endpointtype: stringdescription: Client access endpointpriority: 1 # show in wide viewjsonPath: .status.endpoint- name: agetype: datedescription: Age of the resource# Displayed in all prioritiesjsonPath: .metadata.creationTimestampsubresources:status: {}schema:openAPIV3Schema:description: "define a set of Kubernetes resources (StatefulSet, PVC, Service, ConfigMap) which describe behavior one or more ClickHouse clusters"type: objectrequired:- specproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectstatus:type: objectdescription: "Current ClickHouseInstallation manifest status, contains many fields like a normalized configuration, clickhouse-operator version, current action and all applied action list, current taskID and all applied taskIDs and other"properties:chop-version:type: stringdescription: "ClickHouse operator version"chop-commit:type: stringdescription: "ClickHouse operator git commit SHA"chop-date:type: stringdescription: "ClickHouse operator build date"chop-ip:type: stringdescription: "IP address of the operator's pod which managed this CHI"clusters:type: integerminimum: 0description: "Clusters count"shards:type: integerminimum: 0description: "Shards count"replicas:type: integerminimum: 0description: "Replicas count"hosts:type: integerminimum: 0description: "Hosts count"status:type: stringdescription: "Status"taskID:type: stringdescription: "Current task id"taskIDsStarted:type: arraydescription: "Started task ids"nullable: trueitems:type: stringtaskIDsCompleted:type: arraydescription: "Completed task ids"nullable: trueitems:type: stringaction:type: stringdescription: "Action"actions:type: arraydescription: "Actions"nullable: trueitems:type: stringerror:type: stringdescription: "Last error"errors:type: arraydescription: "Errors"nullable: trueitems:type: stringhostsUnchanged:type: integerminimum: 0description: "Unchanged Hosts count"hostsUpdated:type: integerminimum: 0description: "Updated Hosts count"hostsAdded:type: integerminimum: 0description: "Added Hosts count"hostsCompleted:type: integerminimum: 0description: "Completed Hosts count"hostsDeleted:type: integerminimum: 0description: "Deleted Hosts count"hostsDelete:type: integerminimum: 0description: "About to delete Hosts count"pods:type: arraydescription: "Pods"nullable: trueitems:type: stringpod-ips:type: arraydescription: "Pod IPs"nullable: trueitems:type: stringfqdns:type: arraydescription: "Pods FQDNs"nullable: trueitems:type: stringendpoint:type: stringdescription: "Endpoint"generation:type: integerminimum: 0description: "Generation"normalized:type: objectdescription: "Normalized CHI requested"x-kubernetes-preserve-unknown-fields: truenormalizedCompleted:type: objectdescription: "Normalized CHI completed"x-kubernetes-preserve-unknown-fields: truehostsWithTablesCreated:type: arraydescription: "List of hosts with tables created by the operator"nullable: trueitems:type: stringusedTemplates:type: arraydescription: "List of templates used to build this CHI"nullable: truex-kubernetes-preserve-unknown-fields: trueitems:type: objectx-kubernetes-preserve-unknown-fields: truespec:type: object# x-kubernetes-preserve-unknown-fields: truedescription: |Specification of the desired behavior of one or more ClickHouse clustersMore info: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.mdproperties:taskID:type: stringdescription: |Allows to define custom taskID for CHI update and watch status of this update execution.Displayed in all .status.taskID* fields.By default (if not filled) every update of CHI manifest will generate random taskIDstop: &TypeStringBooltype: stringdescription: |Allows to stop all ClickHouse clusters defined in a CHI.Works as the following:- When `stop` is `1` operator sets `Replicas: 0` in each StatefulSet. Thie leads to having all `Pods` and `Service` deleted. All PVCs are kept intact.- When `stop` is `0` operator sets `Replicas: 1` and `Pod`s and `Service`s will created again and all retained PVCs will be attached to `Pod`s.enum:# List StringBoolXXX constants from model- ""- "0"- "1"- "False"- "false"- "True"- "true"- "No"- "no"- "Yes"- "yes"- "Off"- "off"- "On"- "on"- "Disable"- "disable"- "Enable"- "enable"- "Disabled"- "disabled"- "Enabled"- "enabled"restart:type: stringdescription: |In case 'RollingUpdate' specified, the operator will always restart ClickHouse pods during reconcile.This options is used in rare cases when force restart is required and is typically removed after the use in order to avoid unneeded restarts.enum:- ""- "RollingUpdate"troubleshoot:<<: *TypeStringBooldescription: |Allows to troubleshoot Pods during CrashLoopBack state.This may happen when wrong configuration applied, in this case `clickhouse-server` wouldn't start.Command within ClickHouse container is modified with `sleep` in order to avoid quick restartsand give time to troubleshoot via CLI.Liveness and Readiness probes are disabled as well.namespaceDomainPattern:type: stringdescription: |Custom domain pattern which will be used for DNS names of `Service` or `Pod`.Typical use scenario - custom cluster domain in Kubernetes clusterExample: %s.svc.my.testtemplating:type: object# nullable: truedescription: |Optional, applicable inside ClickHouseInstallationTemplate only.Defines current ClickHouseInstallationTemplate application options to target ClickHouseInstallation(s)."properties:policy:type: stringdescription: |When defined as `auto` inside ClickhouseInstallationTemplate, this ClickhouseInstallationTemplatewill be auto-added into ClickHouseInstallation, selectable by `chiSelector`.Default value is `manual`, meaning ClickHouseInstallation should request this ClickhouseInstallationTemplate explicitly.enum:- ""- "auto"- "manual"chiSelector:type: objectdescription: "Optional, defines selector for ClickHouseInstallation(s) to be templated with ClickhouseInstallationTemplate"# nullable: truex-kubernetes-preserve-unknown-fields: truereconciling:type: objectdescription: "Optional, allows tuning reconciling cycle for ClickhouseInstallation from clickhouse-operator side"# nullable: trueproperties:policy:type: stringdescription: |DISCUSSED TO BE DEPRECATEDSyntax sugarOverrides all three 'reconcile.host.wait.{exclude, queries, include}' values from the operator's configPossible values:- wait - should wait to exclude host, complete queries and include host back into the cluster- nowait - should NOT wait to exclude host, complete queries and include host back into the clusterenum:- ""- "wait"- "nowait"configMapPropagationTimeout:type: integerdescription: |Timeout in seconds for `clickhouse-operator` to wait for modified `ConfigMap` to propagate into the `Pod`More details: https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automaticallyminimum: 0maximum: 3600cleanup:type: objectdescription: "Optional, defines behavior for cleanup Kubernetes resources during reconcile cycle"# nullable: trueproperties:unknownObjects:type: objectdescription: |Describes what clickhouse-operator should do with found Kubernetes resources which should be managed by clickhouse-operator,but do not have `ownerReference` to any currently managed `ClickHouseInstallation` resource.Default behavior is `Delete`"# nullable: trueproperties:statefulSet: &TypeObjectsCleanuptype: stringdescription: "Behavior policy for unknown StatefulSet, `Delete` by default"enum:# List ObjectsCleanupXXX constants from model- ""- "Retain"- "Delete"pvc:type: string<<: *TypeObjectsCleanupdescription: "Behavior policy for unknown PVC, `Delete` by default"configMap:<<: *TypeObjectsCleanupdescription: "Behavior policy for unknown ConfigMap, `Delete` by default"service:<<: *TypeObjectsCleanupdescription: "Behavior policy for unknown Service, `Delete` by default"reconcileFailedObjects:type: objectdescription: |Describes what clickhouse-operator should do with Kubernetes resources which are failed during reconcile.Default behavior is `Retain`"# nullable: trueproperties:statefulSet:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed StatefulSet, `Retain` by default"pvc:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed PVC, `Retain` by default"configMap:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed ConfigMap, `Retain` by default"service:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed Service, `Retain` by default"defaults:type: objectdescription: |define default behavior for whole ClickHouseInstallation, some behavior can be re-define on cluster, shard and replica levelMore info: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specdefaults# nullable: trueproperties:replicasUseFQDN:<<: *TypeStringBooldescription: |define should replicas be specified by FQDN in `<host></host>`.In case of "no" will use short hostname and clickhouse-server will use kubernetes default suffixes for DNS lookup"yes" by defaultdistributedDDL:type: objectdescription: |allows change `<yandex><distributed_ddl></distributed_ddl></yandex>` settingsMore info: https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/#server-settings-distributed_ddl# nullable: trueproperties:profile:type: stringdescription: "Settings from this profile will be used to execute DDL queries"storageManagement:type: objectdescription: default storage management optionsproperties:provisioner: &TypePVCProvisionertype: stringdescription: "defines `PVC` provisioner - be it StatefulSet or the Operator"enum:- ""- "StatefulSet"- "Operator"reclaimPolicy: &TypePVCReclaimPolicytype: stringdescription: |defines behavior of `PVC` deletion.`Delete` by default, if `Retain` specified then `PVC` will be kept when deleting StatefulSetenum:- ""- "Retain"- "Delete"templates: &TypeTemplateNamestype: objectdescription: "optional, configuration of the templates names which will use for generate Kubernetes resources according to one or more ClickHouse clusters described in current ClickHouseInstallation (chi) resource"# nullable: trueproperties:hostTemplate:type: stringdescription: "optional, template name from chi.spec.templates.hostTemplates, which will apply to configure every `clickhouse-server` instance during render ConfigMap resources which will mount into `Pod`"podTemplate:type: stringdescription: "optional, template name from chi.spec.templates.podTemplates, allows customization each `Pod` resource during render and reconcile each StatefulSet.spec resource described in `chi.spec.configuration.clusters`"dataVolumeClaimTemplate:type: stringdescription: "optional, template name from chi.spec.templates.volumeClaimTemplates, allows customization each `PVC` which will mount for clickhouse data directory in each `Pod` during render and reconcile every StatefulSet.spec resource described in `chi.spec.configuration.clusters`"logVolumeClaimTemplate:type: stringdescription: "optional, template name from chi.spec.templates.volumeClaimTemplates, allows customization each `PVC` which will mount for clickhouse log directory in each `Pod` during render and reconcile every StatefulSet.spec resource described in `chi.spec.configuration.clusters`"serviceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for one `Service` resource which will created by `clickhouse-operator` which cover all clusters in whole `chi` resource"clusterServiceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for each `Service` resource which will created by `clickhouse-operator` which cover each clickhouse cluster described in `chi.spec.configuration.clusters`"shardServiceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for each `Service` resource which will created by `clickhouse-operator` which cover each shard inside clickhouse cluster described in `chi.spec.configuration.clusters`"replicaServiceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for each `Service` resource which will created by `clickhouse-operator` which cover each replica inside each shard inside each clickhouse cluster described in `chi.spec.configuration.clusters`"volumeClaimTemplate:type: stringdescription: "DEPRECATED! VolumeClaimTemplate is deprecated in favor of DataVolumeClaimTemplate and LogVolumeClaimTemplate"configuration:type: objectdescription: "allows configure multiple aspects and behavior for `clickhouse-server` instance and also allows describe multiple `clickhouse-server` clusters inside one `chi` resource"# nullable: trueproperties:zookeeper: &TypeZookeeperConfigtype: objectdescription: |allows configure <yandex><zookeeper>..</zookeeper></yandex> section in each `Pod` during generate `ConfigMap` which will mounted in `/etc/clickhouse-server/config.d/``clickhouse-operator` itself doesn't manage Zookeeper, please install Zookeeper separatelly look examples on https://github.com/Altinity/clickhouse-operator/tree/master/deploy/zookeeper/currently, zookeeper (or clickhouse-keeper replacement) used for *ReplicatedMergeTree table engines and for `distributed_ddl`More details: https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/#server-settings_zookeeper# nullable: trueproperties:nodes:type: arraydescription: "describe every available zookeeper cluster node for interaction"# nullable: trueitems:type: object#required:#  - hostproperties:host:type: stringdescription: "dns name or ip address for Zookeeper node"port:type: integerdescription: "TCP port which used to connect to Zookeeper node"minimum: 0maximum: 65535secure:<<: *TypeStringBooldescription: "if a secure connection to Zookeeper is required"session_timeout_ms:type: integerdescription: "session timeout during connect to Zookeeper"operation_timeout_ms:type: integerdescription: "one operation timeout during Zookeeper transactions"root:type: stringdescription: "optional root znode path inside zookeeper to store ClickHouse related data (replication queue or distributed DDL)"identity:type: stringdescription: "optional access credentials string with `user:password` format used when use digest authorization in Zookeeper"users:type: objectdescription: |allows configure <yandex><users>..</users></yandex> section in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/users.d/`you can configure password hashed, authorization restrictions, database level security row filters etc.More details: https://clickhouse.tech/docs/en/operations/settings/settings-users/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationusers# nullable: truex-kubernetes-preserve-unknown-fields: trueprofiles:type: objectdescription: |allows configure <yandex><profiles>..</profiles></yandex> section in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/users.d/`you can configure any aspect of settings profileMore details: https://clickhouse.tech/docs/en/operations/settings/settings-profiles/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationprofiles# nullable: truex-kubernetes-preserve-unknown-fields: truequotas:type: objectdescription: |allows configure <yandex><quotas>..</quotas></yandex> section in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/users.d/`you can configure any aspect of resource quotasMore details: https://clickhouse.tech/docs/en/operations/quotas/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationquotas# nullable: truex-kubernetes-preserve-unknown-fields: truesettings: &TypeSettingstype: objectdescription: |allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/`More details: https://clickhouse.tech/docs/en/operations/settings/settings/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationsettings# nullable: truex-kubernetes-preserve-unknown-fields: truefiles: &TypeFilestype: objectdescription: |allows define content of any setting file inside each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`every key in this object is the file nameevery value in this object is the file contentyou can use `!!binary |` and base64 for binary files, see details here https://yaml.org/type/binary.htmleach key could contains prefix like USERS, COMMON, HOST or config.d, users.d, cond.d, wrong prefixes will ignored, subfolders also will ignoredMore details: https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/05-settings-05-files-nested.yaml# nullable: truex-kubernetes-preserve-unknown-fields: trueclusters:type: arraydescription: |describes ClickHouse clusters layout and allows change settings on cluster-level, shard-level and replica-levelevery cluster is a set of StatefulSet, one StatefulSet contains only one Pod with `clickhouse-server`all Pods will rendered in <remote_server> part of ClickHouse configs, mounted from ConfigMap as `/etc/clickhouse-server/config.d/chop-generated-remote_servers.xml`Clusters will use for Distributed table engine, more details: https://clickhouse.tech/docs/en/engines/table-engines/special/distributed/If `cluster` contains zookeeper settings (could be inherited from top `chi` level), when you can create *ReplicatedMergeTree tables# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "cluster name, used to identify set of ClickHouse servers and wide used during generate names of related Kubernetes resources"minLength: 1# See namePartClusterMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"zookeeper:<<: *TypeZookeeperConfigdescription: |optional, allows configure <yandex><zookeeper>..</zookeeper></yandex> section in each `Pod` only in current ClickHouse cluster, during generate `ConfigMap` which will mounted in `/etc/clickhouse-server/config.d/`override top-level `chi.spec.configuration.zookeeper` settingssettings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` only in one cluster during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/`override top-level `chi.spec.configuration.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` on current cluster during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files`templates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected clusteroverride top-level `chi.spec.configuration.templates`schemaPolicy:type: objectdescription: |describes how schema is propagated within replicas and shardsproperties:replica:type: stringdescription: "how schema is propagated within a replica"enum:# List SchemaPolicyReplicaXXX constants from model- ""- "None"- "All"shard:type: stringdescription: "how schema is propagated between shards"enum:# List SchemaPolicyShardXXX constants from model- ""- "None"- "All"- "DistributedTablesOnly"insecure:<<: *TypeStringBooldescription: optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: optional, open secure ports for clustersecret:type: objectdescription: "optional, shared secret value to secure cluster communications"properties:auto:<<: *TypeStringBooldescription: "Auto-generate shared secret value to secure cluster communications"value:description: "Cluster shared secret value in plain text"type: stringvalueFrom:description: "Cluster shared secret source"type: objectproperties:secretKeyRef:description: |Selects a key of a secret in the clickhouse installation namespace.Should not be used if value is not empty.type: objectproperties:name:description: |Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#namestype: stringkey:description: The key of the secret to select from. Must be a valid secret key.type: stringoptional:description: Specify whether the Secret or its key must be definedtype: booleanrequired:- name- keylayout:type: objectdescription: |describe current cluster layout, how much shards in cluster, how much replica in shardallows override settings on each shard and replica separatelly# nullable: trueproperties:type:type: stringdescription: "DEPRECATED - to be removed soon"shardsCount:type: integerdescription: "how much shards for current ClickHouse cluster will run in Kubernetes, each shard contains shared-nothing part of data and contains set of replicas, cluster contains 1 shard by default"replicasCount:type: integerdescription: "how much replicas in each shards for current ClickHouse cluster will run in Kubernetes, each replica is a separate `StatefulSet` which contains only one `Pod` with `clickhouse-server` instance, every shard contains 1 replica by default"shards:type: arraydescription: "optional, allows override top-level `chi.spec.configuration`, cluster-level `chi.spec.configuration.clusters` settings for each shard separately, use it only if you fully understand what you do"# nullable: trueitems:type: objectproperties:name:type: stringdescription: "optional, by default shard name is generated, but you can override it and setup custom name"minLength: 1# See namePartShardMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"definitionType:type: stringdescription: "DEPRECATED - to be removed soon"weight:type: integerdescription: |optional, 1 by default, allows setup shard <weight> setting which will use during insert into tables with `Distributed` engine,will apply in <remote_servers> inside ConfigMap which will mount in /etc/clickhouse-server/config.d/chop-generated-remote_servers.xmlMore details: https://clickhouse.tech/docs/en/engines/table-engines/special/distributed/internalReplication:<<: *TypeStringBooldescription: |optional, `true` by default when `chi.spec.configuration.clusters[].layout.ReplicaCount` > 1 and 0 otherwiseallows setup <internal_replication> setting which will use during insert into tables with `Distributed` engine for insert only in one live replica and other replicas will download inserted data during replication,will apply in <remote_servers> inside ConfigMap which will mount in /etc/clickhouse-server/config.d/chop-generated-remote_servers.xmlMore details: https://clickhouse.tech/docs/en/engines/table-engines/special/distributed/settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` only in one shard during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/`override top-level `chi.spec.configuration.settings` and cluster-level `chi.spec.configuration.clusters.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` only in one shard during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files` and cluster-level `chi.spec.configuration.clusters.files`templates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected shardoverride top-level `chi.spec.configuration.templates` and cluster-level `chi.spec.configuration.clusters.templates`replicasCount:type: integerdescription: |optional, how much replicas in selected shard for selected ClickHouse cluster will run in Kubernetes, each replica is a separate `StatefulSet` which contains only one `Pod` with `clickhouse-server` instance,shard contains 1 replica by defaultoverride cluster-level `chi.spec.configuration.clusters.layout.replicasCount`minimum: 1replicas:type: arraydescription: |optional, allows override behavior for selected replicas from cluster-level `chi.spec.configuration.clusters` and shard-level `chi.spec.configuration.clusters.layout.shards`# nullable: trueitems:# Hosttype: objectproperties:name:type: stringdescription: "optional, by default replica name is generated, but you can override it and setup custom name"minLength: 1# See namePartReplicaMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"insecure:<<: *TypeStringBooldescription: |optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: |optional, open secure portstcpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `tcp` for selected replica, override `chi.spec.templates.hostTemplates.spec.tcpPort`allows connect to `clickhouse-server` via TCP Native protocol via kubernetes `Service`minimum: 1maximum: 65535tlsPort:type: integerminimum: 1maximum: 65535httpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `http` for selected replica, override `chi.spec.templates.hostTemplates.spec.httpPort`allows connect to `clickhouse-server` via HTTP protocol via kubernetes `Service`minimum: 1maximum: 65535httpsPort:type: integerminimum: 1maximum: 65535interserverHTTPPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `interserver` for selected replica, override `chi.spec.templates.hostTemplates.spec.interserverHTTPPort`allows connect between replicas inside same shard during fetch replicated data parts HTTP protocolminimum: 1maximum: 65535settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`override top-level `chi.spec.configuration.settings`, cluster-level `chi.spec.configuration.clusters.settings` and shard-level `chi.spec.configuration.clusters.layout.shards.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files`, cluster-level `chi.spec.configuration.clusters.files` and shard-level `chi.spec.configuration.clusters.layout.shards.files`templates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected replicaoverride top-level `chi.spec.configuration.templates`, cluster-level `chi.spec.configuration.clusters.templates` and shard-level `chi.spec.configuration.clusters.layout.shards.templates`replicas:type: arraydescription: "optional, allows override top-level `chi.spec.configuration` and cluster-level `chi.spec.configuration.clusters` configuration for each replica and each shard relates to selected replica, use it only if you fully understand what you do"# nullable: trueitems:type: objectproperties:name:type: stringdescription: "optional, by default replica name is generated, but you can override it and setup custom name"minLength: 1# See namePartShardMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`override top-level `chi.spec.configuration.settings`, cluster-level `chi.spec.configuration.clusters.settings` and will ignore if shard-level `chi.spec.configuration.clusters.layout.shards` presentMore details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files` and cluster-level `chi.spec.configuration.clusters.files`, will ignore if `chi.spec.configuration.clusters.layout.shards` presentstemplates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected replicaoverride top-level `chi.spec.configuration.templates`, cluster-level `chi.spec.configuration.clusters.templates`shardsCount:type: integerdescription: "optional, count of shards related to current replica, you can override each shard behavior on low-level `chi.spec.configuration.clusters.layout.replicas.shards`"minimum: 1shards:type: arraydescription: "optional, list of shards related to current replica, will ignore if `chi.spec.configuration.clusters.layout.shards` presents"# nullable: trueitems:# Hosttype: objectproperties:name:type: stringdescription: "optional, by default shard name is generated, but you can override it and setup custom name"minLength: 1# See namePartReplicaMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"insecure:<<: *TypeStringBooldescription: |optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: |optional, open secure portstcpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `tcp` for selected shard, override `chi.spec.templates.hostTemplates.spec.tcpPort`allows connect to `clickhouse-server` via TCP Native protocol via kubernetes `Service`minimum: 1maximum: 65535tlsPort:type: integerminimum: 1maximum: 65535httpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `http` for selected shard, override `chi.spec.templates.hostTemplates.spec.httpPort`allows connect to `clickhouse-server` via HTTP protocol via kubernetes `Service`minimum: 1maximum: 65535httpsPort:type: integerminimum: 1maximum: 65535interserverHTTPPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `interserver` for selected shard, override `chi.spec.templates.hostTemplates.spec.interserverHTTPPort`allows connect between replicas inside same shard during fetch replicated data parts HTTP protocolminimum: 1maximum: 65535settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in `Pod` only in one shard related to current replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`override top-level `chi.spec.configuration.settings`, cluster-level `chi.spec.configuration.clusters.settings` and replica-level `chi.spec.configuration.clusters.layout.replicas.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` only in one shard related to current replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files` and cluster-level `chi.spec.configuration.clusters.files`, will ignore if `chi.spec.configuration.clusters.layout.shards` presentstemplates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected replicaoverride top-level `chi.spec.configuration.templates`, cluster-level `chi.spec.configuration.clusters.templates`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates`templates:type: objectdescription: "allows define templates which will use for render Kubernetes resources like StatefulSet, ConfigMap, Service, PVC, by default, clickhouse-operator have own templates, but you can override it"# nullable: trueproperties:hostTemplates:type: arraydescription: "hostTemplate will use during apply to generate `clickhose-server` config files"# nullable: trueitems:type: object#required:#  - nameproperties:name:description: "template name, could use to link inside top-level `chi.spec.defaults.templates.hostTemplate`, cluster-level `chi.spec.configuration.clusters.templates.hostTemplate`, shard-level `chi.spec.configuration.clusters.layout.shards.temlates.hostTemplate`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates.hostTemplate`"type: stringportDistribution:type: arraydescription: "define how will distribute numeric values of named ports in `Pod.spec.containers.ports` and clickhouse-server configs"# nullable: trueitems:type: object#required:#  - typeproperties:type:type: stringdescription: "type of distribution, when `Unspecified` (default value) then all listen ports on clickhouse-server configuration in all Pods will have the same value, when `ClusterScopeIndex` then ports will increment to offset from base value depends on shard and replica index inside cluster with combination of `chi.spec.templates.podTemlates.spec.HostNetwork` it allows setup ClickHouse cluster inside Kubernetes and provide access via external network bypass Kubernetes internal network"enum:# List PortDistributionXXX constants- ""- "Unspecified"- "ClusterScopeIndex"spec:# Hosttype: objectproperties:name:type: stringdescription: "by default, hostname will generate, but this allows define custom name for each `clickhuse-server`"minLength: 1# See namePartReplicaMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"insecure:<<: *TypeStringBooldescription: |optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: |optional, open secure portstcpPort:type: integerdescription: |optional, setup `tcp_port` inside `clickhouse-server` settings for each Pod where current template will applyif specified, should have equal value with `chi.spec.templates.podTemplates.spec.containers.ports[name=tcp]`More info: https://clickhouse.tech/docs/en/interfaces/tcp/minimum: 1maximum: 65535tlsPort:type: integerminimum: 1maximum: 65535httpPort:type: integerdescription: |optional, setup `http_port` inside `clickhouse-server` settings for each Pod where current template will applyif specified, should have equal value with `chi.spec.templates.podTemplates.spec.containers.ports[name=http]`More info: https://clickhouse.tech/docs/en/interfaces/http/minimum: 1maximum: 65535httpsPort:type: integerminimum: 1maximum: 65535interserverHTTPPort:type: integerdescription: |optional, setup `interserver_http_port` inside `clickhouse-server` settings for each Pod where current template will applyif specified, should have equal value with `chi.spec.templates.podTemplates.spec.containers.ports[name=interserver]`More info: https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/#interserver-http-portminimum: 1maximum: 65535settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` where this template will apply during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` where this template will apply during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`templates:<<: *TypeTemplateNamesdescription: "be careful, this part of CRD allows override template inside template, don't use it if you don't understand what you do"podTemplates:type: arraydescription: |podTemplate will use during render `Pod` inside `StatefulSet.spec` and allows define rendered `Pod.spec`, pod scheduling distribution and pod zoneMore information: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatespodtemplates# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "template name, could use to link inside top-level `chi.spec.defaults.templates.podTemplate`, cluster-level `chi.spec.configuration.clusters.templates.podTemplate`, shard-level `chi.spec.configuration.clusters.layout.shards.temlates.podTemplate`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates.podTemplate`"generateName:type: stringdescription: "allows define format for generated `Pod` name, look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatesservicetemplates for details about aviailable template variables"zone:type: objectdescription: "allows define custom zone name and will separate ClickHouse `Pods` between nodes, shortcut for `chi.spec.templates.podTemplates.spec.affinity.podAntiAffinity`"#required:#  - valuesproperties:key:type: stringdescription: "optional, if defined, allows select kubernetes nodes by label with `name` equal `key`"values:type: arraydescription: "optional, if defined, allows select kubernetes nodes by label with `value` in `values`"# nullable: trueitems:type: stringdistribution:type: stringdescription: "DEPRECATED, shortcut for `chi.spec.templates.podTemplates.spec.affinity.podAntiAffinity`"enum:- ""- "Unspecified"- "OnePerHost"podDistribution:type: arraydescription: "define ClickHouse Pod distribution policy between Kubernetes Nodes inside Shard, Replica, Namespace, CHI, another ClickHouse cluster"# nullable: trueitems:type: object#required:#  - typeproperties:type:type: stringdescription: "you can define multiple affinity policy types"enum:# List PodDistributionXXX constants- ""- "Unspecified"- "ClickHouseAntiAffinity"- "ShardAntiAffinity"- "ReplicaAntiAffinity"- "AnotherNamespaceAntiAffinity"- "AnotherClickHouseInstallationAntiAffinity"- "AnotherClusterAntiAffinity"- "MaxNumberPerNode"- "NamespaceAffinity"- "ClickHouseInstallationAffinity"- "ClusterAffinity"- "ShardAffinity"- "ReplicaAffinity"- "PreviousTailAffinity"- "CircularReplication"scope:type: stringdescription: "scope for apply each podDistribution"enum:# list PodDistributionScopeXXX constants- ""- "Unspecified"- "Shard"- "Replica"- "Cluster"- "ClickHouseInstallation"- "Namespace"number:type: integerdescription: "define, how much ClickHouse Pods could be inside selected scope with selected distribution type"minimum: 0maximum: 65535topologyKey:type: stringdescription: "use for inter-pod affinity look to `pod.spec.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution.podAffinityTerm.topologyKey`, More info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity"metadata:type: objectdescription: |allows pass standard object's metadata from template to PodMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:# TODO specify PodSpectype: objectdescription: "allows define whole Pod.spec inside StaefulSet.spec, look to https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates for details"# nullable: truex-kubernetes-preserve-unknown-fields: truevolumeClaimTemplates:type: arraydescription: "allows define template for rendering `PVC` kubernetes resource, which would use inside `Pod` for mount clickhouse `data`, clickhouse `logs` or something else"# nullable: trueitems:type: object#required:#  - name#  - specproperties:name:type: stringdescription: |template name, could use to link insidetop-level `chi.spec.defaults.templates.dataVolumeClaimTemplate` or `chi.spec.defaults.templates.logVolumeClaimTemplate`,cluster-level `chi.spec.configuration.clusters.templates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.templates.logVolumeClaimTemplate`,shard-level `chi.spec.configuration.clusters.layout.shards.temlates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.layout.shards.temlates.logVolumeClaimTemplate`replica-level `chi.spec.configuration.clusters.layout.replicas.templates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.layout.replicas.templates.logVolumeClaimTemplate`provisioner: *TypePVCProvisionerreclaimPolicy: *TypePVCReclaimPolicymetadata:type: objectdescription: |allows to pass standard object's metadata from template to PVCMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:type: objectdescription: |allows define all aspects of `PVC` resourceMore info: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims# nullable: truex-kubernetes-preserve-unknown-fields: trueserviceTemplates:type: arraydescription: |allows define template for rendering `Service` which would get endpoint from Pods which scoped chi-wide, cluster-wide, shard-wide, replica-wide level# nullable: trueitems:type: object#required:#  - name#  - specproperties:name:type: stringdescription: |template name, could use to link insidechi-level `chi.spec.defaults.templates.serviceTemplate`cluster-level `chi.spec.configuration.clusters.templates.clusterServiceTemplate`shard-level `chi.spec.configuration.clusters.layout.shards.temlates.shardServiceTemplate`replica-level `chi.spec.configuration.clusters.layout.replicas.templates.replicaServiceTemplate` or `chi.spec.configuration.clusters.layout.shards.replicas.replicaServiceTemplate`generateName:type: stringdescription: "allows define format for generated `Service` name, look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatesservicetemplates for details about aviailable template variables"metadata:# TODO specify ObjectMetatype: objectdescription: |allows pass standard object's metadata from template to ServiceCould be use for define specificly for Cloud Provider metadata which impact to behavior of serviceMore info: https://kubernetes.io/docs/concepts/services-networking/service/# nullable: truex-kubernetes-preserve-unknown-fields: truespec:# TODO specify ServiceSpectype: objectdescription: |describe behavior of generated ServiceMore info: https://kubernetes.io/docs/concepts/services-networking/service/# nullable: truex-kubernetes-preserve-unknown-fields: trueuseTemplates:type: arraydescription: "list of `ClickHouseInstallationTemplate` (chit) resource names which will merge with current `Chi` manifest during render Kubernetes resources to create related ClickHouse clusters"# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "name of `ClickHouseInstallationTemplate` (chit) resource"namespace:type: stringdescription: "Kubernetes namespace where need search `chit` resource, depending on `watchNamespaces` settings in `clichouse-operator`"useType:type: stringdescription: "optional, current strategy is only merge, and current `chi` settings have more priority than merged template `chit`"enum:# List useTypeXXX constants from model- ""- "merge"
---
# Template Parameters:
#
# KIND=ClickHouseInstallationTemplate
# SINGULAR=clickhouseinstallationtemplate
# PLURAL=clickhouseinstallationtemplates
# SHORT=chit
# OPERATOR_VERSION=0.23.6
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: clickhouseinstallationtemplates.clickhouse.altinity.comlabels:clickhouse.altinity.com/chop: 0.23.6
spec:group: clickhouse.altinity.comscope: Namespacednames:kind: ClickHouseInstallationTemplatesingular: clickhouseinstallationtemplateplural: clickhouseinstallationtemplatesshortNames:- chitversions:- name: v1served: truestorage: trueadditionalPrinterColumns:- name: versiontype: stringdescription: Operator versionpriority: 1 # show in wide viewjsonPath: .status.chop-version- name: clusterstype: integerdescription: Clusters countjsonPath: .status.clusters- name: shardstype: integerdescription: Shards countpriority: 1 # show in wide viewjsonPath: .status.shards- name: hoststype: integerdescription: Hosts countjsonPath: .status.hosts- name: taskIDtype: stringdescription: TaskIDpriority: 1 # show in wide viewjsonPath: .status.taskID- name: statustype: stringdescription: CHI statusjsonPath: .status.status- name: hosts-unchangedtype: integerdescription: Unchanged hosts countpriority: 1 # show in wide viewjsonPath: .status.hostsUnchanged- name: hosts-updatedtype: integerdescription: Updated hosts countpriority: 1 # show in wide viewjsonPath: .status.hostsUpdated- name: hosts-addedtype: integerdescription: Added hosts countpriority: 1 # show in wide viewjsonPath: .status.hostsAdded- name: hosts-completedtype: integerdescription: Completed hosts countjsonPath: .status.hostsCompleted- name: hosts-deletedtype: integerdescription: Hosts deleted countpriority: 1 # show in wide viewjsonPath: .status.hostsDeleted- name: hosts-deletetype: integerdescription: Hosts to be deleted countpriority: 1 # show in wide viewjsonPath: .status.hostsDelete- name: endpointtype: stringdescription: Client access endpointpriority: 1 # show in wide viewjsonPath: .status.endpoint- name: agetype: datedescription: Age of the resource# Displayed in all prioritiesjsonPath: .metadata.creationTimestampsubresources:status: {}schema:openAPIV3Schema:description: "define a set of Kubernetes resources (StatefulSet, PVC, Service, ConfigMap) which describe behavior one or more ClickHouse clusters"type: objectrequired:- specproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectstatus:type: objectdescription: "Current ClickHouseInstallation manifest status, contains many fields like a normalized configuration, clickhouse-operator version, current action and all applied action list, current taskID and all applied taskIDs and other"properties:chop-version:type: stringdescription: "ClickHouse operator version"chop-commit:type: stringdescription: "ClickHouse operator git commit SHA"chop-date:type: stringdescription: "ClickHouse operator build date"chop-ip:type: stringdescription: "IP address of the operator's pod which managed this CHI"clusters:type: integerminimum: 0description: "Clusters count"shards:type: integerminimum: 0description: "Shards count"replicas:type: integerminimum: 0description: "Replicas count"hosts:type: integerminimum: 0description: "Hosts count"status:type: stringdescription: "Status"taskID:type: stringdescription: "Current task id"taskIDsStarted:type: arraydescription: "Started task ids"nullable: trueitems:type: stringtaskIDsCompleted:type: arraydescription: "Completed task ids"nullable: trueitems:type: stringaction:type: stringdescription: "Action"actions:type: arraydescription: "Actions"nullable: trueitems:type: stringerror:type: stringdescription: "Last error"errors:type: arraydescription: "Errors"nullable: trueitems:type: stringhostsUnchanged:type: integerminimum: 0description: "Unchanged Hosts count"hostsUpdated:type: integerminimum: 0description: "Updated Hosts count"hostsAdded:type: integerminimum: 0description: "Added Hosts count"hostsCompleted:type: integerminimum: 0description: "Completed Hosts count"hostsDeleted:type: integerminimum: 0description: "Deleted Hosts count"hostsDelete:type: integerminimum: 0description: "About to delete Hosts count"pods:type: arraydescription: "Pods"nullable: trueitems:type: stringpod-ips:type: arraydescription: "Pod IPs"nullable: trueitems:type: stringfqdns:type: arraydescription: "Pods FQDNs"nullable: trueitems:type: stringendpoint:type: stringdescription: "Endpoint"generation:type: integerminimum: 0description: "Generation"normalized:type: objectdescription: "Normalized CHI requested"x-kubernetes-preserve-unknown-fields: truenormalizedCompleted:type: objectdescription: "Normalized CHI completed"x-kubernetes-preserve-unknown-fields: truehostsWithTablesCreated:type: arraydescription: "List of hosts with tables created by the operator"nullable: trueitems:type: stringusedTemplates:type: arraydescription: "List of templates used to build this CHI"nullable: truex-kubernetes-preserve-unknown-fields: trueitems:type: objectx-kubernetes-preserve-unknown-fields: truespec:type: object# x-kubernetes-preserve-unknown-fields: truedescription: |Specification of the desired behavior of one or more ClickHouse clustersMore info: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.mdproperties:taskID:type: stringdescription: |Allows to define custom taskID for CHI update and watch status of this update execution.Displayed in all .status.taskID* fields.By default (if not filled) every update of CHI manifest will generate random taskIDstop: &TypeStringBooltype: stringdescription: |Allows to stop all ClickHouse clusters defined in a CHI.Works as the following:- When `stop` is `1` operator sets `Replicas: 0` in each StatefulSet. Thie leads to having all `Pods` and `Service` deleted. All PVCs are kept intact.- When `stop` is `0` operator sets `Replicas: 1` and `Pod`s and `Service`s will created again and all retained PVCs will be attached to `Pod`s.enum:# List StringBoolXXX constants from model- ""- "0"- "1"- "False"- "false"- "True"- "true"- "No"- "no"- "Yes"- "yes"- "Off"- "off"- "On"- "on"- "Disable"- "disable"- "Enable"- "enable"- "Disabled"- "disabled"- "Enabled"- "enabled"restart:type: stringdescription: |In case 'RollingUpdate' specified, the operator will always restart ClickHouse pods during reconcile.This options is used in rare cases when force restart is required and is typically removed after the use in order to avoid unneeded restarts.enum:- ""- "RollingUpdate"troubleshoot:<<: *TypeStringBooldescription: |Allows to troubleshoot Pods during CrashLoopBack state.This may happen when wrong configuration applied, in this case `clickhouse-server` wouldn't start.Command within ClickHouse container is modified with `sleep` in order to avoid quick restartsand give time to troubleshoot via CLI.Liveness and Readiness probes are disabled as well.namespaceDomainPattern:type: stringdescription: |Custom domain pattern which will be used for DNS names of `Service` or `Pod`.Typical use scenario - custom cluster domain in Kubernetes clusterExample: %s.svc.my.testtemplating:type: object# nullable: truedescription: |Optional, applicable inside ClickHouseInstallationTemplate only.Defines current ClickHouseInstallationTemplate application options to target ClickHouseInstallation(s)."properties:policy:type: stringdescription: |When defined as `auto` inside ClickhouseInstallationTemplate, this ClickhouseInstallationTemplatewill be auto-added into ClickHouseInstallation, selectable by `chiSelector`.Default value is `manual`, meaning ClickHouseInstallation should request this ClickhouseInstallationTemplate explicitly.enum:- ""- "auto"- "manual"chiSelector:type: objectdescription: "Optional, defines selector for ClickHouseInstallation(s) to be templated with ClickhouseInstallationTemplate"# nullable: truex-kubernetes-preserve-unknown-fields: truereconciling:type: objectdescription: "Optional, allows tuning reconciling cycle for ClickhouseInstallation from clickhouse-operator side"# nullable: trueproperties:policy:type: stringdescription: |DISCUSSED TO BE DEPRECATEDSyntax sugarOverrides all three 'reconcile.host.wait.{exclude, queries, include}' values from the operator's configPossible values:- wait - should wait to exclude host, complete queries and include host back into the cluster- nowait - should NOT wait to exclude host, complete queries and include host back into the clusterenum:- ""- "wait"- "nowait"configMapPropagationTimeout:type: integerdescription: |Timeout in seconds for `clickhouse-operator` to wait for modified `ConfigMap` to propagate into the `Pod`More details: https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automaticallyminimum: 0maximum: 3600cleanup:type: objectdescription: "Optional, defines behavior for cleanup Kubernetes resources during reconcile cycle"# nullable: trueproperties:unknownObjects:type: objectdescription: |Describes what clickhouse-operator should do with found Kubernetes resources which should be managed by clickhouse-operator,but do not have `ownerReference` to any currently managed `ClickHouseInstallation` resource.Default behavior is `Delete`"# nullable: trueproperties:statefulSet: &TypeObjectsCleanuptype: stringdescription: "Behavior policy for unknown StatefulSet, `Delete` by default"enum:# List ObjectsCleanupXXX constants from model- ""- "Retain"- "Delete"pvc:type: string<<: *TypeObjectsCleanupdescription: "Behavior policy for unknown PVC, `Delete` by default"configMap:<<: *TypeObjectsCleanupdescription: "Behavior policy for unknown ConfigMap, `Delete` by default"service:<<: *TypeObjectsCleanupdescription: "Behavior policy for unknown Service, `Delete` by default"reconcileFailedObjects:type: objectdescription: |Describes what clickhouse-operator should do with Kubernetes resources which are failed during reconcile.Default behavior is `Retain`"# nullable: trueproperties:statefulSet:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed StatefulSet, `Retain` by default"pvc:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed PVC, `Retain` by default"configMap:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed ConfigMap, `Retain` by default"service:<<: *TypeObjectsCleanupdescription: "Behavior policy for failed Service, `Retain` by default"defaults:type: objectdescription: |define default behavior for whole ClickHouseInstallation, some behavior can be re-define on cluster, shard and replica levelMore info: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specdefaults# nullable: trueproperties:replicasUseFQDN:<<: *TypeStringBooldescription: |define should replicas be specified by FQDN in `<host></host>`.In case of "no" will use short hostname and clickhouse-server will use kubernetes default suffixes for DNS lookup"yes" by defaultdistributedDDL:type: objectdescription: |allows change `<yandex><distributed_ddl></distributed_ddl></yandex>` settingsMore info: https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/#server-settings-distributed_ddl# nullable: trueproperties:profile:type: stringdescription: "Settings from this profile will be used to execute DDL queries"storageManagement:type: objectdescription: default storage management optionsproperties:provisioner: &TypePVCProvisionertype: stringdescription: "defines `PVC` provisioner - be it StatefulSet or the Operator"enum:- ""- "StatefulSet"- "Operator"reclaimPolicy: &TypePVCReclaimPolicytype: stringdescription: |defines behavior of `PVC` deletion.`Delete` by default, if `Retain` specified then `PVC` will be kept when deleting StatefulSetenum:- ""- "Retain"- "Delete"templates: &TypeTemplateNamestype: objectdescription: "optional, configuration of the templates names which will use for generate Kubernetes resources according to one or more ClickHouse clusters described in current ClickHouseInstallation (chi) resource"# nullable: trueproperties:hostTemplate:type: stringdescription: "optional, template name from chi.spec.templates.hostTemplates, which will apply to configure every `clickhouse-server` instance during render ConfigMap resources which will mount into `Pod`"podTemplate:type: stringdescription: "optional, template name from chi.spec.templates.podTemplates, allows customization each `Pod` resource during render and reconcile each StatefulSet.spec resource described in `chi.spec.configuration.clusters`"dataVolumeClaimTemplate:type: stringdescription: "optional, template name from chi.spec.templates.volumeClaimTemplates, allows customization each `PVC` which will mount for clickhouse data directory in each `Pod` during render and reconcile every StatefulSet.spec resource described in `chi.spec.configuration.clusters`"logVolumeClaimTemplate:type: stringdescription: "optional, template name from chi.spec.templates.volumeClaimTemplates, allows customization each `PVC` which will mount for clickhouse log directory in each `Pod` during render and reconcile every StatefulSet.spec resource described in `chi.spec.configuration.clusters`"serviceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for one `Service` resource which will created by `clickhouse-operator` which cover all clusters in whole `chi` resource"clusterServiceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for each `Service` resource which will created by `clickhouse-operator` which cover each clickhouse cluster described in `chi.spec.configuration.clusters`"shardServiceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for each `Service` resource which will created by `clickhouse-operator` which cover each shard inside clickhouse cluster described in `chi.spec.configuration.clusters`"replicaServiceTemplate:type: stringdescription: "optional, template name from chi.spec.templates.serviceTemplates, allows customization for each `Service` resource which will created by `clickhouse-operator` which cover each replica inside each shard inside each clickhouse cluster described in `chi.spec.configuration.clusters`"volumeClaimTemplate:type: stringdescription: "DEPRECATED! VolumeClaimTemplate is deprecated in favor of DataVolumeClaimTemplate and LogVolumeClaimTemplate"configuration:type: objectdescription: "allows configure multiple aspects and behavior for `clickhouse-server` instance and also allows describe multiple `clickhouse-server` clusters inside one `chi` resource"# nullable: trueproperties:zookeeper: &TypeZookeeperConfigtype: objectdescription: |allows configure <yandex><zookeeper>..</zookeeper></yandex> section in each `Pod` during generate `ConfigMap` which will mounted in `/etc/clickhouse-server/config.d/``clickhouse-operator` itself doesn't manage Zookeeper, please install Zookeeper separatelly look examples on https://github.com/Altinity/clickhouse-operator/tree/master/deploy/zookeeper/currently, zookeeper (or clickhouse-keeper replacement) used for *ReplicatedMergeTree table engines and for `distributed_ddl`More details: https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/#server-settings_zookeeper# nullable: trueproperties:nodes:type: arraydescription: "describe every available zookeeper cluster node for interaction"# nullable: trueitems:type: object#required:#  - hostproperties:host:type: stringdescription: "dns name or ip address for Zookeeper node"port:type: integerdescription: "TCP port which used to connect to Zookeeper node"minimum: 0maximum: 65535secure:<<: *TypeStringBooldescription: "if a secure connection to Zookeeper is required"session_timeout_ms:type: integerdescription: "session timeout during connect to Zookeeper"operation_timeout_ms:type: integerdescription: "one operation timeout during Zookeeper transactions"root:type: stringdescription: "optional root znode path inside zookeeper to store ClickHouse related data (replication queue or distributed DDL)"identity:type: stringdescription: "optional access credentials string with `user:password` format used when use digest authorization in Zookeeper"users:type: objectdescription: |allows configure <yandex><users>..</users></yandex> section in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/users.d/`you can configure password hashed, authorization restrictions, database level security row filters etc.More details: https://clickhouse.tech/docs/en/operations/settings/settings-users/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationusers# nullable: truex-kubernetes-preserve-unknown-fields: trueprofiles:type: objectdescription: |allows configure <yandex><profiles>..</profiles></yandex> section in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/users.d/`you can configure any aspect of settings profileMore details: https://clickhouse.tech/docs/en/operations/settings/settings-profiles/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationprofiles# nullable: truex-kubernetes-preserve-unknown-fields: truequotas:type: objectdescription: |allows configure <yandex><quotas>..</quotas></yandex> section in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/users.d/`you can configure any aspect of resource quotasMore details: https://clickhouse.tech/docs/en/operations/quotas/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationquotas# nullable: truex-kubernetes-preserve-unknown-fields: truesettings: &TypeSettingstype: objectdescription: |allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/`More details: https://clickhouse.tech/docs/en/operations/settings/settings/Your yaml code will convert to XML, see examples https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#specconfigurationsettings# nullable: truex-kubernetes-preserve-unknown-fields: truefiles: &TypeFilestype: objectdescription: |allows define content of any setting file inside each `Pod` during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`every key in this object is the file nameevery value in this object is the file contentyou can use `!!binary |` and base64 for binary files, see details here https://yaml.org/type/binary.htmleach key could contains prefix like USERS, COMMON, HOST or config.d, users.d, cond.d, wrong prefixes will ignored, subfolders also will ignoredMore details: https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/05-settings-05-files-nested.yaml# nullable: truex-kubernetes-preserve-unknown-fields: trueclusters:type: arraydescription: |describes ClickHouse clusters layout and allows change settings on cluster-level, shard-level and replica-levelevery cluster is a set of StatefulSet, one StatefulSet contains only one Pod with `clickhouse-server`all Pods will rendered in <remote_server> part of ClickHouse configs, mounted from ConfigMap as `/etc/clickhouse-server/config.d/chop-generated-remote_servers.xml`Clusters will use for Distributed table engine, more details: https://clickhouse.tech/docs/en/engines/table-engines/special/distributed/If `cluster` contains zookeeper settings (could be inherited from top `chi` level), when you can create *ReplicatedMergeTree tables# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "cluster name, used to identify set of ClickHouse servers and wide used during generate names of related Kubernetes resources"minLength: 1# See namePartClusterMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"zookeeper:<<: *TypeZookeeperConfigdescription: |optional, allows configure <yandex><zookeeper>..</zookeeper></yandex> section in each `Pod` only in current ClickHouse cluster, during generate `ConfigMap` which will mounted in `/etc/clickhouse-server/config.d/`override top-level `chi.spec.configuration.zookeeper` settingssettings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` only in one cluster during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/`override top-level `chi.spec.configuration.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` on current cluster during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files`templates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected clusteroverride top-level `chi.spec.configuration.templates`schemaPolicy:type: objectdescription: |describes how schema is propagated within replicas and shardsproperties:replica:type: stringdescription: "how schema is propagated within a replica"enum:# List SchemaPolicyReplicaXXX constants from model- ""- "None"- "All"shard:type: stringdescription: "how schema is propagated between shards"enum:# List SchemaPolicyShardXXX constants from model- ""- "None"- "All"- "DistributedTablesOnly"insecure:<<: *TypeStringBooldescription: optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: optional, open secure ports for clustersecret:type: objectdescription: "optional, shared secret value to secure cluster communications"properties:auto:<<: *TypeStringBooldescription: "Auto-generate shared secret value to secure cluster communications"value:description: "Cluster shared secret value in plain text"type: stringvalueFrom:description: "Cluster shared secret source"type: objectproperties:secretKeyRef:description: |Selects a key of a secret in the clickhouse installation namespace.Should not be used if value is not empty.type: objectproperties:name:description: |Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#namestype: stringkey:description: The key of the secret to select from. Must be a valid secret key.type: stringoptional:description: Specify whether the Secret or its key must be definedtype: booleanrequired:- name- keylayout:type: objectdescription: |describe current cluster layout, how much shards in cluster, how much replica in shardallows override settings on each shard and replica separatelly# nullable: trueproperties:type:type: stringdescription: "DEPRECATED - to be removed soon"shardsCount:type: integerdescription: "how much shards for current ClickHouse cluster will run in Kubernetes, each shard contains shared-nothing part of data and contains set of replicas, cluster contains 1 shard by default"replicasCount:type: integerdescription: "how much replicas in each shards for current ClickHouse cluster will run in Kubernetes, each replica is a separate `StatefulSet` which contains only one `Pod` with `clickhouse-server` instance, every shard contains 1 replica by default"shards:type: arraydescription: "optional, allows override top-level `chi.spec.configuration`, cluster-level `chi.spec.configuration.clusters` settings for each shard separately, use it only if you fully understand what you do"# nullable: trueitems:type: objectproperties:name:type: stringdescription: "optional, by default shard name is generated, but you can override it and setup custom name"minLength: 1# See namePartShardMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"definitionType:type: stringdescription: "DEPRECATED - to be removed soon"weight:type: integerdescription: |optional, 1 by default, allows setup shard <weight> setting which will use during insert into tables with `Distributed` engine,will apply in <remote_servers> inside ConfigMap which will mount in /etc/clickhouse-server/config.d/chop-generated-remote_servers.xmlMore details: https://clickhouse.tech/docs/en/engines/table-engines/special/distributed/internalReplication:<<: *TypeStringBooldescription: |optional, `true` by default when `chi.spec.configuration.clusters[].layout.ReplicaCount` > 1 and 0 otherwiseallows setup <internal_replication> setting which will use during insert into tables with `Distributed` engine for insert only in one live replica and other replicas will download inserted data during replication,will apply in <remote_servers> inside ConfigMap which will mount in /etc/clickhouse-server/config.d/chop-generated-remote_servers.xmlMore details: https://clickhouse.tech/docs/en/engines/table-engines/special/distributed/settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` only in one shard during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/`override top-level `chi.spec.configuration.settings` and cluster-level `chi.spec.configuration.clusters.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` only in one shard during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files` and cluster-level `chi.spec.configuration.clusters.files`templates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected shardoverride top-level `chi.spec.configuration.templates` and cluster-level `chi.spec.configuration.clusters.templates`replicasCount:type: integerdescription: |optional, how much replicas in selected shard for selected ClickHouse cluster will run in Kubernetes, each replica is a separate `StatefulSet` which contains only one `Pod` with `clickhouse-server` instance,shard contains 1 replica by defaultoverride cluster-level `chi.spec.configuration.clusters.layout.replicasCount`minimum: 1replicas:type: arraydescription: |optional, allows override behavior for selected replicas from cluster-level `chi.spec.configuration.clusters` and shard-level `chi.spec.configuration.clusters.layout.shards`# nullable: trueitems:# Hosttype: objectproperties:name:type: stringdescription: "optional, by default replica name is generated, but you can override it and setup custom name"minLength: 1# See namePartReplicaMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"insecure:<<: *TypeStringBooldescription: |optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: |optional, open secure portstcpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `tcp` for selected replica, override `chi.spec.templates.hostTemplates.spec.tcpPort`allows connect to `clickhouse-server` via TCP Native protocol via kubernetes `Service`minimum: 1maximum: 65535tlsPort:type: integerminimum: 1maximum: 65535httpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `http` for selected replica, override `chi.spec.templates.hostTemplates.spec.httpPort`allows connect to `clickhouse-server` via HTTP protocol via kubernetes `Service`minimum: 1maximum: 65535httpsPort:type: integerminimum: 1maximum: 65535interserverHTTPPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `interserver` for selected replica, override `chi.spec.templates.hostTemplates.spec.interserverHTTPPort`allows connect between replicas inside same shard during fetch replicated data parts HTTP protocolminimum: 1maximum: 65535settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`override top-level `chi.spec.configuration.settings`, cluster-level `chi.spec.configuration.clusters.settings` and shard-level `chi.spec.configuration.clusters.layout.shards.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files`, cluster-level `chi.spec.configuration.clusters.files` and shard-level `chi.spec.configuration.clusters.layout.shards.files`templates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected replicaoverride top-level `chi.spec.configuration.templates`, cluster-level `chi.spec.configuration.clusters.templates` and shard-level `chi.spec.configuration.clusters.layout.shards.templates`replicas:type: arraydescription: "optional, allows override top-level `chi.spec.configuration` and cluster-level `chi.spec.configuration.clusters` configuration for each replica and each shard relates to selected replica, use it only if you fully understand what you do"# nullable: trueitems:type: objectproperties:name:type: stringdescription: "optional, by default replica name is generated, but you can override it and setup custom name"minLength: 1# See namePartShardMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`override top-level `chi.spec.configuration.settings`, cluster-level `chi.spec.configuration.clusters.settings` and will ignore if shard-level `chi.spec.configuration.clusters.layout.shards` presentMore details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` only in one replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files` and cluster-level `chi.spec.configuration.clusters.files`, will ignore if `chi.spec.configuration.clusters.layout.shards` presentstemplates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected replicaoverride top-level `chi.spec.configuration.templates`, cluster-level `chi.spec.configuration.clusters.templates`shardsCount:type: integerdescription: "optional, count of shards related to current replica, you can override each shard behavior on low-level `chi.spec.configuration.clusters.layout.replicas.shards`"minimum: 1shards:type: arraydescription: "optional, list of shards related to current replica, will ignore if `chi.spec.configuration.clusters.layout.shards` presents"# nullable: trueitems:# Hosttype: objectproperties:name:type: stringdescription: "optional, by default shard name is generated, but you can override it and setup custom name"minLength: 1# See namePartReplicaMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"insecure:<<: *TypeStringBooldescription: |optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: |optional, open secure portstcpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `tcp` for selected shard, override `chi.spec.templates.hostTemplates.spec.tcpPort`allows connect to `clickhouse-server` via TCP Native protocol via kubernetes `Service`minimum: 1maximum: 65535tlsPort:type: integerminimum: 1maximum: 65535httpPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `http` for selected shard, override `chi.spec.templates.hostTemplates.spec.httpPort`allows connect to `clickhouse-server` via HTTP protocol via kubernetes `Service`minimum: 1maximum: 65535httpsPort:type: integerminimum: 1maximum: 65535interserverHTTPPort:type: integerdescription: |optional, setup `Pod.spec.containers.ports` with name `interserver` for selected shard, override `chi.spec.templates.hostTemplates.spec.interserverHTTPPort`allows connect between replicas inside same shard during fetch replicated data parts HTTP protocolminimum: 1maximum: 65535settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in `Pod` only in one shard related to current replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`override top-level `chi.spec.configuration.settings`, cluster-level `chi.spec.configuration.clusters.settings` and replica-level `chi.spec.configuration.clusters.layout.replicas.settings`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` only in one shard related to current replica during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`override top-level `chi.spec.configuration.files` and cluster-level `chi.spec.configuration.clusters.files`, will ignore if `chi.spec.configuration.clusters.layout.shards` presentstemplates:<<: *TypeTemplateNamesdescription: |optional, configuration of the templates names which will use for generate Kubernetes resources according to selected replicaoverride top-level `chi.spec.configuration.templates`, cluster-level `chi.spec.configuration.clusters.templates`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates`templates:type: objectdescription: "allows define templates which will use for render Kubernetes resources like StatefulSet, ConfigMap, Service, PVC, by default, clickhouse-operator have own templates, but you can override it"# nullable: trueproperties:hostTemplates:type: arraydescription: "hostTemplate will use during apply to generate `clickhose-server` config files"# nullable: trueitems:type: object#required:#  - nameproperties:name:description: "template name, could use to link inside top-level `chi.spec.defaults.templates.hostTemplate`, cluster-level `chi.spec.configuration.clusters.templates.hostTemplate`, shard-level `chi.spec.configuration.clusters.layout.shards.temlates.hostTemplate`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates.hostTemplate`"type: stringportDistribution:type: arraydescription: "define how will distribute numeric values of named ports in `Pod.spec.containers.ports` and clickhouse-server configs"# nullable: trueitems:type: object#required:#  - typeproperties:type:type: stringdescription: "type of distribution, when `Unspecified` (default value) then all listen ports on clickhouse-server configuration in all Pods will have the same value, when `ClusterScopeIndex` then ports will increment to offset from base value depends on shard and replica index inside cluster with combination of `chi.spec.templates.podTemlates.spec.HostNetwork` it allows setup ClickHouse cluster inside Kubernetes and provide access via external network bypass Kubernetes internal network"enum:# List PortDistributionXXX constants- ""- "Unspecified"- "ClusterScopeIndex"spec:# Hosttype: objectproperties:name:type: stringdescription: "by default, hostname will generate, but this allows define custom name for each `clickhuse-server`"minLength: 1# See namePartReplicaMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"insecure:<<: *TypeStringBooldescription: |optional, open insecure ports for cluster, defaults to "yes"secure:<<: *TypeStringBooldescription: |optional, open secure portstcpPort:type: integerdescription: |optional, setup `tcp_port` inside `clickhouse-server` settings for each Pod where current template will applyif specified, should have equal value with `chi.spec.templates.podTemplates.spec.containers.ports[name=tcp]`More info: https://clickhouse.tech/docs/en/interfaces/tcp/minimum: 1maximum: 65535tlsPort:type: integerminimum: 1maximum: 65535httpPort:type: integerdescription: |optional, setup `http_port` inside `clickhouse-server` settings for each Pod where current template will applyif specified, should have equal value with `chi.spec.templates.podTemplates.spec.containers.ports[name=http]`More info: https://clickhouse.tech/docs/en/interfaces/http/minimum: 1maximum: 65535httpsPort:type: integerminimum: 1maximum: 65535interserverHTTPPort:type: integerdescription: |optional, setup `interserver_http_port` inside `clickhouse-server` settings for each Pod where current template will applyif specified, should have equal value with `chi.spec.templates.podTemplates.spec.containers.ports[name=interserver]`More info: https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/#interserver-http-portminimum: 1maximum: 65535settings:<<: *TypeSettingsdescription: |optional, allows configure `clickhouse-server` settings inside <yandex>...</yandex> tag in each `Pod` where this template will apply during generate `ConfigMap` which will mount in `/etc/clickhouse-server/conf.d/`More details: https://clickhouse.tech/docs/en/operations/settings/settings/files:<<: *TypeFilesdescription: |optional, allows define content of any setting file inside each `Pod` where this template will apply during generate `ConfigMap` which will mount in `/etc/clickhouse-server/config.d/` or `/etc/clickhouse-server/conf.d/` or `/etc/clickhouse-server/users.d/`templates:<<: *TypeTemplateNamesdescription: "be careful, this part of CRD allows override template inside template, don't use it if you don't understand what you do"podTemplates:type: arraydescription: |podTemplate will use during render `Pod` inside `StatefulSet.spec` and allows define rendered `Pod.spec`, pod scheduling distribution and pod zoneMore information: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatespodtemplates# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "template name, could use to link inside top-level `chi.spec.defaults.templates.podTemplate`, cluster-level `chi.spec.configuration.clusters.templates.podTemplate`, shard-level `chi.spec.configuration.clusters.layout.shards.temlates.podTemplate`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates.podTemplate`"generateName:type: stringdescription: "allows define format for generated `Pod` name, look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatesservicetemplates for details about aviailable template variables"zone:type: objectdescription: "allows define custom zone name and will separate ClickHouse `Pods` between nodes, shortcut for `chi.spec.templates.podTemplates.spec.affinity.podAntiAffinity`"#required:#  - valuesproperties:key:type: stringdescription: "optional, if defined, allows select kubernetes nodes by label with `name` equal `key`"values:type: arraydescription: "optional, if defined, allows select kubernetes nodes by label with `value` in `values`"# nullable: trueitems:type: stringdistribution:type: stringdescription: "DEPRECATED, shortcut for `chi.spec.templates.podTemplates.spec.affinity.podAntiAffinity`"enum:- ""- "Unspecified"- "OnePerHost"podDistribution:type: arraydescription: "define ClickHouse Pod distribution policy between Kubernetes Nodes inside Shard, Replica, Namespace, CHI, another ClickHouse cluster"# nullable: trueitems:type: object#required:#  - typeproperties:type:type: stringdescription: "you can define multiple affinity policy types"enum:# List PodDistributionXXX constants- ""- "Unspecified"- "ClickHouseAntiAffinity"- "ShardAntiAffinity"- "ReplicaAntiAffinity"- "AnotherNamespaceAntiAffinity"- "AnotherClickHouseInstallationAntiAffinity"- "AnotherClusterAntiAffinity"- "MaxNumberPerNode"- "NamespaceAffinity"- "ClickHouseInstallationAffinity"- "ClusterAffinity"- "ShardAffinity"- "ReplicaAffinity"- "PreviousTailAffinity"- "CircularReplication"scope:type: stringdescription: "scope for apply each podDistribution"enum:# list PodDistributionScopeXXX constants- ""- "Unspecified"- "Shard"- "Replica"- "Cluster"- "ClickHouseInstallation"- "Namespace"number:type: integerdescription: "define, how much ClickHouse Pods could be inside selected scope with selected distribution type"minimum: 0maximum: 65535topologyKey:type: stringdescription: "use for inter-pod affinity look to `pod.spec.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution.podAffinityTerm.topologyKey`, More info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity"metadata:type: objectdescription: |allows pass standard object's metadata from template to PodMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:# TODO specify PodSpectype: objectdescription: "allows define whole Pod.spec inside StaefulSet.spec, look to https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates for details"# nullable: truex-kubernetes-preserve-unknown-fields: truevolumeClaimTemplates:type: arraydescription: "allows define template for rendering `PVC` kubernetes resource, which would use inside `Pod` for mount clickhouse `data`, clickhouse `logs` or something else"# nullable: trueitems:type: object#required:#  - name#  - specproperties:name:type: stringdescription: |template name, could use to link insidetop-level `chi.spec.defaults.templates.dataVolumeClaimTemplate` or `chi.spec.defaults.templates.logVolumeClaimTemplate`,cluster-level `chi.spec.configuration.clusters.templates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.templates.logVolumeClaimTemplate`,shard-level `chi.spec.configuration.clusters.layout.shards.temlates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.layout.shards.temlates.logVolumeClaimTemplate`replica-level `chi.spec.configuration.clusters.layout.replicas.templates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.layout.replicas.templates.logVolumeClaimTemplate`provisioner: *TypePVCProvisionerreclaimPolicy: *TypePVCReclaimPolicymetadata:type: objectdescription: |allows to pass standard object's metadata from template to PVCMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:type: objectdescription: |allows define all aspects of `PVC` resourceMore info: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims# nullable: truex-kubernetes-preserve-unknown-fields: trueserviceTemplates:type: arraydescription: |allows define template for rendering `Service` which would get endpoint from Pods which scoped chi-wide, cluster-wide, shard-wide, replica-wide level# nullable: trueitems:type: object#required:#  - name#  - specproperties:name:type: stringdescription: |template name, could use to link insidechi-level `chi.spec.defaults.templates.serviceTemplate`cluster-level `chi.spec.configuration.clusters.templates.clusterServiceTemplate`shard-level `chi.spec.configuration.clusters.layout.shards.temlates.shardServiceTemplate`replica-level `chi.spec.configuration.clusters.layout.replicas.templates.replicaServiceTemplate` or `chi.spec.configuration.clusters.layout.shards.replicas.replicaServiceTemplate`generateName:type: stringdescription: "allows define format for generated `Service` name, look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatesservicetemplates for details about aviailable template variables"metadata:# TODO specify ObjectMetatype: objectdescription: |allows pass standard object's metadata from template to ServiceCould be use for define specificly for Cloud Provider metadata which impact to behavior of serviceMore info: https://kubernetes.io/docs/concepts/services-networking/service/# nullable: truex-kubernetes-preserve-unknown-fields: truespec:# TODO specify ServiceSpectype: objectdescription: |describe behavior of generated ServiceMore info: https://kubernetes.io/docs/concepts/services-networking/service/# nullable: truex-kubernetes-preserve-unknown-fields: trueuseTemplates:type: arraydescription: "list of `ClickHouseInstallationTemplate` (chit) resource names which will merge with current `Chi` manifest during render Kubernetes resources to create related ClickHouse clusters"# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "name of `ClickHouseInstallationTemplate` (chit) resource"namespace:type: stringdescription: "Kubernetes namespace where need search `chit` resource, depending on `watchNamespaces` settings in `clichouse-operator`"useType:type: stringdescription: "optional, current strategy is only merge, and current `chi` settings have more priority than merged template `chit`"enum:# List useTypeXXX constants from model- ""- "merge"
---
# Template Parameters:
#
# NONE
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: clickhouseoperatorconfigurations.clickhouse.altinity.comlabels:clickhouse.altinity.com/chop: 0.23.6
spec:group: clickhouse.altinity.comscope: Namespacednames:kind: ClickHouseOperatorConfigurationsingular: clickhouseoperatorconfigurationplural: clickhouseoperatorconfigurationsshortNames:- chopconfversions:- name: v1served: truestorage: trueadditionalPrinterColumns:- name: namespacestype: stringdescription: Watch namespacesjsonPath: .status- name: agetype: datedescription: Age of the resource# Displayed in all prioritiesjsonPath: .metadata.creationTimestampschema:openAPIV3Schema:type: objectdescription: "allows customize `clickhouse-operator` settings, need restart clickhouse-operator pod after adding, more details https://github.com/Altinity/clickhouse-operator/blob/master/docs/operator_configuration.md"x-kubernetes-preserve-unknown-fields: trueproperties:status:type: objectx-kubernetes-preserve-unknown-fields: truespec:type: objectdescription: |Allows to define settings of the clickhouse-operator.More info: https://github.com/Altinity/clickhouse-operator/blob/master/config/config.yamlCheck into etc-clickhouse-operator* ConfigMaps if you need more controlx-kubernetes-preserve-unknown-fields: trueproperties:watch:type: objectdescription: "Parameters for watch kubernetes resources which used by clickhouse-operator deployment"properties:namespaces:type: arraydescription: "List of namespaces where clickhouse-operator watches for events."items:type: stringclickhouse:type: objectdescription: "Clickhouse related parameters used by clickhouse-operator"properties:configuration:type: objectproperties:file:type: objectproperties:path:type: objectdescription: |Each 'path' can be either absolute or relative.In case path is absolute - it is used as is.In case path is relative - it is relative to the folder where configuration file you are reading right now is located.properties:common:type: stringdescription: |Path to the folder where ClickHouse configuration files common for all instances within a CHI are located.Default value - config.dhost:type: stringdescription: |Path to the folder where ClickHouse configuration files unique for each instance (host) within a CHI are located.Default value - conf.duser:type: stringdescription: |Path to the folder where ClickHouse configuration files with users settings are located.Files are common for all instances within a CHI.Default value - users.duser:type: objectdescription: "Default parameters for any user which will create"properties:default:type: objectproperties:profile:type: stringdescription: "ClickHouse server configuration `<profile>...</profile>` for any <user>"quota:type: stringdescription: "ClickHouse server configuration `<quota>...</quota>` for any <user>"networksIP:type: arraydescription: "ClickHouse server configuration `<networks><ip>...</ip></networks>` for any <user>"items:type: stringpassword:type: stringdescription: "ClickHouse server configuration `<password>...</password>` for any <user>"network:type: objectdescription: "Default network parameters for any user which will create"properties:hostRegexpTemplate:type: stringdescription: "ClickHouse server configuration `<host_regexp>...</host_regexp>` for any <user>"configurationRestartPolicy:type: objectdescription: "Configuration restart policy describes what configuration changes require ClickHouse restart"properties:rules:type: arraydescription: "Array of set of rules per specified ClickHouse versions"items:type: objectproperties:version:type: stringdescription: "ClickHouse version expression"rules:type: arraydescription: "Set of configuration rules for specified ClickHouse version"items:type: objectdescription: "setting: value pairs for configuration restart policy"access:type: objectdescription: "parameters which use for connect to clickhouse from clickhouse-operator deployment"properties:scheme:type: stringdescription: "The scheme to user for connecting to ClickHouse. Possible values: http, https, auto"username:type: stringdescription: "ClickHouse username to be used by operator to connect to ClickHouse instances, deprecated, use chCredentialsSecretName"password:type: stringdescription: "ClickHouse password to be used by operator to connect to ClickHouse instances, deprecated, use chCredentialsSecretName"rootCA:type: stringdescription: "Root certificate authority that clients use when verifying server certificates. Used for https connection to ClickHouse"secret:type: objectproperties:namespace:type: stringdescription: "Location of k8s Secret with username and password to be used by operator to connect to ClickHouse instances"name:type: stringdescription: "Name of k8s Secret with username and password to be used by operator to connect to ClickHouse instances"port:type: integerminimum: 1maximum: 65535description: "Port to be used by operator to connect to ClickHouse instances"timeouts:type: objectdescription: "Timeouts used to limit connection and queries from the operator to ClickHouse instances, In seconds"properties:connect:type: integerminimum: 1maximum: 10description: "Timout to setup connection from the operator to ClickHouse instances. In seconds."query:type: integerminimum: 1maximum: 600description: "Timout to perform SQL query from the operator to ClickHouse instances. In seconds."metrics:type: objectdescription: "parameters which use for connect to fetch metrics from clickhouse by clickhouse-operator"properties:timeouts:type: objectdescription: |Timeouts used to limit connection and queries from the metrics exporter to ClickHouse instancesSpecified in seconds.properties:collect:type: integerminimum: 1maximum: 600description: |Timeout used to limit metrics collection request. In seconds.Upon reaching this timeout metrics collection is aborted and no more metrics are collected in this cycle.All collected metrics are returned.template:type: objectdescription: "Parameters which are used if you want to generate ClickHouseInstallationTemplate custom resources from files which are stored inside clickhouse-operator deployment"properties:chi:type: objectproperties:policy:type: stringdescription: |CHI template updates handling policyPossible policy values:- ReadOnStart. Accept CHIT updates on the operators start only.- ApplyOnNextReconcile. Accept CHIT updates at all time. Apply news CHITs on next regular reconcile of the CHIenum:- ""- "ReadOnStart"- "ApplyOnNextReconcile"path:type: stringdescription: "Path to folder where ClickHouseInstallationTemplate .yaml manifests are located."reconcile:type: objectdescription: "allow tuning reconciling process"properties:runtime:type: objectdescription: "runtime parameters for clickhouse-operator process which are used during reconcile cycle"properties:reconcileCHIsThreadsNumber:type: integerminimum: 1maximum: 65535description: "How many goroutines will be used to reconcile CHIs in parallel, 10 by default"reconcileShardsThreadsNumber:type: integerminimum: 1maximum: 65535description: "How many goroutines will be used to reconcile shards of a cluster in parallel, 1 by default"reconcileShardsMaxConcurrencyPercent:type: integerminimum: 0maximum: 100description: "The maximum percentage of cluster shards that may be reconciled in parallel, 50 percent by default."statefulSet:type: objectdescription: "Allow change default behavior for reconciling StatefulSet which generated by clickhouse-operator"properties:create:type: objectdescription: "Behavior during create StatefulSet"properties:onFailure:type: stringdescription: |What to do in case created StatefulSet is not in Ready after `statefulSetUpdateTimeout` secondsPossible options:1. abort - do nothing, just break the process and wait for admin.2. delete - delete newly created problematic StatefulSet.3. ignore (default) - ignore error, pretend nothing happened and move on to the next StatefulSet.update:type: objectdescription: "Behavior during update StatefulSet"properties:timeout:type: integerdescription: "How many seconds to wait for created/updated StatefulSet to be Ready"pollInterval:type: integerdescription: "How many seconds to wait between checks for created/updated StatefulSet status"onFailure:type: stringdescription: |What to do in case updated StatefulSet is not in Ready after `statefulSetUpdateTimeout` secondsPossible options:1. abort - do nothing, just break the process and wait for admin.2. rollback (default) - delete Pod and rollback StatefulSet to previous Generation. Pod would be recreated by StatefulSet based on rollback-ed configuration.3. ignore - ignore error, pretend nothing happened and move on to the next StatefulSet.host:type: objectdescription: |Whether the operator during reconcile procedure should wait for a ClickHouse host:- to be excluded from a ClickHouse cluster- to complete all running queries- to be included into a ClickHouse clusterrespectfully before moving forwardproperties:wait:type: objectproperties:exclude:  &TypeStringBooltype: stringdescription: "Whether the operator during reconcile procedure should wait for a ClickHouse host to be excluded from a ClickHouse cluster"enum:# List StringBoolXXX constants from model- ""- "0"- "1"- "False"- "false"- "True"- "true"- "No"- "no"- "Yes"- "yes"- "Off"- "off"- "On"- "on"- "Disable"- "disable"- "Enable"- "enable"- "Disabled"- "disabled"- "Enabled"- "enabled"queries:<<: *TypeStringBooldescription: "Whether the operator during reconcile procedure should wait for a ClickHouse host to complete all running queries"include:<<: *TypeStringBooldescription: "Whether the operator during reconcile procedure should wait for a ClickHouse host to be included into a ClickHouse cluster"annotation:type: objectdescription: "defines which metadata.annotations items will include or exclude during render StatefulSet, Pod, PVC resources"properties:include:type: arraydescription: |When propagating labels from the chi's `metadata.annotations` section to child objects' `metadata.annotations`,include annotations with names from the following listitems:type: stringexclude:type: arraydescription: |When propagating labels from the chi's `metadata.annotations` section to child objects' `metadata.annotations`,exclude annotations with names from the following listitems:type: stringlabel:type: objectdescription: "defines which metadata.labels will include or exclude during render StatefulSet, Pod, PVC resources"properties:include:type: arraydescription: |When propagating labels from the chi's `metadata.labels` section to child objects' `metadata.labels`,include labels from the following listitems:type: stringexclude:type: arrayitems:type: stringdescription: |When propagating labels from the chi's `metadata.labels` section to child objects' `metadata.labels`,exclude labels from the following listappendScope:<<: *TypeStringBooldescription: |Whether to append *Scope* labels to StatefulSet and Pod- "LabelShardScopeIndex"- "LabelReplicaScopeIndex"- "LabelCHIScopeIndex"- "LabelCHIScopeCycleSize"- "LabelCHIScopeCycleIndex"- "LabelCHIScopeCycleOffset"- "LabelClusterScopeIndex"- "LabelClusterScopeCycleSize"- "LabelClusterScopeCycleIndex"- "LabelClusterScopeCycleOffset"statefulSet:type: objectdescription: "define StatefulSet-specific parameters"properties:revisionHistoryLimit:type: integerdescription: |revisionHistoryLimit is the maximum number of revisions that will bemaintained in the StatefulSet's revision history.                         Look details in `statefulset.spec.revisionHistoryLimit`pod:type: objectdescription: "define pod specific parameters"properties:terminationGracePeriod:type: integerdescription: |Optional duration in seconds the pod needs to terminate gracefully. Look details in `pod.spec.terminationGracePeriodSeconds`logger:type: objectdescription: "allow setup clickhouse-operator logger behavior"properties:logtostderr:type: stringdescription: "boolean, allows logs to stderr"alsologtostderr:type: stringdescription: "boolean allows logs to stderr and files both"v:type: stringdescription: "verbosity level of clickhouse-operator log, default - 1 max - 9"stderrthreshold:type: stringvmodule:type: stringdescription: |Comma-separated list of filename=N, where filename (can be a pattern) must have no .go ext, and N is a V level.Ex.: file*=2 sets the 'V' to 2 in all files with names like file*.log_backtrace_at:type: stringdescription: |It can be set to a file and line number with a logging line.Ex.: file.go:123Each time when this line is being executed, a stack trace will be written to the Info log.
---
# Template Parameters:
#
# OPERATOR_VERSION=0.23.6
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: clickhousekeeperinstallations.clickhouse-keeper.altinity.comlabels:clickhouse-keeper.altinity.com/chop: 0.23.6
spec:group: clickhouse-keeper.altinity.comscope: Namespacednames:kind: ClickHouseKeeperInstallationsingular: clickhousekeeperinstallationplural: clickhousekeeperinstallationsshortNames:- chkversions:- name: v1served: truestorage: trueadditionalPrinterColumns:- name: statustype: stringdescription: CHK statusjsonPath: .status.status- name: replicastype: integerdescription: Replica countpriority: 1 # show in wide viewjsonPath: .status.replicas- name: agetype: datedescription: Age of the resource# Displayed in all prioritiesjsonPath: .metadata.creationTimestampsubresources:status: {}schema:openAPIV3Schema:type: objectrequired:- specdescription: "define a set of Kubernetes resources (StatefulSet, PVC, Service, ConfigMap) which describe behavior one ClickHouse Keeper cluster"properties:apiVersion:type: stringdescription: |APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resourceskind:type: stringdescription: |Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindsmetadata:type: objectstatus:type: objectdescription: |Current ClickHouseKeeperInstallation status, contains many fields like overall status, desired replicas and ready replica list with their endpointsproperties:chop-version:type: stringdescription: "ClickHouse operator version"chop-commit:type: stringdescription: "ClickHouse operator git commit SHA"chop-date:type: stringdescription: "ClickHouse operator build date"chop-ip:type: stringdescription: "IP address of the operator's pod which managed this CHI"status:type: stringdescription: "Status"replicas:type: integerformat: int32description: Replicas is the number of number of desired replicas in the clusterreadyReplicas:type: arraydescription: ReadyReplicas is the array of endpoints of those ready replicas in the clusteritems:type: objectproperties:host:type: stringdescription: dns name or ip address for Keeper nodeport:type: integerminimum: 0maximum: 65535description: TCP port which used to connect to Keeper nodesecure:type: stringdescription: if a secure connection to Keeper is requirednormalized:type: objectdescription: "Normalized CHK requested"x-kubernetes-preserve-unknown-fields: truenormalizedCompleted:type: objectdescription: "Normalized CHK completed"x-kubernetes-preserve-unknown-fields: truespec:type: objectdescription: KeeperSpec defines the desired state of a Keeper clusterproperties:namespaceDomainPattern:type: stringdescription: |Custom domain pattern which will be used for DNS names of `Service` or `Pod`.Typical use scenario - custom cluster domain in Kubernetes clusterExample: %s.svc.my.testreplicas:type: integerformat: int32description: |Replicas is the expected size of the keeper cluster.The valid range of size is from 1 to 7.minimum: 1maximum: 7configuration:type: objectdescription: "allows configure multiple aspects and behavior for `clickhouse-server` instance and also allows describe multiple `clickhouse-server` clusters inside one `chi` resource"# nullable: trueproperties:settings:type: objectdescription: "allows configure multiple aspects and behavior for `clickhouse-keeper` instance"x-kubernetes-preserve-unknown-fields: trueclusters:type: arraydescription: |describes ClickHouseKeeper clusters layout and allows change settings on cluster-level and replica-level# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "cluster name, used to identify set of ClickHouseKeeper servers and wide used during generate names of related Kubernetes resources"minLength: 1# See namePartClusterMaxLen constmaxLength: 15pattern: "^[a-zA-Z0-9-]{0,15}$"layout:type: objectdescription: |describe current cluster layout, how many replicas# nullable: trueproperties:replicasCount:type: integerdescription: "how many replicas in ClickHouseKeeper cluster"templates:type: objectdescription: "allows define templates which will use for render Kubernetes resources like StatefulSet, ConfigMap, Service, PVC, by default, clickhouse-operator have own templates, but you can override it"# nullable: trueproperties:podTemplates:type: arraydescription: |podTemplate will use during render `Pod` inside `StatefulSet.spec` and allows define rendered `Pod.spec`, pod scheduling distribution and pod zoneMore information: https://github.com/Altinity/clickhouse-operator/blob/master/docs/custom_resource_explained.md#spectemplatespodtemplates# nullable: trueitems:type: object#required:#  - nameproperties:name:type: stringdescription: "template name, could use to link inside top-level `chi.spec.defaults.templates.podTemplate`, cluster-level `chi.spec.configuration.clusters.templates.podTemplate`, shard-level `chi.spec.configuration.clusters.layout.shards.temlates.podTemplate`, replica-level `chi.spec.configuration.clusters.layout.replicas.templates.podTemplate`"metadata:type: objectdescription: |allows pass standard object's metadata from template to PodMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:# TODO specify PodSpectype: objectdescription: "allows define whole Pod.spec inside StaefulSet.spec, look to https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates for details"# nullable: truex-kubernetes-preserve-unknown-fields: truevolumeClaimTemplates:type: arraydescription: "allows define template for rendering `PVC` kubernetes resource, which would use inside `Pod` for mount clickhouse `data`, clickhouse `logs` or something else"# nullable: trueitems:type: object#required:#  - name#  - specproperties:name:type: stringdescription: |template name, could use to link insidetop-level `chi.spec.defaults.templates.dataVolumeClaimTemplate` or `chi.spec.defaults.templates.logVolumeClaimTemplate`,cluster-level `chi.spec.configuration.clusters.templates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.templates.logVolumeClaimTemplate`,shard-level `chi.spec.configuration.clusters.layout.shards.temlates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.layout.shards.temlates.logVolumeClaimTemplate`replica-level `chi.spec.configuration.clusters.layout.replicas.templates.dataVolumeClaimTemplate` or `chi.spec.configuration.clusters.layout.replicas.templates.logVolumeClaimTemplate`metadata:type: objectdescription: |allows to pass standard object's metadata from template to PVCMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:type: objectdescription: |allows define all aspects of `PVC` resourceMore info: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims# nullable: truex-kubernetes-preserve-unknown-fields: trueserviceTemplates:type: arraydescription: |allows define template for rendering `Service` which would get endpoint from Pods which scoped chi-wide, cluster-wide, shard-wide, replica-wide level# nullable: trueitems:type: object#required:#  - name#  - specproperties:name:type: stringdescription: |template name, could use to link insidechi-level `chi.spec.defaults.templates.serviceTemplate`cluster-level `chi.spec.configuration.clusters.templates.clusterServiceTemplate`shard-level `chi.spec.configuration.clusters.layout.shards.temlates.shardServiceTemplate`replica-level `chi.spec.configuration.clusters.layout.replicas.templates.replicaServiceTemplate` or `chi.spec.configuration.clusters.layout.shards.replicas.replicaServiceTemplate`metadata:# TODO specify ObjectMetatype: objectdescription: |allows pass standard object's metadata from template to ServiceCould be use for define specificly for Cloud Provider metadata which impact to behavior of serviceMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata# nullable: truex-kubernetes-preserve-unknown-fields: truespec:# TODO specify ServiceSpectype: objectdescription: |describe behavior of generated ServiceMore info: https://kubernetes.io/docs/concepts/services-networking/service/# nullable: truex-kubernetes-preserve-unknown-fields: true
---
# Template Parameters:
#
# COMMENT=
# NAMESPACE=kube-system
# NAME=clickhouse-operator
#
# Setup ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:name: clickhouse-operatornamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6
---
# Template Parameters:
#
# NAMESPACE=kube-system
# COMMENT=#
# ROLE_KIND=ClusterRole
# ROLE_NAME=clickhouse-operator-kube-system
# ROLE_BINDING_KIND=ClusterRoleBinding
# ROLE_BINDING_NAME=clickhouse-operator-kube-system
## Specifies either
#   ClusterRole
# or
#   Role
# to be bound to ServiceAccount.
# ClusterRole is namespace-less and must have unique name
# Role is namespace-bound
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: clickhouse-operator-kube-system#namespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6
rules:## Core API group#- apiGroups:- ""resources:- configmaps- services- persistentvolumeclaims- secretsverbs:- get- list- patch- update- watch- create- delete- apiGroups:- ""resources:- endpointsverbs:- get- list- watch- apiGroups:- ""resources:- eventsverbs:- create- apiGroups:- ""resources:- persistentvolumesverbs:- get- list- patch- update- watch- apiGroups:- ""resources:- podsverbs:- get- list- patch- update- watch- delete- apiGroups:- ""resources:- secretsverbs:- get- list## apps.* resources#- apiGroups:- appsresources:- statefulsetsverbs:- get- list- patch- update- watch- create- delete- apiGroups:- appsresources:- replicasetsverbs:- get- patch- update- delete# The operator deployment personally, identified by name- apiGroups:- appsresources:- deploymentsresourceNames:- clickhouse-operatorverbs:- get- patch- update- delete## policy.* resources#- apiGroups:- policyresources:- poddisruptionbudgetsverbs:- get- list- patch- update- watch- create- delete## apiextensions#- apiGroups:- apiextensions.k8s.ioresources:- customresourcedefinitionsverbs:- get- list## The operator's specific Custom Resources## clickhouse - related resources- apiGroups:- clickhouse.altinity.comresources:- clickhouseinstallationsverbs:- get- list- watch- patch- update- delete- apiGroups:- clickhouse.altinity.comresources:- clickhouseinstallationtemplates- clickhouseoperatorconfigurationsverbs:- get- list- watch- apiGroups:- clickhouse.altinity.comresources:- clickhouseinstallations/finalizers- clickhouseinstallationtemplates/finalizers- clickhouseoperatorconfigurations/finalizersverbs:- update- apiGroups:- clickhouse.altinity.comresources:- clickhouseinstallations/status- clickhouseinstallationtemplates/status- clickhouseoperatorconfigurations/statusverbs:- get- update- patch- create- delete# clickhouse-keeper - related resources- apiGroups:- clickhouse-keeper.altinity.comresources:- clickhousekeeperinstallationsverbs:- get- list- watch- patch- update- delete- apiGroups:- clickhouse-keeper.altinity.comresources:- clickhousekeeperinstallations/finalizersverbs:- update- apiGroups:- clickhouse-keeper.altinity.comresources:- clickhousekeeperinstallations/statusverbs:- get- update- patch- create- delete
---
# Specifies either
#   ClusterRoleBinding between ClusterRole and ServiceAccount.
# or
#   RoleBinding between Role and ServiceAccount.
# ClusterRoleBinding is namespace-less and must have unique name
# RoleBinding is namespace-bound
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: clickhouse-operator-kube-system#namespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: clickhouse-operator-kube-system
subjects:- kind: ServiceAccountname: clickhouse-operatornamespace: kube-system
---
# Template Parameters:
#
# NAME=etc-clickhouse-operator-files
# NAMESPACE=kube-system
# COMMENT=
#
apiVersion: v1
kind: ConfigMap
metadata:name: etc-clickhouse-operator-filesnamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
data:config.yaml: |# IMPORTANT# This file is auto-generated# Do not edit this file - all changes would be lost# Edit appropriate template in the following folder:# deploy/builder/templates-config# IMPORTANT## Template parameters available:#   WATCH_NAMESPACES=#   CH_USERNAME_PLAIN=#   CH_PASSWORD_PLAIN=#   CH_CREDENTIALS_SECRET_NAMESPACE=#   CH_CREDENTIALS_SECRET_NAME=clickhouse-operator#   VERBOSITY=1#################################################### Watch section##################################################watch:# List of namespaces where clickhouse-operator watches for events.# Concurrently running operators should watch on different namespaces.# IMPORTANT# Regexp is applicable.#namespaces: ["dev", "test"]namespaces: []clickhouse:configuration:#################################################### Configuration files section##################################################file:# Each 'path' can be either absolute or relative.# In case path is absolute - it is used as is# In case path is relative - it is relative to the folder where configuration file you are reading right now is located.path:# Path to the folder where ClickHouse configuration files common for all instances within a CHI are located.common: config.d# Path to the folder where ClickHouse configuration files unique for each instance (host) within a CHI are located.host: conf.d# Path to the folder where ClickHouse configuration files with users' settings are located.# Files are common for all instances within a CHI.user: users.d#################################################### Configuration users section##################################################user:# Default settings for user accounts, created by the operator.# IMPORTANT. These are not access credentials or settings for 'default' user account,# it is a template for filling out missing fields for all user accounts to be created by the operator,# with the following EXCEPTIONS:# 1. 'default' user account DOES NOT use provided password, but uses all the rest of the fields.#    Password for 'default' user account has to be provided explicitly, if to be used.# 2. CHOP user account DOES NOT use:#    - profile setting. It uses predefined profile called 'clickhouse_operator'#    - quota setting. It uses empty quota name.#    - networks IP setting. Operator specifies 'networks/ip' user setting to match operators' pod IP only.#    - password setting. Password for CHOP account is used from 'clickhouse.access.*' sectiondefault:# Default values for ClickHouse user account(s) created by the operator#   1. user/profile - string#   2. user/quota - string#   3. user/networks/ip - multiple strings#   4. user/password - string# These values can be overwritten on per-user basis.profile: "default"quota: "default"networksIP:- "::1"- "127.0.0.1"password: "default"#################################################### Configuration network section##################################################network:# Default host_regexp to limit network connectivity from outsidehostRegexpTemplate: "(chi-{chi}-[^.]+\\d+-\\d+|clickhouse\\-{chi})\\.{namespace}\\.svc\\.cluster\\.local$"#################################################### Configuration restart policy section## Configuration restart policy describes what configuration changes require ClickHouse restart##################################################configurationRestartPolicy:rules:# IMPORTANT!# Special version of "*" - default version - has to satisfy all ClickHouse versions.# Default version will also be used in case ClickHouse version is unknown.# ClickHouse version may be unknown due to host being down - for example, because of incorrect "settings" section.# ClickHouse is not willing to start in case incorrect/unknown settings are provided in config file.- version: "*"rules:# see https://kb.altinity.com/altinity-kb-setup-and-maintenance/altinity-kb-server-config-files/#server-config-configxml-sections-which-dont-require-restart# to be replaced with "select * from system.server_settings where changeable_without_restart = 'No'"- settings/*: "yes"# single values- settings/access_control_path: "no"- settings/dictionaries_config: "no"- settings/max_server_memory_*: "no"- settings/max_*_to_drop: "no"- settings/max_concurrent_queries: "no"- settings/models_config: "no"- settings/user_defined_executable_functions_config: "no"# structured XML- settings/logger/*: "no"- settings/macros/*: "no"- settings/remote_servers/*: "no"- settings/user_directories/*: "no"- zookeeper/*: "yes"- files/*.xml: "yes"- files/config.d/*.xml: "yes"- files/config.d/*dict*.xml: "no"# exceptions in default profile- profiles/default/background_*_pool_size: "yes"- profiles/default/max_*_for_server: "yes"- version: "21.*"rules:- settings/logger: "yes"##################################################### Access to ClickHouse instances##################################################access:# Possible values for 'scheme' are:#   1. http - force http to be used to connect to ClickHouse instances#   2. https - force https to be used to connect to ClickHouse instances#   3. auto - either http or https is selected based on open portsscheme: "auto"# ClickHouse credentials (username, password and port) to be used by the operator to connect to ClickHouse instances.# These credentials are used for:#   1. Metrics requests#   2. Schema maintenance#   3. DROP DNS CACHE# User with these credentials can be specified in additional ClickHouse .xml config files,# located in 'clickhouse.configuration.file.path.user' folderusername: ""password: ""rootCA: ""# Location of the k8s Secret with username and password to be used by the operator to connect to ClickHouse instances.# Can be used instead of explicitly specified username and password available in sections:#   - clickhouse.access.username#   - clickhouse.access.password# Secret should have two keys:#   1. username#   2. passwordsecret:# Empty `namespace` means that k8s secret would be looked in the same namespace where operator's pod is running.namespace: ""# Empty `name` means no k8s Secret would be looked forname: "clickhouse-operator"# Port where to connect to ClickHouse instances toport: 8123# Timeouts used to limit connection and queries from the operator to ClickHouse instances# Specified in seconds.timeouts:# Timout to setup connection from the operator to ClickHouse instances. In seconds.connect: 1# Timout to perform SQL query from the operator to ClickHouse instances. In seconds.query: 4##################################################### Metrics collection##################################################metrics:# Timeouts used to limit connection and queries from the metrics exporter to ClickHouse instances# Specified in seconds.timeouts:# Timeout used to limit metrics collection request. In seconds.# Upon reaching this timeout metrics collection is aborted and no more metrics are collected in this cycle.# All collected metrics are returned.collect: 9#################################################### Template(s) management section##################################################template:chi:# CHI template updates handling policy# Possible policy values:#   - ReadOnStart. Accept CHIT updates on the operators start only.#   - ApplyOnNextReconcile. Accept CHIT updates at all time. Apply news CHITs on next regular reconcile of the CHIpolicy: ApplyOnNextReconcile# Path to the folder where ClickHouseInstallation templates .yaml manifests are located.# Templates are added to the list of all templates and used when CHI is reconciled.# Templates are applied in sorted alpha-numeric order.path: templates.d#################################################### Reconcile section##################################################reconcile:# Reconcile runtime settingsruntime:# Max number of concurrent CHI reconciles in progressreconcileCHIsThreadsNumber: 10# The operator reconciles shards concurrently in each CHI with the following limitations:#   1. Number of shards being reconciled (and thus having hosts down) in each CHI concurrently#      can not be greater than 'reconcileShardsThreadsNumber'.#   2. Percentage of shards being reconciled (and thus having hosts down) in each CHI concurrently#      can not be greater than 'reconcileShardsMaxConcurrencyPercent'.#   3. The first shard is always reconciled alone. Concurrency starts from the second shard and onward.# Thus limiting number of shards being reconciled (and thus having hosts down) in each CHI by both number and percentage# Max number of concurrent shard reconciles within one CHI in progressreconcileShardsThreadsNumber: 5# Max percentage of concurrent shard reconciles within one CHI in progressreconcileShardsMaxConcurrencyPercent: 50# Reconcile StatefulSet scenariostatefulSet:# Create StatefulSet scenariocreate:# What to do in case created StatefulSet is not in 'Ready' after `reconcile.statefulSet.update.timeout` seconds# Possible options:# 1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is,#    do not try to fix or delete or update it, just abort reconcile cycle.#    Do not proceed to the next StatefulSet(s) and wait for an admin to assist.# 2. delete - delete newly created problematic StatefulSet and follow 'abort' path afterwards.# 3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.onFailure: ignore# Update StatefulSet scenarioupdate:# How many seconds to wait for created/updated StatefulSet to be 'Ready'timeout: 300# How many seconds to wait between checks/polls for created/updated StatefulSet statuspollInterval: 5# What to do in case updated StatefulSet is not in 'Ready' after `reconcile.statefulSet.update.timeout` seconds# Possible options:# 1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is,#    do not try to fix or delete or update it, just abort reconcile cycle.#    Do not proceed to the next StatefulSet(s) and wait for an admin to assist.# 2. rollback - delete Pod and rollback StatefulSet to previous Generation.#    Pod would be recreated by StatefulSet based on rollback-ed StatefulSet configuration.#    Follow 'abort' path afterwards.# 3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.onFailure: abort# Reconcile Host scenariohost:# Whether the operator during reconcile procedure should wait for a ClickHouse host:#   - to be excluded from a ClickHouse cluster#   - to complete all running queries#   - to be included into a ClickHouse cluster# respectfully before moving forwardwait:exclude: truequeries: trueinclude: false#################################################### Annotations management section##################################################annotation:# Applied when:#  1. Propagating annotations from the CHI's `metadata.annotations` to child objects' `metadata.annotations`,#  2. Propagating annotations from the CHI Template's `metadata.annotations` to CHI's `metadata.annotations`,# Include annotations from the following list:# Applied only when not empty. Empty list means "include all, no selection"include: []# Exclude annotations from the following list:exclude: []#################################################### Labels management section##################################################label:# Applied when:#  1. Propagating labels from the CHI's `metadata.labels` to child objects' `metadata.labels`,#  2. Propagating labels from the CHI Template's `metadata.labels` to CHI's `metadata.labels`,# Include labels from the following list:# Applied only when not empty. Empty list means "include all, no selection"include: []# Exclude labels from the following list:# Applied only when not empty. Empty list means "nothing to exclude, no selection"exclude: []# Whether to append *Scope* labels to StatefulSet and Pod.# Full list of available *scope* labels check in 'labeler.go'#  LabelShardScopeIndex#  LabelReplicaScopeIndex#  LabelCHIScopeIndex#  LabelCHIScopeCycleSize#  LabelCHIScopeCycleIndex#  LabelCHIScopeCycleOffset#  LabelClusterScopeIndex#  LabelClusterScopeCycleSize#  LabelClusterScopeCycleIndex#  LabelClusterScopeCycleOffsetappendScope: "no"#################################################### StatefulSet management section##################################################statefulSet:revisionHistoryLimit: 0#################################################### Pod management section##################################################pod:# Grace period for Pod termination.# How many seconds to wait between sending# SIGTERM and SIGKILL during Pod termination process.# Increase this number is case of slow shutdown.terminationGracePeriod: 30#################################################### Log parameters section##################################################logger:logtostderr: "true"alsologtostderr: "false"v: "1"stderrthreshold: ""vmodule: ""log_backtrace_at: ""---
# Template Parameters:
#
# NAME=etc-clickhouse-operator-confd-files
# NAMESPACE=kube-system
# COMMENT=
#
apiVersion: v1
kind: ConfigMap
metadata:name: etc-clickhouse-operator-confd-filesnamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
data:
---
# Template Parameters:
#
# NAME=etc-clickhouse-operator-configd-files
# NAMESPACE=kube-system
# COMMENT=
#
apiVersion: v1
kind: ConfigMap
metadata:name: etc-clickhouse-operator-configd-filesnamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
data:01-clickhouse-01-listen.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><yandex><!-- Listen wildcard address to allow accepting connections from other containers and host network. --><listen_host>::</listen_host><listen_host>0.0.0.0</listen_host><listen_try>1</listen_try></yandex>01-clickhouse-02-logger.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><yandex><logger><!-- Possible levels: https://github.com/pocoproject/poco/blob/devel/Foundation/include/Poco/Logger.h#L439 --><level>debug</level><log>/var/log/clickhouse-server/clickhouse-server.log</log><errorlog>/var/log/clickhouse-server/clickhouse-server.err.log</errorlog><size>1000M</size><count>10</count><!-- Default behavior is autodetection (log to console if not daemon mode and is tty) --><console>1</console></logger></yandex>01-clickhouse-03-query_log.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><yandex><query_log replace="1"><database>system</database><table>query_log</table><engine>Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + interval 30 day</engine><flush_interval_milliseconds>7500</flush_interval_milliseconds></query_log><query_thread_log remove="1"/></yandex>01-clickhouse-04-part_log.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><yandex><part_log replace="1"><database>system</database><table>part_log</table><engine>Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + interval 30 day</engine><flush_interval_milliseconds>7500</flush_interval_milliseconds></part_log></yandex>01-clickhouse-05-trace_log.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><yandex><trace_log replace="1"><database>system</database><table>trace_log</table><engine>Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + interval 30 day</engine><flush_interval_milliseconds>7500</flush_interval_milliseconds></trace_log></yandex>---
# Template Parameters:
#
# NAME=etc-clickhouse-operator-templatesd-files
# NAMESPACE=kube-system
# COMMENT=
#
apiVersion: v1
kind: ConfigMap
metadata:name: etc-clickhouse-operator-templatesd-filesnamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
data:001-templates.json.example: |{"apiVersion": "clickhouse.altinity.com/v1","kind": "ClickHouseInstallationTemplate","metadata": {"name": "01-default-volumeclaimtemplate"},"spec": {"templates": {"volumeClaimTemplates": [{"name": "chi-default-volume-claim-template","spec": {"accessModes": ["ReadWriteOnce"],"resources": {"requests": {"storage": "2Gi"}}}}],"podTemplates": [{"name": "chi-default-oneperhost-pod-template","distribution": "OnePerHost","spec": {"containers" : [{"name": "clickhouse","image": "clickhouse/clickhouse-server:23.8","ports": [{"name": "http","containerPort": 8123},{"name": "client","containerPort": 9000},{"name": "interserver","containerPort": 9009}]}]}}]}}}default-pod-template.yaml.example: |apiVersion: "clickhouse.altinity.com/v1"kind: "ClickHouseInstallationTemplate"metadata:name: "default-oneperhost-pod-template"spec:templates:podTemplates:- name: default-oneperhost-pod-templatedistribution: "OnePerHost"default-storage-template.yaml.example: |apiVersion: "clickhouse.altinity.com/v1"kind: "ClickHouseInstallationTemplate"metadata:name: "default-storage-template-2Gi"spec:templates:volumeClaimTemplates:- name: default-storage-template-2Gispec:accessModes:- ReadWriteOnceresources:requests:storage: 2Gireadme: |Templates in this folder are packaged with an operator and available via 'useTemplate'
---
# Template Parameters:
#
# NAME=etc-clickhouse-operator-usersd-files
# NAMESPACE=kube-system
# COMMENT=
#
apiVersion: v1
kind: ConfigMap
metadata:name: etc-clickhouse-operator-usersd-filesnamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
data:01-clickhouse-operator-profile.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><!--## Template parameters available:#--><yandex><!-- clickhouse-operator user is generated by the operator based on config.yaml in runtime --><profiles><clickhouse_operator><log_queries>0</log_queries><skip_unavailable_shards>1</skip_unavailable_shards><http_connection_timeout>10</http_connection_timeout><max_concurrent_queries_for_all_users>0</max_concurrent_queries_for_all_users><os_thread_priority>0</os_thread_priority></clickhouse_operator></profiles></yandex>02-clickhouse-default-profile.xml: |<!-- IMPORTANT --><!-- This file is auto-generated --><!-- Do not edit this file - all changes would be lost --><!-- Edit appropriate template in the following folder: --><!-- deploy/builder/templates-config --><!-- IMPORTANT --><yandex><profiles><default><os_thread_priority>2</os_thread_priority><log_queries>1</log_queries><connect_timeout_with_failover_ms>1000</connect_timeout_with_failover_ms><distributed_aggregation_memory_efficient>1</distributed_aggregation_memory_efficient><parallel_view_processing>1</parallel_view_processing><do_not_merge_across_partitions_select_final>1</do_not_merge_across_partitions_select_final><load_balancing>nearest_hostname</load_balancing><prefer_localhost_replica>0</prefer_localhost_replica><!-- materialize_ttl_recalculate_only>1</materialize_ttl_recalculate_only> 21.10 and above --></default></profiles></yandex>
---
#
# Template parameters available:
#   NAMESPACE=kube-system
#   COMMENT=
#   OPERATOR_VERSION=0.23.6
#   CH_USERNAME_SECRET_PLAIN=clickhouse_operator
#   CH_PASSWORD_SECRET_PLAIN=clickhouse_operator_password
#
apiVersion: v1
kind: Secret
metadata:name: clickhouse-operatornamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
type: Opaque
stringData:username: clickhouse_operatorpassword: clickhouse_operator_password
---
# Template Parameters:
#
# NAMESPACE=kube-system
# COMMENT=
# OPERATOR_IMAGE=altinity/clickhouse-operator:0.23.6
# OPERATOR_IMAGE_PULL_POLICY=Always
# METRICS_EXPORTER_IMAGE=altinity/metrics-exporter:0.23.6
# METRICS_EXPORTER_IMAGE_PULL_POLICY=Always
#
# Setup Deployment for clickhouse-operator
# Deployment would be created in kubectl-specified namespace
kind: Deployment
apiVersion: apps/v1
metadata:name: clickhouse-operatornamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
spec:replicas: 1selector:matchLabels:app: clickhouse-operatortemplate:metadata:labels:app: clickhouse-operatorannotations:prometheus.io/port: '8888'prometheus.io/scrape: 'true'clickhouse-operator-metrics/port: '9999'clickhouse-operator-metrics/scrape: 'true'spec:serviceAccountName: clickhouse-operatorvolumes:- name: etc-clickhouse-operator-folderconfigMap:name: etc-clickhouse-operator-files- name: etc-clickhouse-operator-confd-folderconfigMap:name: etc-clickhouse-operator-confd-files- name: etc-clickhouse-operator-configd-folderconfigMap:name: etc-clickhouse-operator-configd-files- name: etc-clickhouse-operator-templatesd-folderconfigMap:name: etc-clickhouse-operator-templatesd-files- name: etc-clickhouse-operator-usersd-folderconfigMap:name: etc-clickhouse-operator-usersd-filescontainers:- name: clickhouse-operatorimage: altinity/clickhouse-operator:0.23.6imagePullPolicy: AlwaysvolumeMounts:- name: etc-clickhouse-operator-foldermountPath: /etc/clickhouse-operator- name: etc-clickhouse-operator-confd-foldermountPath: /etc/clickhouse-operator/conf.d- name: etc-clickhouse-operator-configd-foldermountPath: /etc/clickhouse-operator/config.d- name: etc-clickhouse-operator-templatesd-foldermountPath: /etc/clickhouse-operator/templates.d- name: etc-clickhouse-operator-usersd-foldermountPath: /etc/clickhouse-operator/users.denv:# Pod-specific# spec.nodeName: ip-172-20-52-62.ec2.internal- name: OPERATOR_POD_NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName# metadata.name: clickhouse-operator-6f87589dbb-ftcsf- name: OPERATOR_POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name# metadata.namespace: kube-system- name: OPERATOR_POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace# status.podIP: 100.96.3.2- name: OPERATOR_POD_IPvalueFrom:fieldRef:fieldPath: status.podIP# spec.serviceAccount: clickhouse-operator# spec.serviceAccountName: clickhouse-operator- name: OPERATOR_POD_SERVICE_ACCOUNTvalueFrom:fieldRef:fieldPath: spec.serviceAccountName# Container-specific- name: OPERATOR_CONTAINER_CPU_REQUESTvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: requests.cpu- name: OPERATOR_CONTAINER_CPU_LIMITvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: limits.cpu- name: OPERATOR_CONTAINER_MEM_REQUESTvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: requests.memory- name: OPERATOR_CONTAINER_MEM_LIMITvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: limits.memoryports:- containerPort: 9999name: metrics- name: metrics-exporterimage: altinity/metrics-exporter:0.23.6imagePullPolicy: AlwaysvolumeMounts:- name: etc-clickhouse-operator-foldermountPath: /etc/clickhouse-operator- name: etc-clickhouse-operator-confd-foldermountPath: /etc/clickhouse-operator/conf.d- name: etc-clickhouse-operator-configd-foldermountPath: /etc/clickhouse-operator/config.d- name: etc-clickhouse-operator-templatesd-foldermountPath: /etc/clickhouse-operator/templates.d- name: etc-clickhouse-operator-usersd-foldermountPath: /etc/clickhouse-operator/users.denv:# Pod-specific# spec.nodeName: ip-172-20-52-62.ec2.internal- name: OPERATOR_POD_NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName# metadata.name: clickhouse-operator-6f87589dbb-ftcsf- name: OPERATOR_POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name# metadata.namespace: kube-system- name: OPERATOR_POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace# status.podIP: 100.96.3.2- name: OPERATOR_POD_IPvalueFrom:fieldRef:fieldPath: status.podIP# spec.serviceAccount: clickhouse-operator# spec.serviceAccountName: clickhouse-operator- name: OPERATOR_POD_SERVICE_ACCOUNTvalueFrom:fieldRef:fieldPath: spec.serviceAccountName# Container-specific- name: OPERATOR_CONTAINER_CPU_REQUESTvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: requests.cpu- name: OPERATOR_CONTAINER_CPU_LIMITvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: limits.cpu- name: OPERATOR_CONTAINER_MEM_REQUESTvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: requests.memory- name: OPERATOR_CONTAINER_MEM_LIMITvalueFrom:resourceFieldRef:containerName: clickhouse-operatorresource: limits.memoryports:- containerPort: 8888name: metrics
---
# Template Parameters:
#
# NAMESPACE=kube-system
# COMMENT=
#
# Setup ClusterIP Service to provide monitoring metrics for Prometheus
# Service would be created in kubectl-specified namespace
# In order to get access outside of k8s it should be exposed as:
# kubectl --namespace prometheus port-forward service/prometheus 9090
# and point browser to localhost:9090
kind: Service
apiVersion: v1
metadata:name: clickhouse-operator-metricsnamespace: kube-systemlabels:clickhouse.altinity.com/chop: 0.23.6app: clickhouse-operator
spec:ports:- port: 8888name: clickhouse-metrics- port: 9999name: operator-metricsselector:app: clickhouse-operator

部署clickhouse-operator

kubectl apply -f clickhouse-operator.yaml

默认部署在kube-system中

3.2. 部署clickhouse

注意几个配置的地方

zookeeper:填写zookeeper的host信息

users:可以自定义用户信息

cluster:定义集群信息 最简单的方式是直接使用shardsCount和replicaCount定义副本数量和分配数量

clickhouse-operator有一套自定义的命名规则 参考官网文档GitCode - 全球开发者的开源社区,开源代码托管平台

apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:name: "ck-cluster"finalizers:- finalizer.clickhouseinstallation.altinity.com
spec:defaults:templates:podTemplate: pod-templateserviceTemplate: svc-templateconfiguration:zookeeper:nodes:- host: zookeeper-0.clickhouse-keepers.default.svc.cluster.localport: 2181- host: zookeeper-1.clickhouse-keepers.default.svc.cluster.localport: 2181
#     clickhouse-keeper:
#       nodes:
#         - host: clickhouse-keeper-0.clickhouse-keepers.svc.cluster.local
#           port: 2181users:user2/networks/ip: "::/0"user2/password: qwertyuser2/profile: defaultclusters:- name: "ck-cluster"layout:shardsCount: 1replicasCount: 2templates:podTemplates:- name: pod-templatemetadata:labels:app: "ck-cluster"spec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- "node1"- "node2"- "node3"podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: appoperator: Invalues:- "ck-cluster-1"topologyKey: "kubernetes.io/hostname"containers:- name: clickhouseimage: clickhouse/clickhouse-server:24.1.2.5volumeMounts:# Specify reference to volume on local filesystem- name: local-pathmountPath: /var/lib/clickhousevolumes:- name: local-pathemptyDir: {}
#              hostPath:
#                path: /data/test/clickhouse/{chi}
#                type: DirectoryOrCreateserviceTemplates:- name: svc-templategenerateName: chendpoint-{chi}metadata:labels:custom.label: "custom.value"spec:ports:- name: httpport: 8123nodePort: 31124targetPort: 8123- name: tcpport: 9000nodePort: 31001targetPort: 9000type: NodePort

创建表

create table test on CLUSTER 'ck-cluster'(id UInt32,sku_id String,total_amount Decimal(16,2),create_time Datetime) engine =ReplicatedMergeTreepartition by toYYYYMMDD(create_time)primary key (id)order by (id,sku_id);insert into  test4 values
(101,'sku_001',1000.00,'2020-06-01 12:00:00') ,
(102,'sku_002',2000.00,'2020-06-01 11:00:00'),
(102,'sku_004',2500.00,'2020-06-01 12:00:00'),
(102,'sku_002',2000.00,'2020-06-01 13:00:00'),
(102,'sku_002',12000.00,'2020-06-01 13:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(102,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(103,'sku_002',600.00,'2020-06-02 12:00:00'),
(104,'sku_002',600.00,'2020-06-02 12:00:00'),
(104,'sku_002',600.00,'2020-06-02 12:00:00'),
(104,'sku_002',600.00,'2020-06-02 12:00:00'),
(104,'sku_002',600.00,'2020-06-02 12:00:00'),
(104,'sku_002',600.00,'2020-06-02 12:00:00'),
(104,'sku_002',600.00,'2020-06-02 12:00:00');

可以发现clickhouse operator会自动创建两个副本表

执行插入数据测试

发现数据是一致的

3.3. 分布式存储

修改分区和分片分别为2

3.3.1. 自定义用户

operator提供灵活的标签来创建自定义用户(默认的default用户用于内部分布式查询)

3.3.2. 建表

定义了用户后 可以使用dbeaver操作数据库

四个节点都成功创建表 进行插入数据测试

如果直接在dbeaver中执行插入 发现 1-1-0和其副本1-0-0插入了32条数据 但是0-0-0和o-1-o没有数据 这说明clickhouse没有进行分片存储 这是因为clickhouse 需要借助分布式表来进行分布式存储

create table test on CLUSTER 'ck-cluster2'(id UInt32,sku_id String,total_amount Decimal(16,2),create_time Datetime) engine =ReplicatedMergeTreepartition by toYYYYMMDD(create_time)primary key (id)order by (id,sku_id);CREATE TABLE distributed_table on cluster 'ck-cluster'
ENGINE = Distributed('ck-cluster', 'default', 'test_distribute', rand());

后续所有操作都操作分布式表 分布式表不存储数据

查询后可以发现数据

再分别观察不同分片上的数据

0分片上有13条数据

1分片上有19条数据

这篇关于Clickhouse集群化(四)使用clickhouse-operator部署clickhouse集群的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1111017

相关文章

闲置电脑也能活出第二春?鲁大师AiNAS让你动动手指就能轻松部署

对于大多数人而言,在这个“数据爆炸”的时代或多或少都遇到过存储告急的情况,这使得“存储焦虑”不再是个别现象,而将会是随着软件的不断臃肿而越来越普遍的情况。从不少手机厂商都开始将存储上限提升至1TB可以见得,我们似乎正处在互联网信息飞速增长的阶段,对于存储的需求也将会不断扩大。对于苹果用户而言,这一问题愈发严峻,毕竟512GB和1TB版本的iPhone可不是人人都消费得起的,因此成熟的外置存储方案开

服务器集群同步时间手记

1.时间服务器配置(必须root用户) (1)检查ntp是否安装 [root@node1 桌面]# rpm -qa|grep ntpntp-4.2.6p5-10.el6.centos.x86_64fontpackages-filesystem-1.41-1.1.el6.noarchntpdate-4.2.6p5-10.el6.centos.x86_64 (2)修改ntp配置文件 [r

中文分词jieba库的使用与实景应用(一)

知识星球:https://articles.zsxq.com/id_fxvgc803qmr2.html 目录 一.定义: 精确模式(默认模式): 全模式: 搜索引擎模式: paddle 模式(基于深度学习的分词模式): 二 自定义词典 三.文本解析   调整词出现的频率 四. 关键词提取 A. 基于TF-IDF算法的关键词提取 B. 基于TextRank算法的关键词提取

使用SecondaryNameNode恢复NameNode的数据

1)需求: NameNode进程挂了并且存储的数据也丢失了,如何恢复NameNode 此种方式恢复的数据可能存在小部分数据的丢失。 2)故障模拟 (1)kill -9 NameNode进程 [lytfly@hadoop102 current]$ kill -9 19886 (2)删除NameNode存储的数据(/opt/module/hadoop-3.1.4/data/tmp/dfs/na

HDFS—集群扩容及缩容

白名单:表示在白名单的主机IP地址可以,用来存储数据。 配置白名单步骤如下: 1)在NameNode节点的/opt/module/hadoop-3.1.4/etc/hadoop目录下分别创建whitelist 和blacklist文件 (1)创建白名单 [lytfly@hadoop102 hadoop]$ vim whitelist 在whitelist中添加如下主机名称,假如集群正常工作的节

Hadoop集群数据均衡之磁盘间数据均衡

生产环境,由于硬盘空间不足,往往需要增加一块硬盘。刚加载的硬盘没有数据时,可以执行磁盘数据均衡命令。(Hadoop3.x新特性) plan后面带的节点的名字必须是已经存在的,并且是需要均衡的节点。 如果节点不存在,会报如下错误: 如果节点只有一个硬盘的话,不会创建均衡计划: (1)生成均衡计划 hdfs diskbalancer -plan hadoop102 (2)执行均衡计划 hd

Hadoop数据压缩使用介绍

一、压缩原则 (1)运算密集型的Job,少用压缩 (2)IO密集型的Job,多用压缩 二、压缩算法比较 三、压缩位置选择 四、压缩参数配置 1)为了支持多种压缩/解压缩算法,Hadoop引入了编码/解码器 2)要在Hadoop中启用压缩,可以配置如下参数

Makefile简明使用教程

文章目录 规则makefile文件的基本语法:加在命令前的特殊符号:.PHONY伪目标: Makefilev1 直观写法v2 加上中间过程v3 伪目标v4 变量 make 选项-f-n-C Make 是一种流行的构建工具,常用于将源代码转换成可执行文件或者其他形式的输出文件(如库文件、文档等)。Make 可以自动化地执行编译、链接等一系列操作。 规则 makefile文件

使用opencv优化图片(画面变清晰)

文章目录 需求影响照片清晰度的因素 实现降噪测试代码 锐化空间锐化Unsharp Masking频率域锐化对比测试 对比度增强常用算法对比测试 需求 对图像进行优化,使其看起来更清晰,同时保持尺寸不变,通常涉及到图像处理技术如锐化、降噪、对比度增强等 影响照片清晰度的因素 影响照片清晰度的因素有很多,主要可以从以下几个方面来分析 1. 拍摄设备 相机传感器:相机传

阿里开源语音识别SenseVoiceWindows环境部署

SenseVoice介绍 SenseVoice 专注于高精度多语言语音识别、情感辨识和音频事件检测多语言识别: 采用超过 40 万小时数据训练,支持超过 50 种语言,识别效果上优于 Whisper 模型。富文本识别:具备优秀的情感识别,能够在测试数据上达到和超过目前最佳情感识别模型的效果。支持声音事件检测能力,支持音乐、掌声、笑声、哭声、咳嗽、喷嚏等多种常见人机交互事件进行检测。高效推