本文主要是介绍elasticsearch-curator: es索引生命周期(关闭、删除索引),希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
1,下载安装
- rpm包下载安装
# 防止安装rpm包报错:error: [upel]: elasticsearch-curator NOKEY ;
# error: [upel]: elasticsearch-curator signature check fail
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch #参考https://www.elastic.co/guide/en/elasticsearch/reference/7.17/rpm.htmlrpm -ivh https://packages.elastic.co/curator/5/centos/7/Packages/elasticsearch-curator-5.8.4-1.x86_64.rpm
- 参考:https://www.elastic.co/guide/en/elasticsearch/client/curator/current/installation.html
2, 配置使用
配置
- 关闭索引,action.yaml 加上
ignore_empty_list: True
, 防止条件不满足时报错,导致任务不能继续执行 - 删除索引,action.yaml 加上
continue_if_exception: True
, 防止删除索引报错,导致任务不能继续执行 - action.yaml 配置参考:
action-close模板:https://www.elastic.co/guide/en/elasticsearch/client/curator/current/ex_close.html - 索引名匹配-正则参考:https://www.elastic.co/guide/en/elasticsearch/client/curator/current/filtertype_pattern.html
TEST1:/root # cat curator-close.yml
---
actions:1:action: closedescription: >-关闭指定索引,5天前的:test_logstash_options:skip_flush: Falsedelete_aliases: Falseignore_sync_failures: Trueignore_empty_list: Truedisable_action: Falsefilters:- filtertype: patternkind: prefix#value: test_logstash_2023_11_12value: test_logstash_- filtertype: agesource: namedirection: older#timestring: '%Y.%m.%d'timestring: '%Y_%m_%d'unit: daysunit_count: 52:action: closedescription: >-关闭指定索引,10天前的:^(\w+)_logstash_.*$options:skip_flush: Falsedelete_aliases: Falseignore_sync_failures: Trueignore_empty_list: Truedisable_action: Falsefilters:- filtertype: patternkind: regex#value: logstash-#value: test_logstash_2023_11_12value: '^(\w+)_logstash_.*$'- filtertype: agesource: namedirection: older#timestring: '%Y.%m.%d'timestring: '%Y_%m_%d'unit: daysunit_count: 103:action: delete_indicesdescription: >-删除过期30天的索引: 按天存储options:ignore_empty_list: Truecontinue_if_exception: Truedisable_action: Falsefilters:- filtertype: patternkind: regex#value: test_logstash_2023_11_12value: '^(\w+)_logstash_.*$'- filtertype: agesource: namedirection: older#timestring: '%Y.%m.%d'timestring: '%Y_%m_%d'unit: daysunit_count: 314:action: delete_indicesdescription: >-删除过期30天的索引: 按月存储options:ignore_empty_list: Truecontinue_if_exception: Truedisable_action: Falsefilters:- filtertype: patternkind: regex#value: logstash-#value: test_logstash_2023_11_12value: '^(\w+)_logstash_(\d{4}\S\d{2})$'- filtertype: agesource: namedirection: older#timestring: '%Y.%m.%d'timestring: '%Y_%m_%d'unit: daysunit_count: 60TEST1:/root # cat curator/curatorConfig.yaml
client:hosts:- 185.86.0.106- 185.86.0.107- 185.86.0.108port: 9200url_prefix:use_ssl: Falsecertificate:client_cert:client_key:ssl_no_validate: Falseusername: elasticpassword: elk123456timeout: 60master_only: Falselogging:loglevel: INFOlogfile: /applog/elk/curator/curator.loglogformat: defaultblacklist: ['elasticsearch', 'urllib3']
使用
curator --config 全局配置文件 action动作配置文件
TEST1:/root # curator --config curator/curatorConfig.yaml curator-close.yml TEST1:/root # tail /applog/elk/curator/curator.log -f
2024-01-02 11:19:02,339 INFO Preparing Action ID: 1, "close"
2024-01-02 11:19:02,339 INFO Creating client object and testing connection
2024-01-02 11:19:02,341 INFO Instantiating client object
2024-01-02 11:19:02,342 INFO Testing client connectivity
2024-01-02 11:19:02,345 INFO Successfully created Elasticsearch client object with provided settings
2024-01-02 11:19:02,348 INFO Trying Action ID: 1, "close": 关闭指定索引,5天前的:test_logstash_
2024-01-02 11:19:04,088 INFO Skipping action "close" due to empty list: <class 'curator.exceptions.NoIndices'>
2024-01-02 11:19:04,089 INFO Action ID: 1, "close" completed.2024-01-02 11:19:04,089 INFO Preparing Action ID: 2, "close"
2024-01-02 11:19:04,089 INFO Creating client object and testing connection
2024-01-02 11:19:04,089 INFO Instantiating client object
2024-01-02 11:19:04,090 INFO Testing client connectivity
2024-01-02 11:19:04,095 INFO Successfully created Elasticsearch client object with provided settings
2024-01-02 11:19:04,097 INFO Trying Action ID: 2, "close": 关闭指定索引,10天前的:^(\w+)_logstash_.*$
2024-01-02 11:19:04,610 INFO Closing 30 selected indices: [ 'test_net_logstash_2023_12_15', 'metric_net_logstash_2023_12_16', 'mytest_logstash_2023_12_15',... ]
2024-01-02 11:19:06,240 INFO Action ID: 2, "close" completed.2024-01-02 13:49:45,975 INFO Preparing Action ID: 3, "delete_indices"
2024-01-02 13:49:45,975 INFO Creating client object and testing connection
2024-01-02 13:49:45,975 INFO Instantiating client object
2024-01-02 13:49:45,976 INFO Testing client connectivity
2024-01-02 13:49:45,978 INFO Successfully created Elasticsearch client object with provided settings
2024-01-02 13:49:45,981 INFO Trying Action ID: 3, "delete_indices": 删除过期30天的索引
2024-01-02 13:49:46,218 INFO Deleting 8 selected indices: ['test123_logstash_2023_12_01', 'metric_logstash_2023_12_02', 'monitor_logstash_2023_12_02', ...]
2024-01-02 13:49:46,218 INFO ---deleting index test123_logstash_2023_12_01
2024-01-02 13:49:46,218 INFO ---deleting index metric_logstash_2023_12_02
2024-01-02 13:49:46,218 INFO ---deleting index monitor_logstash_2023_12_02
2024-01-02 13:49:47,475 INFO Action ID: 3, "delete_indices" completed.2024-01-02 13:50:49,047 INFO Trying Action ID: 4, "delete_indices": 删除过期60天的索引: 按月存储
2024-01-02 13:50:49,326 INFO Deleting 22 selected indices: ['test1_logstash_2023_11', 'test2_logstash_2023_12', 'test3_logstash_2023_11'...]
2024-01-02 13:50:49,326 INFO ---deleting index test1_logstash_2023_11
2024-01-02 13:50:49,326 INFO ---deleting index test2_logstash_2023_12
2024-01-02 13:50:49,326 INFO ---deleting index test3_logstash_2023_11
...
2024-01-02 13:50:56,240 INFO Job completed.
这篇关于elasticsearch-curator: es索引生命周期(关闭、删除索引)的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!