本文主要是介绍Cassandra_ Cassandra 定期删除数据方案 设计,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
虽然 Cassandra 本身就是针对于 大数据设计的。但是难免会数据量过大,所以可以定期清除下数据。
场景:清除N天之前 visitor 表内的数据。
经过一天,我总共设计了这几种方案。
方案 一·,二 都使用了 TTL, 这里对TTL进行一个简单的介绍。
TTL :生存时长。在 Cassandra 的计算单位 为 Second 秒
方案一:
插入数据的时候指定 TTL
示例:
INSERT INTO latest_temperatures(weatherstation_id,event_time,temperature)
VALUES (’1234ABCD’,’2013-04-03 07:02:00′,’73F’) USING TTL 20;
关于时间相关属性的参考文档:
https://academy.datastax.com/resources/getting-started-time-series-data-modeling
-----------------
方案二:
给表设定TTL
相关属性:
default_time_to_live 默认的生存时长
表的全部属性介绍:
http://docs.datastax.com/en/cql/3.1/cql/cql_reference/tabProp.html
示例:
创建表
CREATE TABLE test_ttl(
id int PRIMARY KEY,
value text
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 30
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
插入数据:
INSERT INTO test_ttl(id,value)
VALUES (1, '2013-04-03');
校验数据:
SELECT * FROM test.test_ttl;
修改默认的生命时长:
ALTER TABLE test.test_ttl WITH default_time_to_live = 120;
-------------------------------------
方案三:
维护一个额外的插入时间字段
类似于:
https://academy.datastax.com/resources/getting-started-time-series-data-modeling
-------------------------------
方案四: (该方案不可行)
利用WRITETIME 筛选数据
方案不可行的原因:
WRITETIME 的使用示例:
CREATE TABLE test_index(
id int PRIMARY KEY,
value text
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
INSERT INTO test_index(id,value) VALUES (2, 'GoGo');
错误:
SELECT WRITETIME(id) FROM test_index WHERE id=2;
正确:
SELECT WRITETIME(value) FROM test_index WHERE id=2;
WRITETIME 可以转换为 unix 时间戳, 只要截取前10位即可
1481268509217414 -> 1481268509
更多关于 WRITETIME 的介绍:
https://docs.datastax.com/en/cql/3.1/cql/cql_using/use_writetime.html
http://stackoverflow.com/questions/31184376/how-to-filter-cassandra-result-based-on-writetime
这篇关于Cassandra_ Cassandra 定期删除数据方案 设计的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!