Redis 数据的持久化 RDB、AOF、RDB + AOF、No persistence 各自优缺点

2023-12-11 02:01

本文主要是介绍Redis 数据的持久化 RDB、AOF、RDB + AOF、No persistence 各自优缺点,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

文章目录

    • 一、RDB (Redis Database)
      • 1.1 RDB 优势
      • 1.2 RDB 缺点
      • 1.3 RDB 如何工作
      • 1.4 RDB配置
      • 1.5 开启/关闭,RDB快照策略,save指令
      • 1.6 持久化硬盘文件,dbfilename指令
      • 1.7 持久化硬盘文件的存储地址,dir指令
    • 二、AOF (Append Only File)
      • 2.1 AOF 优势
      • 2.2 AOF 缺点
      • 2.3 AOF 如何工作
      • 2.4 AOF配置
      • 2.5 开启/关闭,appendonly指令
      • 2.6 AOF文件,appendfilename指令
      • 2.7 AOF文件目录,dir指令
      • 2.8 AOF追加策略,appendfsync指令
      • 2.9 AOF写入被意外中断,数据也能恢复到上一句命令,可靠性高
    • 三、No persistence 禁用持久化
    • 四、RDB 和 AOF 能否同时开启?
    • 五、如何选择 RDB 和 AOF?
    • 六、官方文档

持久化,是指将Redis内存中的数据写入磁盘,当遇到重启Redis或重启服务器时,再将磁盘上的数据恢复到内存中

Redis持久化选项有4种
RDB (Redis Database) 以指定的时间间隔进行数据的快照备份。
AOF (Append Only File) 记录更改数据的命令(例如SET), 然后在重启时再次执行这些命令,从而恢复数据。
RDB + AOF 组合使用
No persistence 禁用持久化


一、RDB (Redis Database)

RDB开启后,Redis 可以自动创建某个时间点的快照,以便重启的时候,将快照中的数据恢复在内存中。

1.1 RDB 优势

  • 以快照的形式备份,非常适合灾难后进行数据的恢复;大数据时,更快地重新启动。
  • 因为是以时间节点的形式备份,节省了资源,也最大限度地提高了Redis的性能。

1.2 RDB 缺点

  • 因为是以快照的形式备份,会阻塞线程,来保证数据完整性,当遇到大数据时,阻塞时间会被放大。
  • 以时间节点的形式处理,服务停止工作时,可能还没到达时间点,会导致部分数据丢失,只能恢复到上一个时间节点上。
  • Redis运行时,手动复制RDB文件是安全的,那怕Redis正在写入临时RDB文件,原始的RDB文件也是完整的。
  • 每个时刻,RDB文件会有一份,可以创建cron任务,备份RDB文件。记录不同时刻的RDB文件。

1.3 RDB 如何工作

达到配置中save指令条件时,子进程将内存中的数据集写入临时RDB文件,再进行原子替换操作,替换旧的RDB文件。 此时RDB文件只会有一份

1.4 RDB配置

以windows版为例, 在redis.conf中 (redis.windows-service.conf 和 redis.windows.conf)

redis.windows-service.conf 旨在作为服务/守护进程运行。这意味着它应该在后台运行并由操作系统管理(重启时启动,崩溃时重新启动等)。

redis.windows.conf 旨在从命令行或脚本运行并在用户空间中进行管理。

两个配置文件基本相同,更细的区别,不再赘述。

windows服务管理中启用、停用redis服务,仅修改redis.windows-service.conf配置即可

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""save 900 1
save 300 10
save 60 10000# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes# The filename where to dump the DB
dbfilename dump.rdb# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./

1.5 开启/关闭,RDB快照策略,save指令

# 900秒(15分钟)后,如果至少有一个按键发生变化
save 900 1  
# 300秒(5分钟)后,如果至少有10个按键发生变化  
save 300 10 
# 60秒后,如果至少有10000个密钥发生更改   
save 60 10000  
  • 注释或删除掉所有策略,就可以禁用RDB模式
  • 添加自定义策略,只需要满足save指令规则即可

1.6 持久化硬盘文件,dbfilename指令

dbfilename dump.rdb

指定了文件的名称、扩展名;dump.rdb是一个的二进制文件


1.7 持久化硬盘文件的存储地址,dir指令

dir ./

默认是:当前安装目录下

AOF和RDB模式,使用了这个公共配置项,如果修改RDB和AOF都受影响


二、AOF (Append Only File)

2.1 AOF 优势

  • 持久化的同步有不同的策略,fsync策略,参照后面文章,可以自行灵活配置
  • 即使由于某种原因(磁盘已满或其他原因),AOF文件中存在不完整命令,redis check aof工具也能很容易地修复它。
  • AOF文件过大时,文件会被拆分和重写,用当前数据集所需的最小操作命令集生成一个全新的AOF文件,再进行原子替换操作,切换新旧AOF文件

2.2 AOF 缺点

  • AOF文件通常比RDB文件大;因为RDB是时间点的数据快照,AOF是整个数据变动过程的完整命令。
  • AOF重写过程时, 可能会占用更多内存, 先缓冲在内存中,并在最后写入新的AOF

2.3 AOF 如何工作

Redis 收到更改数据集的命令(例如SET)时,将该命令写入到缓冲区,最后依据fsync策略,从缓冲区再追加AOF文件中。需要恢复时,执行中AOF文件中命令即可。

Redis 7.0.0以后,AOF文件过大时,原始的单个AOF文件被拆分为基本文件(最多一个)和增量文件(可能有多个)

2.4 AOF配置

以windows版为例, 在redis.conf中 (redis.windows-service.conf 和 redis.windows.conf)

redis.windows-service.conf 旨在作为服务/守护进程运行。这意味着它应该在后台运行并由操作系统管理(重启时启动,崩溃时重新启动等)。

redis.windows.conf 旨在从命令行或脚本运行并在用户空间中进行管理。

两个配置文件基本相同,更细的区别,不再赘述。

windows服务管理中启用、停用redis服务,仅修改redis.windows-service.conf配置即可

############################## APPEND ONLY MODE ################################ By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.appendonly no# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".# appendfsync always
appendfsync everysec
# appendfsync no# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

2.5 开启/关闭,appendonly指令

默认情况下 Redis 没有开启 AOF(Redis 6.0 之后已经默认是开启了)
通过 appendonly 参数开启:yes/no 开启/关闭

appendonly yes

2.6 AOF文件,appendfilename指令

appendfilename "appendonly.aof"

2.7 AOF文件目录,dir指令

dir ./

默认是:当前安装目录下

AOF和RDB模式,使用了这个公共配置项,如果修改RDB和AOF都受影响


2.8 AOF追加策略,appendfsync指令

appendfsync everysec  
# appendfsync always    
# appendfsync no        
  • everysec 默认值, 后台线程每秒主动同步一次,将缓冲区数据写入到aof文件;性能和数据一致性的折中方案。

  • always 每次缓冲区有更改数据集的命令时,直接写入AOF文件;性能差一点,但数据一致性更高。

  • no 不主动进行写入到aof文件,等待操作系统刷新数据;数据一致比较差,性能更好;Linux 下一般为 30 秒一次。


2.9 AOF写入被意外中断,数据也能恢复到上一句命令,可靠性高

当写入AOF文件时,redis或服务器发生故障,可能导致命令写入了一半(命令不完整),当服务恢复时,redis能自己检测出来异常的命令,将其截断,然后将数据恢复到上一个命令记录的时刻,可能最多损失1秒的数据(配置是 appendfsync everysec)

* Reading RDB preamble from AOF file...
* Reading the remaining AOF tail...
# !!! Warning: short read while loading the AOF file !!!
# !!! Truncating the AOF at offset 439 !!!
# AOF loaded anyway because aof-load-truncated is enabled

三、No persistence 禁用持久化

在上面文章中,已经讲述到了RDB、AOF的开启关闭, 将RDB、AOF都关闭,其实就是不使用持久化。

关闭后,redis对于数据的操作,不会被同步到RDB、AOF文件;
关闭后,重启服务后,也不会从RDB、AOF文件恢复数据,所以重启服务后,redis没有数据。
关闭后,向redis写入key-val数据,如果重启服务,redis就回丢失数据

如果再次开启RDB、AOF模式,重启服务后,会再次加载RDB、AOF文件中的数据。


四、RDB 和 AOF 能否同时开启?

AOF和RDB可以同时启用

AOF文件将优先使用,因为它保证是最完整的


五、如何选择 RDB 和 AOF?

个人理解,RDB是必须要开启的,Redis默认也是开启的, 简单的快照是一个有效的恢复手段,哪怕中间有小部分数据丢失。

普通场景, 能承受一定的数据丢失,可能只开启RDB即可。
数据安全性要求高的场景,可能需要RDB和AOF同时开启,

任何时候,都应该再借助数据库(mqsql、 sql server)进行持久化,做最后一道防线。


六、官方文档

https://redis.io/docs/management/persistence/

这篇关于Redis 数据的持久化 RDB、AOF、RDB + AOF、No persistence 各自优缺点的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/479074

相关文章

大模型研发全揭秘:客服工单数据标注的完整攻略

在人工智能(AI)领域,数据标注是模型训练过程中至关重要的一步。无论你是新手还是有经验的从业者,掌握数据标注的技术细节和常见问题的解决方案都能为你的AI项目增添不少价值。在电信运营商的客服系统中,工单数据是客户问题和解决方案的重要记录。通过对这些工单数据进行有效标注,不仅能够帮助提升客服自动化系统的智能化水平,还能优化客户服务流程,提高客户满意度。本文将详细介绍如何在电信运营商客服工单的背景下进行

基于MySQL Binlog的Elasticsearch数据同步实践

一、为什么要做 随着马蜂窝的逐渐发展,我们的业务数据越来越多,单纯使用 MySQL 已经不能满足我们的数据查询需求,例如对于商品、订单等数据的多维度检索。 使用 Elasticsearch 存储业务数据可以很好的解决我们业务中的搜索需求。而数据进行异构存储后,随之而来的就是数据同步的问题。 二、现有方法及问题 对于数据同步,我们目前的解决方案是建立数据中间表。把需要检索的业务数据,统一放到一张M

关于数据埋点,你需要了解这些基本知识

产品汪每天都在和数据打交道,你知道数据来自哪里吗? 移动app端内的用户行为数据大多来自埋点,了解一些埋点知识,能和数据分析师、技术侃大山,参与到前期的数据采集,更重要是让最终的埋点数据能为我所用,否则可怜巴巴等上几个月是常有的事。   埋点类型 根据埋点方式,可以区分为: 手动埋点半自动埋点全自动埋点 秉承“任何事物都有两面性”的道理:自动程度高的,能解决通用统计,便于统一化管理,但个性化定

使用SecondaryNameNode恢复NameNode的数据

1)需求: NameNode进程挂了并且存储的数据也丢失了,如何恢复NameNode 此种方式恢复的数据可能存在小部分数据的丢失。 2)故障模拟 (1)kill -9 NameNode进程 [lytfly@hadoop102 current]$ kill -9 19886 (2)删除NameNode存储的数据(/opt/module/hadoop-3.1.4/data/tmp/dfs/na

异构存储(冷热数据分离)

异构存储主要解决不同的数据,存储在不同类型的硬盘中,达到最佳性能的问题。 异构存储Shell操作 (1)查看当前有哪些存储策略可以用 [lytfly@hadoop102 hadoop-3.1.4]$ hdfs storagepolicies -listPolicies (2)为指定路径(数据存储目录)设置指定的存储策略 hdfs storagepolicies -setStoragePo

Hadoop集群数据均衡之磁盘间数据均衡

生产环境,由于硬盘空间不足,往往需要增加一块硬盘。刚加载的硬盘没有数据时,可以执行磁盘数据均衡命令。(Hadoop3.x新特性) plan后面带的节点的名字必须是已经存在的,并且是需要均衡的节点。 如果节点不存在,会报如下错误: 如果节点只有一个硬盘的话,不会创建均衡计划: (1)生成均衡计划 hdfs diskbalancer -plan hadoop102 (2)执行均衡计划 hd

【Prometheus】PromQL向量匹配实现不同标签的向量数据进行运算

✨✨ 欢迎大家来到景天科技苑✨✨ 🎈🎈 养成好习惯,先赞后看哦~🎈🎈 🏆 作者简介:景天科技苑 🏆《头衔》:大厂架构师,华为云开发者社区专家博主,阿里云开发者社区专家博主,CSDN全栈领域优质创作者,掘金优秀博主,51CTO博客专家等。 🏆《博客》:Python全栈,前后端开发,小程序开发,人工智能,js逆向,App逆向,网络系统安全,数据分析,Django,fastapi

零基础学习Redis(10) -- zset类型命令使用

zset是有序集合,内部除了存储元素外,还会存储一个score,存储在zset中的元素会按照score的大小升序排列,不同元素的score可以重复,score相同的元素会按照元素的字典序排列。 1. zset常用命令 1.1 zadd  zadd key [NX | XX] [GT | LT]   [CH] [INCR] score member [score member ...]

烟火目标检测数据集 7800张 烟火检测 带标注 voc yolo

一个包含7800张带标注图像的数据集,专门用于烟火目标检测,是一个非常有价值的资源,尤其对于那些致力于公共安全、事件管理和烟花表演监控等领域的人士而言。下面是对此数据集的一个详细介绍: 数据集名称:烟火目标检测数据集 数据集规模: 图片数量:7800张类别:主要包含烟火类目标,可能还包括其他相关类别,如烟火发射装置、背景等。格式:图像文件通常为JPEG或PNG格式;标注文件可能为X

pandas数据过滤

Pandas 数据过滤方法 Pandas 提供了多种方法来过滤数据,可以根据不同的条件进行筛选。以下是一些常见的 Pandas 数据过滤方法,结合实例进行讲解,希望能帮你快速理解。 1. 基于条件筛选行 可以使用布尔索引来根据条件过滤行。 import pandas as pd# 创建示例数据data = {'Name': ['Alice', 'Bob', 'Charlie', 'Dav