Redis基础知识（学习笔记16--持久化 (2)）

三. AOF持久化

AOF，Append Only File，是指Redis将每一次的写操作（命令）都以日志的形式记录到一个AOF文件中的持久化技术。当需要恢复内存数据时，将这些写操作重新执行一次，便会恢复到之前的内存数据状态。

3.1 AOF基础配置

（1）AOF的开启

直接修改配置文件的方式

# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information.

appendonly no

默认情况下AOF持久化是没有开启的，通过修改配置文件中的appendonly的属性为yes。这种方式需要重启生效。

 命令的查看修改方式

状态查看的，可通过下面的命令：

> config get appendonly

修改的话

>config set appendonly yes

注意，这个命令只是修改了内存中的配置。这种命令修改的方式，不需要重启生效。

如果要刷新到配置文件中

>config rewrite

 （2）文件名的配置

# - appendonly.aof.1.base.rdb as a base file.
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"  ###注意：这只是定义了这类文件的前缀

Redis 7 在这里发生了重大变化。原来只有一个appendonly.aof文件，现在具有了三类多个文件：

*** 基础文件：可以是RDF格式也可以是AOF格式。其存放的内容是由RDB转为AOF当时内存的快照数据。该文件可以有多个。

*** 增量文件：以操作日志形式记录转为AOF后的写入操作。该文件可以有多个。

*** 清单文件：用于维护AOF文件的创建顺序，保障激活时的应用顺序，该文件只有一个。

（3）混合式持久化开启

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

对于基本文件(例如：appendonly.aof.1.base.rdb as a base file) 可以是RDF格式也可以是AOF格式，通过aof-use-rdb-preamble 属性可以选择。其默认值为yes，即默认AOF持久化的基本文件为rdb格式文件，也就是默认采用混合式持久化。

（4）AOF完整配置参数

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously【异步】 dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).###概况说RDB存在的风险
#
# The Append Only File is an alternative【可供替代的】 persistence mode that provides
# much better durability【持久性】. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic【巨大的，引人注目的】 event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.###概况说AOF开启带来的好处
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Note that changing this value in a config file of an existing database and
# restarting the server can lead to data loss. A conversion needs to be done
# by setting it via CONFIG command on a live server first.
#
# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information.

appendonly no

# The base name of the append only file.
#
# Redis 7 and newer use a set of append-only files to persist the dataset
# and changes applied to it. There are two basic types of files in use:
#
# - Base files, which are a snapshot representing the complete state of the
#   dataset at the time the file was created. Base files can be either in
#   the form of RDB (binary serialized) or AOF (textual commands).
# - Incremental files, which contain additional commands that were applied
#   to the dataset following the previous file.
#
# In addition, manifest【货单；清单】 files are used to track the files and the order in
# which they were created and should be applied.
#
# Append-only file names are created by Redis following a specific pattern.
# The file name's prefix is based on the 'appendfilename' configuration
# parameter, followed by additional information about the sequence【序号】 and type【类型】.
#
# For example, if appendfilename is set to appendonly.aof, the following file
# names could be derived【产生；获得】:
#
# - appendonly.aof.1.base.rdb as a base file.【大多数情况下，这个类型的文件就一个；如果内存里面的数据量很大时，也可能会不止一个】
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.【这个类型的文件会同时有多个】
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"

# For convenience, Redis stores all persistent append-only files in a dedicated【专门的；特定的】
# directory. The name of the directory is determined by the appenddirname
# configuration parameter.

appenddirname "appendonlydir"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync no". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling【隐式调用】
# BGREWRITEAOF when the AOF log size grows by the specified percentage【指定的百分比】.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

# Redis supports recording timestamp annotations in the AOF to support restoring
# the data from a specific point-in-time. However, using this capability changes
# the AOF format in a way that may not be compatible with existing AOF parsers.
aof-timestamp-enabled no

################################ SHUTDOWN #####################################

 3.2 AOF 文件格式

AOF文件包含三类文件：基本文件、增量文件与清单文件。其中基本文件一般为rdb格式，不再赘述，下面主要看下增量文件和清单文件的内容格式。

（1）Redis 协议

 增量文件扩展名为.aof，采用AOF格式。AOF格式其实就是redis通讯协议格式，AOF持久化文件的本质就是基于redis通讯协议的文本，将命令以纯文本的方式写入到文件中。

redis协议规定，redis文本是以行来划分，每行以\r\n行结束。每一行都有一个消息头，以表示消息类型。消息头由六种不同的符号表示，其意义如下：

• （+）表示一个正确的状态消息；
• （-）表示一个错误的消息；
• （*）表示消息体总共有多少行，不包括当前行；
• （$）表示下一行消息数据的长度，不包括换行符长度\r\n；
• （空）表示一个消息数据；
• （:）表示返回一个数值。

（2）查看AOF文件

打开appendonly.aof.1.incr.aof文件，可以看到格式如下：

（3）清单文件

打开appendonly.aof.manifest文件，其格式如下：

 tpye有两种类型：b（base file） 和 i（incremental file）。

该文件首先会按照seq序号列举出所有基本文件，基本文件type类型为b，然后再按照seq序号再列举出所有增量文件，增量文件type为i。

对redis启动时的数据恢复，也会按照该文件从上到下依次加载他们中的数据。

3.3 Rewrite 机制

随着使用时间的推移，AOF文件会越来越大。为了防止AOF文件由于太大而占用大量的磁盘空间，降低性能，redis引入了rewrite机制来对AOF文件进行压缩。

（1） rewrite的定义

所谓rewrite其实就是对AOF文件进行重写整理。当rewrite开启后，主进程redis-server创建出一个子进程bgrewriteaof，由该子进程完成rewrite过程。其首先对现有aof文件进行rewrite计算，将计算结果写入到一个临时文件，写入完毕后，再rename该临时文件为原aof文件名，覆盖原有文件。

（2）rewrite计算

rewrite计算也称为rewrite策略。rewrite计算遵循以下策略：

• 读操作命令不写入文件；
• 无效命令不写入文件；
• 过期数据不写入文件；
• 多条命令合并写入文件。

（3）手动开启rewrite

rewrite过程的执行有两种方式。一种是通过bgrewriteaof命令手动开启，一种是通过设置条件自动开启。

手动启动（触发）的命令：

> bgrewriteaof

该命令会使主进程redis-server创建出一个子进程bgrewriteaof，由该子进程完成rewrite过程。而在rewrite期间，redis-server仍是可以对外提供读写服务的 。

（4）自动开启rewrite 

其实就是配置文件中的参数，如下：

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

学习参阅特别声明

【Redis视频从入门到高级】

【https://www.bilibili.com/video/BV1U24y1y7jF?p=11&vd_source=0e347fbc6c2b049143afaa5a15abfc1c】