线上问题：redis 内存使用率 95%，排查bigkey

一、问题现象

　　某一天，突然发现用户没权限，查看redis的某个用户key被清理了，再查看redis内存使用率，如图

 

 

 

 二、排查是哪个key占用内存大

root@redis-release-master-0:/opt/bitnami/redis/bin# redis-cli -h 127.0.0.1 -a password --bigkeys
Warning: Using a password with '-a' option on the command line interface may not be safe.

# Scanning the entire keyspace to find biggest keys as well as
# average sizes per key type.  You can use -i 0.1 to sleep 0.1 sec
# per 100 SCAN commands (not usually needed).

[00.00%] Biggest list   found so far 'uname_to_access:superuser:13424346650' with 1 items
[00.00%] Biggest string found so far 'auth:ywHRuX_lWAxXHl9tdNB+MpHmsl3gAY250ssOiYgv6PDCb/ofUo=' with 51570 bytes
[00.00%] Biggest string found so far 'access:eZqJpv_GhvkrwdIoSF38TzUsHRS/Ehxg0BLC7XVWhiIpevmhVQ=' with 51934 bytes
[00.12%] Biggest string found so far 'access:iywFsE_wN8d5gqHX0dO2qgXiEPuroaFUhC5dOoJfqmc9xcuuTY=' with 51936 bytes
[00.26%] Biggest string found so far 'access:LZmIZp_lAcs9g69dhzOTViUmd6Qlaxtqqkn1lwyx6IMMHco14U=' with 52824 bytes
[01.03%] Biggest list   found so far 'uname_to_access:superuser:15897829202' with 2 items
[01.54%] Biggest string found so far 'AIaZdE_w9rrkIzssV91fmYrL/ih2RZ2UVECyWWDvmxNeQduVGY=' with 64949 bytes
[04.38%] Biggest string found so far 'BfGnlB_V9dGHqQhXVQW6uACiRw9pCraL3hmOF+gBHnnJ4dOFnM=' with 68979 bytes
[07.23%] Biggest list   found so far 'uname_to_access:superuser:13669856178' with 3 items
[08.26%] Biggest list   found so far 'uname_to_access:superuser:13537578165' with 4 items
[12.69%] Biggest string found so far 'access:bpFISU_5hX8dM6Xapmwgy0RvOvOdRTnKvjIJtGfQfAt2vD+480=' with 81687 bytes
[13.86%] Biggest list   found so far 'uname_to_access:superuser:yunwei01' with 20 items
[17.31%] Biggest zset   found so far 'ifaascloud_cmp_monitor_run_rank_middleware' with 8 members
[18.78%] Biggest zset   found so far 'ifaascloud_cmp_monitor_run_rank_platform' with 12 members
[30.13%] Biggest list   found so far 'uname_to_access:superuser:rhgenetech' with 1149 items
[44.41%] Biggest list   found so far 'client_id_to_access:superuser' with 11316 items
[66.43%] Biggest hash   found so far 'ifaascloud_cmp_monitor_resource_overview' with 3 fields
[69.27%] Biggest list   found so far 'uname_to_access:superuser:ub' with 17849 items
[82.58%] Biggest string found so far 'ifaas-hotel-robot-platform:buildingData' with 817326 bytes
[87.52%] Biggest hash   found so far 'ifaascloud_cmp_alert_prometheus_alertrules' with 101 fields

-------- summary -------

Sampled 6107 keys in the keyspace!
Total key length in bytes is 277942 (avg len 45.51)

Biggest string found 'ifaas-hotel-robot-platform:buildingData' has 817326 bytes
Biggest   list found 'uname_to_access:superuser:ubtech' has 17849 items
Biggest   hash found 'ifaascloud_cmp_alert_prometheus_alertrules' has 101 fields
Biggest   zset found 'ifaascloud_cmp_monitor_run_rank_platform' has 12 members

5114 strings with 206443381 bytes (83.74% of keys, avg size 40368.28)
988 lists with 31845 items (16.18% of keys, avg size 32.23)
0 sets with 0 members (00.00% of keys, avg size 0.00)
2 hashs with 104 fields (00.03% of keys, avg size 52.00)
3 zsets with 26 members (00.05% of keys, avg size 8.67)
0 streams with 0 entries (00.00% of keys, avg size 0.00)
root@redis-release-master-0:/opt/bitnami/redis/bin#

　　上图红色字体，发现这个list 有17849 items，每个item有50k，那就占用了900M多，加上另一个 11316 items 共占用1.4G内存。

　　解决方案：查看代码，排查uname_to_access:superuser:ub的使用方式即可

 

技术参考：https://blog.csdn.net/weixin_36642528/article/details/111238496