how to cleanup keys without exipiration in Redis

2018-10-24 12:02:18

在排查redis内存一直增长问题的时候，我们首先想到的是如何找出有多少key没有设置失效时间

我们可以通过 info keyspace 命令查看：

1
2
3
127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=473688,expires=1645,avg_ttl=1625894

keys是总共的key, expires是要失效的key,ave_ttl是这些key的平均失效时间，单位是ms

注意，我们不能再生产环境使用keys命令，如果没有开启RDB,需要通过执行bgsave 来获得rdb文件，之后拷贝到测试环境

我们可以使用docker来加载

1
2
3
docker run --name redis_dump -d -v `pwd`:/data -p 6379 redis:3.2

docker exec -ti redis_dump bash

现在我们来统计未设置失效时间的key吧

1
2
3
4
5
6
redis-cli keys "*" > keys
cat keys | xargs -n 1 -L 1 redis-cli ttl > ttl
paste -d " " keys ttl | grep .*-1$ | cut -d " " -f 1 > without_ttl

# We can create a script for deleting the keys 
cat without_ttl | awk '{print "redis-cli del "$1}' > redis.sh

再来一个大红包吧，怎么找出最大的key呢

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
[root@localhost ~]# redis-cli --bigkeys

# Scanning the entire keyspace to find biggest keys as well as
# average sizes per key type.  You can use -i 0.1 to sleep 0.1 sec
# per 100 SCAN commands (not usually needed).

[00.00%] Biggest string found so far 'courseinfo_201703827_2018-10-29' with 14189 bytes
[00.00%] Biggest string found so far 'courseinfo_3160205116_2017-09-04' with 16717 bytes
[00.01%] Biggest string found so far 'courseinfo_201707708_2018-12-03' with 20835 bytes
[00.02%] Biggest string found so far 'courseinfo_3170204119_2017-10-02' with 23037 bytes
[00.07%] Biggest string found so far 'courseinfo_3160204209_2017-09-18' with 29201 bytes
[00.38%] Biggest string found so far 'courseinfo_201504330342_2018-05-21' with 138565 bytes
[00.77%] Biggest string found so far 'courseinfo_201606060919_2018-03-26' with 201381 bytes
[00.98%] Biggest string found so far 'courseinfo_201501310518_2018-05-14' with 215428 bytes
[01.07%] Biggest string found so far 'courseinfo_201706061709_2018-06-11' with 223421 bytes
[04.43%] Biggest string found so far 'courseinfo_201706061633_2018-03-05' with 249319 bytes
[05.33%] Biggest string found so far 'courseinfo_201706062917_2018-03-05' with 278392 bytes
[06.03%] Biggest string found so far 'courseinfo_201707030224_2018-03-19' with 281098 bytes
[19.89%] Biggest string found so far 'courseinfo_201706062917_2018-03-19' with 296014 bytes

-------- summary -------

Sampled 472908 keys in the keyspace!
Total key length in bytes is 17874174 (avg len 37.80)

Biggest string found 'courseinfo_201706062917_2018-03-19' has 296014 bytes

472908 strings with 2259279653 bytes (100.00% of keys, avg size 4777.42)
0 lists with 0 items (00.00% of keys, avg size 0.00)
0 sets with 0 members (00.00% of keys, avg size 0.00)
0 hashs with 0 fields (00.00% of keys, avg size 0.00)
0 zsets with 0 members (00.00% of keys, avg size 0.00)

很快我们就会发现我们的问题了，courseinfo_201706062917_2018-03-19 就是我们要找的

ref
Found and cleanup keys without expiration in Redis

您的鼓励是我写作最大的动力

俗话说，投资效率是最好的投资。 如果您感觉我的文章质量不错，读后收获很大，预计能为您提高 10% 的工作效率，不妨小额捐助我一下，让我有动力继续写出更多好文章。