stackexchange.redis - Is there any recommended value of COUNT for SCAN / HSCAN command in REDIS?

Question

I have understood the meaning of COUNT in the case of REDIS SCAN. But, what is the ideal value for REDIS COUNT ?

Answer 1

The default value is 10. It means the command will bring back more or less 10 keys, could be less if the keys are sparsely populated in the hash slots, or filtered out by the MATCH pattern. It could be more if some keys are sharing a hash slot. Anyhow, the work performed is proportional to the COUNT parameter.

Redis is single-threaded. One of the reasons SCAN was introduced is to allow going through all the keys without blocking the server for a long time, by going a few steps at a time.

And that's precisely the criteria to decide what's a good number. For how long are you willing to block your Redis server by running a SCAN command. The higher the COUNT, the longer the block.

Let's use a Lua script to get a sense of the COUNT impact. Use it on your environment to get the results based on your server resources.

The Lua script:

local t0 = redis.call('TIME')
local res = redis.call('SCAN', ARGV[1], 'COUNT', ARGV[2])
local t1 = redis.call('TIME')
local micros = (t1[1]-t0[1])*1000000 + t1[2]-t0[2]
table.insert(res,'Time taken: '..micros..' microseconds')
table.insert(res,'T0: '..t0[1]..string.format('%06d', t0[2]))
table.insert(res,'T1: '..t1[1]..string.format('%06d', t1[2]))
return res

Here we use Redis TIME command. The command returns:

• unix time in seconds
• microseconds

A few runs in my machine, with 1 million keys:

COUNT    TIME IN MICROSECONDS
   10            37
  100           257
 1000          1685
10000         14438

Note these times don't include the time used to read from the socket and to buffer and send the response. Actual times will be larger. The time it takes once is out of Redis, including time traveling the network is not time your Redis server is blocked though.

This is how I called the Lua script and the results:

> EVAL "local t0 = redis.call('TIME') 
 local res = redis.call('SCAN', ARGV[1], 'COUNT', ARGV[2]) 
 local t1 = redis.call('TIME') 
 local micros = (t1[1]-t0[1])*1000000 + t1[2]-t0[2] 
 table.insert(res,'Time taken: '..micros..' microseconds') 
 table.insert(res,'T0: '..t0[1]..string.format('%06d', t0[2])) 
 table.insert(res,'T1: '..t1[1]..string.format('%06d', t1[2])) 
 return res" 0 0 5
1) "851968"
2) 1) "key:560785"
   2) "key:114611"
   3) "key:970983"
   4) "key:626494"
   5) "key:23865"
3) "Time taken: 36 microseconds"
4) "T0: 1580816056349600"
5) "T1: 1580816056349636"