What Redis data type fit the most for following example -

i have following scenario:

1. fetch array of numbers (from redis) conditionally
2. for each number async stuff (fetch db based on number)
3. for each thing in result set db async stuff

periodically repeat 1. 2. 3. because new numbers added redis structure.those numbers represent unix timestamp in milliseconds out of box numbers sorted in time of addition

conditionally means fetch unix timestamp redis less or equal current unix timestamp in milliseconds(date.now())

question redis data type fit use case having in mind code scaled n instances, n instances share access single redis instance. equally share load each instance read example first(oldest) 5 numbers redis. numbers unique (adding same number should fail silently) redis set seems choice reading m first elements redis set seems impossible.

to prevent 2 different instance of code read same numbers redis read operation should atomic, should read numbers , delete them. if async operation fail on specific number (steps 2. , 3.), numbers should added again redis handled again. should re-added head not end handled again possible. far know sadd push tail.

smembers key read everything, looks hammer me. need include application logic first 5 check less or equal date.now() , delete , wrap somehow in single transaction. besides set cardinality can huge.
sscan sounds interesting don't have clue how works in "scaled" environment described above. besides that, per redis docs: scan family of commands offer limited guarantees returned elements since collection incrementally iterate can change during iteration process. described above collection changed frequently

a more appropriate data structure sorted set - members have float score suitable storing timestamp , can perform range searches (i.e. less or equal given value).

the relevant starting points zadd, zrangebyscore , zremrangebyscore commands.

to ensure atomicity when reading , removing members, can choose between the following options: redis transactions, redis lua script , in next version (v4) redis module.

transactions

using transactions means doing following code running on instances:

multi zrangebyscore <keyname> -inf <now-timestamp> zremrangebyscore <keyname> -inf <now-timestamp> exec 

where <keyname> key's name , <now-timestamp> current time.

lua script

a lua script can cached , runs embedded in server, in cases preferable approach. best approach short snippets of atomic logic if need flow control (remember multi transaction returns values after execution). such script follows:

local r = redis.call('zrangebyscore', keys[1], '-inf', argv[1]) redis.call('zremrangebyscore', keys[1], '-inf', argv[1]) return r 

to run this, first cache using script load , call evalsha so:

evalsha <script-sha> 1 <key-name> <now-timestamp> 

where <script-sha> sha1 of script returned script load.

redis modules

in near future, once v4 ga you'll able write , use modules. once becomes reality, you'll able use module we've made provides zpop command , extended cover use case well.