Im kind of new to redis, so this question might be stupid, but Im trying to wrap around SINTER command here.
The problem is I have multiple SET, and Im trying to create an INTER on them.
When I do it individually its getting me the results, but when I pass multiple args its returning empty. I read documentation on INTER, it tells if any supplied arg is an empty SET it would return nothing, but all my SET is non empty could any one help me please!
# below statement should return {'758', '762', '752'}
127.0.0.1:6379> SINTER Asset:all Asset:id:2275 Asset:id:2280 Asset:id:2269
(empty list or set)
127.0.0.1:6379> SINTER Asset:all Asset:id:2275
"758"
127.0.0.1:6379> SINTER Asset:all Asset:id:2280
"762"
127.0.0.1:6379> SINTER Asset:all Asset:id:2269
"752"
But all other commands like SUNION, SDIFF are working fine.
got the required behaviour by performing SUNION on sets and then SINTER on newly created set
127.0.0.1:6379> SUNION ~unionset Asset:id:2275 Asset:id:2280 Asset:id:2269
127.0.0.1:6379> SINTER Asset:all ~unionset
"758"
"762"
"752"
Related
When I scan entire redis instance using redis-cli --bigkeys following shortened result is returned
-------- summary -------
Sampled 241145 keys in the keyspace!
Total key length in bytes is 13013217 (avg len 53.96)
Biggest string found 'celery-task-meta-52b14b66-b924-4c40-b7dc-7d5b9b633470' has 6510 bytes
**Biggest list found 'celery9' has 156519 items**
Biggest set found '_kombu.binding.celeryev' has 52 members
Biggest hash found 'unacked' has 544 fields
Biggest zset found 'unacked_index' has 550 members
As you can see my biggest list is celery9 with length 156519. I am using only one keyspace
127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=256672,expires=256659,avg_ttl=1701804
But when I connect to redis instance using redis-cli or even with redis connector from python and run following commands
127.0.0.1:6379> get celery9
(nil)
127.0.0.1:6379> llen celery9
(integer) 0
127.0.0.1:6379>
nil or zero is returned as if there was no key celery9.
So the question is, how to get correct length of this key? All others keys are working perfectly
I am trying to run the code from the redis transactions page. Specifically, this part:
WATCH zset
element = ZRANGE zset 0 0
MULTI
ZREM zset element
EXEC
If I try to do it from the cli, line by line, I get this:
localhost:6380> zadd set 1 a
(integer) 1
localhost:6380> WATCH zset
localhost:6380> element = ZRANGE zset 0 0
(error) ERR unknown command 'element'
OK
which probably means I'm doing something wrong? I remember working with lua about 9 years ago, so this doesn't really look like lua either to me.
How does someone run that snippet? Is it only some kind of pseudocode?
As #Dinei said, the example given is pseudocode.
Let's look at it (I added line numbers for us to refer to):
1 WATCH zset
2 element = ZRANGE zset 0 0
3 MULTI
4 ZREM zset element
5 EXEC
The point of the exercise is to solve the race condition that would occur if we only read the key (with ZRANGE, in line 2), and then modify the key (with ZREM in line 4). I assume you understand the problem if we didn't use the "CAS" semantics, so no need to get into it.
As pointed out, redis-cli just gives you the ability to run redis commands and see their replies, but not save values in variables, etc.
So the idea of the example is that in line 2, we are "saving" the result of the "read" operation, into a pseudo-variable element.
Then, in line 4, we are using that value in our "set" operation, and of course lines 1, 3 and 5 are just the "CAS" commands to ensure there is no race condition.
Presumably the actual usage of such commands would be done from a redis client in a programming language that would allow us to save the return value of the ZRANGE and then use it later in the ZREM command.
But if you wanted to run it in redis-cli, you'd see this, where we pretend that our client-side code would have read and saved "a" that was returned from zrange and then passed that value to the zrem command:
127.0.0.1:6379> zadd zset 1 a
(integer) 1
127.0.0.1:6379> watch zset
OK
127.0.0.1:6379> zrange zset 0 0
1) "a"
127.0.0.1:6379> multi
OK
127.0.0.1:6379> zrem zset a
QUEUED
127.0.0.1:6379> exec
1) (integer) 1
127.0.0.1:6379>
Yes, it is some kind of pseudocode.
redis-cli only accepts Redis commands, it is not a full-fledged editor nor supports direct Lua scripting (neither variables like the element variable in the pseudocode).
I remember working with lua about 9 years ago, so this doesn't really look like lua either to me.
This is not Lua, it is pseudocode. Actually, the Redis Transactions page you linked to does not refer to Lua at all (and that's why #Piglet's comment in your post makes sense).
However, it is possible to execute Lua scripts by using Redis' EVAL command.
I am facing a quite strange issue in our deployment
After certain time in operation
I could not add a particular list with a particular keyname or listname to Redis using RPUSH.
Example
RPUSH client-send-process-servername TEST
I am unable to add that key to Redis database.
Anyways the output after executing that command i get
(Integer) 1
But i could not see the list
redis-cli keys *client-send*
Return empty list
However when this problem appears
I am able successfully execute the following command.
RPUSH client-send-process-ser TEST
And
redis-cli keys *client-send*
NOTE: one ASTRICK before n aftet client-send string
Which is not displayed here
It is listing the Queue
"client-send-process-ser"
However strangely i could not add a List with specific key say
client-send-process-servername
Any ideas to debug further.. where to look and what to look.
Redis Server Version is 2.8
I tired enabling debug logs in redis and tried to use redis-monitor command.
However nothing relevant could be found explaining this issue. I am eager to find a solution. Please if some one could help me to pursue further would be a great help.
It seems that this problem is not reproducible. However I've tested it in my local pc and found that it is working fine.
Example:
➜ ~ redis-cli
127.0.0.1:6379> RPUSH client-send-process-servername TEST
(integer) 1
127.0.0.1:6379> keys *client-send*
1) "client-send-process-servername"
127.0.0.1:6379> keys client-send*
1) "client-send-process-servername"
127.0.0.1:6379> RPUSH client-send-process-ser TEST
(integer) 1
127.0.0.1:6379> keys client-send*
1) "client-send-process-ser"
2) "client-send-process-servername"
127.0.0.1:6379>
If there is any option, you can try updating your redis server. current updated versions is 5.0.7 and yours is 2.8.
Calling XREAD after XDEL will not block on the stream, but return immediately. Expected behavior is for XREAD to block again.
127.0.0.1:6379> XADD my-stream * field1 string1
"1554300150697-0"
127.0.0.1:6379> XREAD BLOCK 5000 STREAMS my-stream 1554300150697-0
(nil)
(5.07s)
127.0.0.1:6379> XADD my-stream * field2 string2
"1554300285984-0"
127.0.0.1:6379> XREAD BLOCK 5000 STREAMS my-stream 1554300150697-0
1) 1) "my-stream"
2) 1) 1) "1554300285984-0"
2) 1) "field2"
2) "string2"
127.0.0.1:6379> XDEL my-stream 1554300285984-0
(integer) 1
127.0.0.1:6379> XLEN my-stream
(integer) 1
127.0.0.1:6379> XREAD BLOCK 5000 STREAMS my-stream 1554300150697-0
1) 1) "my-stream"
2) (empty list or set)
127.0.0.1:6379>
As you can see above, the first time XREAD is called it blocks for 5s - expected.
The second call to XREAD returns immediately, giving the new entry - expected.
The third call to XREAD return immediately with (empty list or set) - not expected!
Expected: The command should block for 5s.
I'm not sure if this is a bug or if there's something that I'm missing out. Please advise.
Thank you
It looks like you're running into this known bug.
See the second comment in particular, which points out that the partial fix supplied does not fix the issue you're running into:
It's not an entire fix for the blocking issue, since it only fixes the blocking behaviour for empty streams.
If the stream still contains some entries, but none with a larger ID than requested by the last-received-id parameter, then the request is still answered synchronously with an empty result list.
Looking through 5.0.4's source code I've found a way to (re)set ->last_id member through an undocumented command: XSETID
Although in the source code https://github.com/antirez/redis/blob/f72f4ea311d31f7ce209218a96afb97490971d39/src/t_stream.c#L1837 it says the syntax is XSETID <stream> <groupname> <id>, it's in fact XSETID <stream> <id>(there's an open issue on this one: https://github.com/antirez/redis/issues/5519 but I hope they'd add a new command for groups, like XGROUPSETID, and let this one as it is), which was exactly what I was looking for, so doing:
XSETID my-stream 1554300150697-0
would make:
127.0.0.1:6379> XREAD BLOCK 5000 STREAMS my-stream 1554300150697-0
(nil)
(5.08s)
127.0.0.1:6379>
to work as expected - it will block.
For anyone using this solution(which is more like a workaround in my opinion): Please use it with caution because in a high throughput machine/system/environment Redis could generate/add a new my-stream entry with the same ID as the deleted one 1554300285984-0 leading to possible duplicate data on client's side.
In Redis, keys user* will print all keys starting with user.
For example:
keys user*
1) "user2"
2) "user1"
Now, I want all keys that don't start with user to be printed.
How could I do that?
IMPORTANT: always use SCAN instead of (the evil) KEYS
Redis' pattern matching is somewhat functionally limited (see the implementation of stringmatchlen in util.c) and does not provide that which you seek ATM. That said, consider the following possible routes:
Extend stringmatchlen to match your requirements, possibly submitting it as a PR.
Consider what you're trying to do - fetching a subset of keys is always going to be inefficient unless you index them, consider tracking the names of all non-user keys (i.e.g. in a Redis Set) instead.
If you are really insistent on scanning the entire keyspace and match against negative patterns, one way to accomplish that is with a little bit of Lua magic.
Consider the following dataset and script:
127.0.0.1:6379> dbsize
(integer) 0
127.0.0.1:6379> set user:1 1
OK
127.0.0.1:6379> set use:the:force luke
OK
127.0.0.1:6379> set non:user a
OK
Lua (save this as scanregex.lua):
local re = ARGV[1]
local nt = ARGV[2]
local cur = 0
local rep = {}
local tmp
if not re then
re = ".*"
end
repeat
tmp = redis.call("SCAN", cur, "MATCH", "*")
cur = tonumber(tmp[1])
if tmp[2] then
for k, v in pairs(tmp[2]) do
local fi = v:find(re)
if (fi and not nt) or (not fi and nt) then
rep[#rep+1] = v
end
end
end
until cur == 0
return rep
Output - first time regular matching, 2nd time the complement:
foo#bar:~$ redis-cli --eval scanregex.lua , "^user"
1) "user:1"
foo#bar:~$ redis-cli --eval scanregex.lua , "^user" 1
1) "use:the:force"
2) "non:user"
#Karthikeyan Gopall you nailed it in your comment above and this saved me a bunch of time. Thanks!
Here's how you can use it in various combinations to get what you want:
redis.domain.com:6379[1]> set "hello" "foo"
OK
redis.domain.com:6379[1]> set "hillo" "bar"
OK
redis.domain.com:6379[1]> set "user" "baz"
OK
redis.domain.com:6379[1]> set "zillo" "bash"
OK
redis.domain.com:6379[1]> scan 0
1) "0"
2) 1) "zillo"
2) "hello"
3) "user"
4) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u]*"
1) "0"
2) 1) "zillo"
2) "hello"
3) "hillo"
redis.domain.com:6379[1]> scan 0 match "[^u^z]*"
1) "0"
2) 1) "hello"
2) "hillo"
redis.domain.com:6379[1]> scan 0 match "h[^i]*"
1) "0"
2) 1) "hello"
According to redis keys documentation the command supports glob style patterns, not regular expressions.
and if you look at the documentation, you'll see that the "!" character is not special as opposites to regular expressions.
Here is a simple test I ran in my own db:
redis 127.0.0.1:6379> set a 0
OK
redis 127.0.0.1:6379> set b 1
OK
redis 127.0.0.1:6379> keys *
1) "a"
2) "b"
redis 127.0.0.1:6379> keys !a
(empty list or set) // I expected "b" here
redis 127.0.0.1:6379> keys !b
(empty list or set) // I expected "a" here
redis 127.0.0.1:6379> keys [!b]
1) "b"
redis 127.0.0.1:6379> keys [b]
1) "b"
redis 127.0.0.1:6379> keys [ab]
1) "a"
2) "b"
redis 127.0.0.1:6379> keys ![b]
(empty list or set)
So I just don't think what you are trying to achieve is possible via the keys command.
Besides, the keys command is not very suitable for production environment as it locks your whole redis database.
I would recommend getting all the keys with the scan command, store them locally, and then remove them using LUA
Here's a trick to achieve this with native redis commands (no need for Lua scripts or anything).
If you are able to control the timing of when you insert the new keys (the ones you want to keep, deleting all other stuff like in your question), you can:
Before setting the new keys, set the expiration to all existing keys (by pattern or everything) to expire right now (see how)
Load the new keys
Redis will automatically delete all the older keys and you will be left just with the new ones you want.
You also can print all keys and pass it to grep. For example:
redis-cli -a <password> keys "*" | grep -v "user"