Redis - SET overwriting other types - redis

The following code example will be done/written via a Python REPL and the redis-cli.
Redis server v=2.8.4
Background: Storing a long-running key (hash) in a redis key-value store, then attempting to store another key (with the same name, but different type - string) in the same key-value store.
Code will come first, then question(s):
>>> import redis
>>> db = redis.Redis(
... host='127.0.0.1',
... port=6379,
... password='',
... db=3)
>>> db.hset("123456", "field1", True)
1
>>> db.type("123456")
b'hash'
>>> db.hgetall("123456")
{b'field1': b'True'}
>>> db.set("123456", "new-value")
True
>>> db.type("123456")
b'string'
>>> db.get("123456")
b'new-value'
You will first notice that the SET option overwrites the HSET. Now when I try to overwrite the SET with:
>>> db.lset("123456", "list1", "list1value")
Traceback (most recent call last):
...
redis.exceptions.ResponseError: WRONGTYPE Operation against a key holding the wrong kind of value
WRONGTYPE Operation against a key holding the wrong kind of value
OR replacing SET with the same HSET:
>>> db.hset("123456", "field1", True)
Traceback (most recent call last):
...
redis.exceptions.ResponseError: WRONGTYPE Operation against a key holding the wrong kind of value
WRONGTYPE Operation against a key holding the wrong kind of value
In order to make sure this isn't a redis-py flaw, I tested in the redis-cli:
127.0.0.1:6379> HSET 12345 "field" "value1"
(integer) 0
127.0.0.1:6379> TYPE 12345
hash
127.0.0.1:6379> SET 12345 "newvalue"
OK
127.0.0.1:6379> TYPE 12345
string
127.0.0.1:6379> HSET 12345 "field" "value1"
(error) WRONGTYPE Operation against a key holding the wrong kind of value
Questions:
1) Is this a flaw in Redis or is this actually how it is supposed to work?
2) If this is "how it is supposed to work", why can I not overwrite the SET type with others?
** Edit: As the person answering the question did not understand 3) .. I am editing it
3) Which other type, besides SET, can I use for storing a STRING in the structure (KEY, VALUE) where I can also have a HASH as (KEY, FIELD, VALUE) - where the key is same but of different TYPE(s)?
Eg. I want to do:
127.0.0.1:6379> HSET 12345 "field" "value1"
(integer) 0
127.0.0.1:6379> TYPE 12345
hash
127.0.0.1:6379> SOME-COMMAND 12345 "newvalue"
OK
So that I have 1 hash and 1 "other" type of the same "key" 12345

This is the designed behavior, the second sentence in SET's documentation is.
If key already holds a value, it is overwritten, regardless of its type.
No, only SET has that power, other commands will error when presented with the wrong type of value.
Sorry, not following you.

Related

TYPE option does not work for REDIS SCAN command

There is a command in Redis - SCAN. It has an option TYPE which return objects that match a given type. When I try to run the set of commands which is provided in the example https://redis.io/commands/scan#the-type-option I get an error ERR syntax error when I run the last command SCAN 0 TYPE zset.
I have prepared objects with the list and zset types, but neither of them works, I always get an exception. Even if I add something on my own, it does not work.
My question is next. Does SCAN actually support TYPE option? I found this issue https://github.com/antirez/redis/issues/3323, but it's not closed and on Redis docs there are such details
Redis version:
redis> INFO
# Server
redis_version:5.0.5
redis> RPUSH list_object "list_element"
redis> TYPE list_object
list
redis> ZADD zset_object 1 "zset_element"
redis> TYPE zset_object
zset
redis> SCAN 0 TYPE zset
ERR syntax error
redis> SCAN 0 type list
ERR syntax error
The code for TYPE option is still in the unstable branch, and haven't been release to the latest version of Redis. So far, you cannot use that command. You have to wait for the new release to support this feature, or take the risk to use the unstable branch.
However, you can also achieve this goal on the client side:
Use the SCAN command to iterate the key space
For each key, call the type command to do the filter on the client side.
In order to make this operation faster, you can wrap the logic into a Lua script.
UPDATE
Redis 6.0 already supports this feature.

In Redis, is it possible to sort a member across multiple Sorted Sets?

I'm tracking members in multiple Sorted Sets in Redis as a way to do multi-column indexing on the members. As an example, let's say I have two Sorted Sets, lastseen (which is epoch time) and points, and I store usernames as members in these Sorted Sets.
I'm wanting to first sort by lastseen so I can get the users seen within the last day or month, then I'm wanting to sort the resulting members by points so I effectively have the members seen within the last day or month sorted by points.
This would be easy if I could store the result of a call to ZREVRANGEBYSCORE to a new Sorted Set (we'll call the new Sorted Set temp), because then I could sort lastseen with limits, store the result to temp, use ZINTERSTORE against temp and points with a weight of zero for out (stored to result), and finally use ZREVRANGEBYSCORE again on result. However, there's no built-in way in Redis to store the result of ZRANGE to a new Sorted Set.
I looked into using the solution posted here, and while it does seem to order the results correctly, the resulting scores in the Sorted Set can no longer be used to accurately limit results based on time (ie. only want ones within the last day).
For example:
redis> ZADD lastseen 12345 "foo"
redis> ZADD lastseen 12350 "bar"
redis> ZADD lastseen 12355 "sucka"
redis> ZADD points 5 "foo"
redis> ZADD points 3 "bar"
redis> ZADD points 9 "sucka"
What I'd like to end up with, assuming my time window is between 12349 and 12356, is the list of members ['sucka', 'bar'].
The solutions I can think of are:
1) Your wish was to ZREVRANGEBYSCORE and somehow save the temporary result. Instead you could copy the zset (which can be done with a ZINTERSTORE with only one set as an argument), then do a ZREMRANGEBYSCORE on the new copy to get rid of the times you're not interested in, then do the final ZINTERSTORE.
2) Do it in a loop on the client, as Eli suggested.
3) Do the same thing in a Lua script.
These are all potentially expensive operations, so what's going to work best will depend on your data and use case. Without knowing more, I would personally lean towards the Lua solution.
For queries that get this complex, you want to supplement Redis' built-in commands with another processing language. The easiest way to do that is calling from within whatever your backend language is and using that to process. An example in Python using redis-py is:
import redis
finish_time, start_time = 12356, 12349
r = redis.Redis(host='localhost', port=6379, db=0, password='some_pass')
entries_in_time_frame = r.zrevrangebyscore('lastseen', finish_time, start_time)
p = r.pipeline()
for entry in entries_in_time_frame:
p.zscore('points', entry)
scores = zip(entries_in_time_frame, p.execute())
sorted_entries = [tup[0] for tup in sorted(scores, key=lambda tup: tup[1])]
>>> ['sucka', 'bar']
Note the pipeline, so we're only ever sending two calls to the Redis server, so network latency shouldn't slow us down much. If you need to go even faster (perhaps if what's returned by the first ZREVRANGEBYSCORE is very long), you can rewrite the same logic as above as a Lua script. Here's a working example (note my lua is rusty, so this can be optimized):
local start_time = ARGV[1]
local finish_time = ARGV[2]
local entries_in_time_frame = redis.call('ZREVRANGEBYSCORE', KEYS[1], finish_time, start_time)
local sort_function = function (k0, k1)
local s0 = redis.call('ZSCORE', KEYS[2], k0)
local s1 = redis.call('ZSCORE', KEYS[2], k1)
return (s0 > s1)
end
table.sort(entries_in_time_frame, sort_function)
return entries_in_time_frame
You can call it like so:
redis-cli -a some_pass EVAL "$(cat script.lua)" 2 lastseen points 12349 12356
Returning:
1) "bar"
2) "foo"

how to get values of multiple keys value at a time in redis?

how to get multiple keys' values in redis? for example, keys are x, y, and z. they have their own values a, b, and c respectively. I want to get all values at a time for all such keys.
The native protocol supports the MGET method as shown in the documentation:
redis> SET key2 "World"
OK
redis> MGET key1 key2 nonexisting
1) "Hello"
2) "World"
3) (nil)
redis>
This method allows you to retrieve the values of multiple keys in a single roundtrip to the server. Depending on the actual platform you are using and the client code, the method might be called differently in your client library. For example if you are using .NET and the ServiceStack.Redis client you could use the GetValues method on the IRedisClient:
List<string> GetValues(List<string> keys);

Add a value to a Redis list only if it doesn't already exist in the list?

I'm trying to add a value to a list but only if it hasn't been added yet.
Is there a command to do this or is there a way to test for the existence of a value within a list?
Thanks!
I need to do the same.
I think about to remove the element from the list and then add it again. If the element is not in the list, redis will return 0, so there is no error
lrem mylist 0 myitem
rpush mylist myitem
As Tommaso Barbugli mentioned you should use a set instead a list if you need only unique values.
see REDIS documentation SADD
redis> SADD myset "Hello"
(integer) 1
redis> SADD myset "World"
(integer) 1
redis> SADD myset "World"
(integer) 0
redis> SMEMBERS myset
1) "World"
2) "Hello"
If you want to check the presence of a value in the set you may use SISMEMBER
redis> SADD myset "one"
(integer) 1
redis> SISMEMBER myset "one"
(integer) 1
redis> SISMEMBER myset "two"
(integer) 0
It looks like you need a set or a sorted set.
Sets have O(1) membership test and enforced uniqueness.
If you can't use the SETs (in case you want to achieve some blocking POP/PUSH list features) you can use a simple script:
script load 'local exists = false; for idx=1, redis.call("LLEN",KEYS[1]) do if (redis.call("LINDEX", KEYS[1], idx) == ARGV[1]) then exists = true; break; end end; if (not exists) then redis.call("RPUSH", KEYS[1], ARGV[1]) end; return not exists or 0'
This will return the SHA code of the script you've added.
Just call then:
evalsha 3e31bb17571f819bea95ca5eb5747a373c575ad9 1 test-list myval
where
3e31bb17571f819bea95ca5eb5747a373c575ad9 (the SHA code of the script you added)
1 — is number of parameters (1 is constant for this function)
test-list — the name of your list
myval - the value you need to add
it returns 1 if the new item was added or 0 if it was already in the list.
Such feature is available in set using hexistshexists command in redis.
Checking a list to see if a member exists within it is O(n), which can get quite expensive for big lists and is definitely not ideal. That said, everyone else seems to be giving you alternatives. I'll just tell you how to do what you're asking to do, and assume you have good reasons for doing it the way you're doing it. I'll do it in Python, assuming you have a connection to Redis called r, some list called some_list and some new item to add called new_item:
lst = r.lrange(list_name, -float('Inf'), float('Inf'))
if new_item not in lst:
r.rpush(list_name, new_item)
I encountered this problem while adding to a task worker queue, because I wanted to avoid adding many duplicate tasks. Using a Redis set (as many people are suggesting) would be nice, but Redis sets don't have a "blocking pop" like BRPOPLPUSH, so they're not good for task queues.
So, here's my slightly non-ideal solution (in Python):
def pushOnlyNewItemsToList(redis, list_name, items):
""" Adds only the items that aren't already in the list.
Though if run simultaneously in multiple threads, there's still a tiny chance of adding duplicate items.
O(n) on the size of the list."""
existing_items = set(redis.lrange(list_name,0,-1))
new_items = set(items).difference(existing_items)
if new_items:
redis.lpush(list_name, *new_items)
Note the caveats in the docstring.
If you need to truly guarantee no duplicates, the alternative is to run LREM, LPUSH inside a Redis pipeline, as in 0xAffe's answer. That approach causes less network traffic, but has the downside of reordering the list. It's probably the best general answer if you don't care about list order.

Looking for a copy set command (or alternative) in Redis

I'm a newcomer to Redis and I'm looking for some specific help around sets. To give some background: I'm building a web-app which consists of a large number of card decks which each have a set of individual cards with unique ids. I want users to have a set of 5 cards drawn for them at random from a specific deck.
My plan is to have all of the card ids of a given deck stored as a set in Redis; then I want to use the SPOP function to draw individual cards and remove them from the set so that they are not drawn again within that hand. It would seem to make sense to do this by copying the deck's 'master set' of card IDs into a new temporary set, performing the popping on the copy and then deleting the copied set when I'm done.
But: I can't find any Redis function to command a set copy - the closest thing I can see would be to also create an empty set and then 'join' the empty set and the 'master copy' of the set into a new (if temporary) set with SUNIONSTORE, but that seems hacky. I suppose an alternative would be to copy the set items out into my 'host language' (node.js) and then manually insert the items back into a new Redis set, but this also seems clunky. There's probably a better third option that I haven't even thought of.
Am I doing something wrong - am I not 'getting' Redis, or is the command-set still a bit immature?
redis> sadd mydeck 1
(integer) 1
redis> sadd mydeck 2
(integer) 1
redis> sadd mydeck 3
(integer) 1
redis> smembers mydeck
1) "1"
2) "2"
3) "3"
redis> sunionstore tempdeck mydeck
(integer) 3
redis> smembers mydeck
1) "1"
2) "2"
3) "3"
redis> smembers tempdeck
1) "1"
2) "2"
3) "3"
Have fun with Redis!
Salvatore