Is there an easy way to create an empty sorted set in Redis? The documentation states
If key does not exist, a new sorted set with the specified members as
sole members is created, like if the sorted set was empty. If the key
exists but does not hold a sorted set, an error is returned.
However, it does not say you can create an empty sorted set. The following doesn't create an empty sorted set:
127.0.0.1:6379> zadd likes:0 1 one
(integer) 1
127.0.0.1:6379> exists likes:0
(integer) 1
127.0.0.1:6379> zcard likes:0
(integer) 1
127.0.0.1:6379> dbsize
(integer) 1
127.0.0.1:6379> zrem likes:0 one
(integer) 1
127.0.0.1:6379> exists likes:0
(integer) 0
The use case is reliably migrating data from another database to Redis, namely Postgres:
When a post is created, create an empty sorted set likes:<postId>
When a like is created, perform a zadd likes:<postId> <createdAt> <userId> if exists likes:<postId. Otherwise, query Postgres for the likes, and store them in likes:<postId>.
Creating an empty sorted set enables the assertion which removes an excessive query to Postgres when the first like is made on that post, yet still supports posts that have not been migrated over to Redis yet. This optimization will save our database 100k+ reads per day.
If the key doesn't exist, it behaves as an empty instance of any type. In your case, you can avoid the call to EXISTS likes:<postId> and assume the key is there, because the resulting behavior will be the same: when you add a member, the list will be created if it didn't exist until that point.
Related
I'm new to nosql databases so forgive my sql mentality but I'm looking to store data that can be 'queried' by one of 2 keys. Here's the structure:
{user_id, business_id, last_seen_ts, first_seen_ts}
where if this were a sql DB I'd use the user_id and business_id as a primary composite key. The sort of querying I'm looking for is a
1.'get all where business_id = x'
2.'get all where user_id = x'
Any tips? I don't think I can make a simple secondary index based on the 2 retrieval types above. I looked into commands like 'zadd' and 'zrange' but there isn't really any sorting involved here.
The use case for Redis for me is to alleviate writes and reads on my SQL database while this program computes (doing its storage in redis) what eventually will be written to the SQL DB.
Note: given the OP's self-proclaimed experience, this answer is intentionally simplified for educational purposes.
(one of) The first thing(s) you need to understand about Redis is that you design the data so every query will be what you're used to think about as access by primary key. It is convenient, in that sense, to imagine Redis' keyspace (the global dictionary) as something like this relational table:
CREATE TABLE redis (
key VARCHAR(512MB) NOT NULL,
value VARCHAR(512MB),
PRIMARY KEY (key)
);
Note: in Redis, value can be more than just a String of course.
Keeping that in mind, and unlike other database models where normalizing data is the practice, you want to have your Redis ready to handle both of your queries efficiently. That means you'll be saving the data twice: once under a primary key that allows searching for businesses by id, and another time that allows querying by user id.
To answer the first query ("'get all where business_id = x'"), you want to have a key for each x that hold the relevant data (in Redis we use the colon, ':', as separator as a matter of convention) - so for x=1 you'd probably call your key business:1, for x=a1b2c3 business:a1b2c3 and so forth.
Each such business:x key could be a Redis Set, where each member represents the rest of the tuple. So, if the data is something like:
{user_id: foo, business_id: bar, last_seen_ts: 987, first_seen_ts: 123}
You'd be storing it with Redis with something like:
SADD business:bar foo
Note: you can use any serialization you want, Set members are just Strings.
With this in place, answering the first query is just a matter of SMEMBERS business:bar (or SSCANing it for larger Sets).
If you've followed through, you already know how to serve the second query. First, use a Set for each user (e.g. user:foo) to which you SADD user:foo bar. Then SMEMBERS/SSCAN and you're almost home.
The last thing you'll need is another set of keys, but this time you can use Hashes. Each such Hash will store the additional information of the tuple, namely the timestamps. We can use a "Primary Key" made up of the bussiness and the user ids (or vice versa) like so:
HMSET foo:bar first 123 last 987
After you've gotten the results from the 1st or 2nd query, you can fetch the contents of the relevant Hashes to complete the query (assuming that the queries return the timestamps as well).
The idiomatic way of doing this in Redis is to use a SET for each type of query you want to do.
In your case you would create:
a hash for each tuple (user_id, business_id, last_seen_ts, first_seen_ts)
a set with a name like user:<user_id>:business:<business_id>, to store the keys of the hashes for this user and this business (you have to add the ID of the hashes with SADD)
Then to get all data for a given user and business, you have to get the SET content with SMEMBERS first, and then to GET every HASH whose ID is in the SET.
It seems that Redis has no any entity corresponding to "table" in relational database.
For instance, I have to store:
(token, user_id)
(cart_id, token, [{product_id, count}])
If it doesn't separate store those two, the get method would search from both, which would cause chaos.
By the way, (cart_id, token, [{product_id, count}]) is a shopping cart, how to design such data structure in redis?
It seems that Redis has no any entity corresponding to "table" in relational database.
Right, because it is not a relational database. It is a data structure server which is very different and requires a different approach to be used well.
Ultimately to use Redis in the way it is intended you need to not think in relational terms, but think of the data structures you use in the code. More specifically, how do you need the data when you want to consume it? That will be the most likely way to store it in Redis.
In this case there are a few options, but the hash method works incredibly well for this one so I'll detail it here.
First, create a hash, call it users:to:tokens. Store as the key in the hash the user id, and the value the token. Next create the inverse, a hash called 'tokens:to:users'. You will probably be wanting both of these - the ability to look one up from the other - and this foundation will provide that.
Next, for your carts. This, too, will be a hash: carts:cart_id. In this hash you have the product_id and the count.
Finally up is your third hash token:to:cart which builds an index of tokens to cart id. I'd go a step further and do user:to:cart to be able to pull carts by user as well.
Now as to whether to store the keynote in the map or not, I tend to go with "no". By just storing the ID you can easily build the Redis cart key and not store the key's full path in the data store as well the saving memory usage.
Indeed, if you can do so use integers for all of your IDs. By using integers you can take advantage of Redis' integer storage optimizations to keep memory usage down. Hashes storing integers are quite efficient and very fast.
If needed you can use Redis to build your IDs. You can use the INCR command to build a counter for each data type such as userid:counter, cartid:counter, and tokenid:counter. As INCR returns the new value you make a single call to increment and get the new ID and get cartid:counter will always give you the largest ID if you wanted to quickly see how many carts have been created. Kinda neat , IMO.
Now, where it gets tricky is if you want to use expiration to automatically expire carts as opposed to leaving them to "lie around" until you want to clean things up. By setting an expiration on the cart hash (which has the product,count mapping) your carts will automatically expire. However, their references will still be hanging out in the token:to:cart hash. Removing that is a simple periodic task which treats over the members of token:to:cart and does an exists check on the cart's key. If it doesn't exist delete it from the hash.
Redis is a key-value storage. From redis.io:
Redis is an open source (BSD licensed), in-memory data structure
store, used as database, cache and message broker. It supports data
structures such as strings, hashes, lists, sets, sorted sets with
range queries, bitmaps, hyperloglogs and geospatial indexes with
radius queries.
So if you want to store two diffetent types (tokens and carts) you will need to store two keys for different datatypes. For example:
127.0.0.1:6379> hset tokens.token_id#123 user user123
(integer) 1
127.0.0.1:6379> hget tokens.token_id#123 user
"user123"
Where tokens is a namespace for tokens only. It is stored as Redis-Hash:
Redis Hashes are maps between string fields and string values, so they
are the perfect data type to represent objects
To store lists I would do the following:
127.0.0.1:6379> hmset carts.cart_1 token token_id#123 cart_contents cart_contents_key1
OK
127.0.0.1:6379> hmget carts.cart_1 token cart_contents
1) "token_id#123"
2) "cart_contents_key1" # cart_contents is a list of receipts.
cart_contents are represented as a Redis-List:
127.0.0.1:6379> rpush cart_contents.cart_contents_key1 receipt_key1
(integer) 1
127.0.0.1:6379> lrange cart_contents.cart_contents_key1 0 -1
1) "receipt_key1"
Receipt is Redis-Hash for a tuple (product_id, count):
127.0.0.1:6379> hmset receipts.receipt_key1 product_id 43 count 2
OK
127.0.0.1:6379> hmget receipts.receipt_key1 product_id count
1) "43" # Your final product id.
2) "2"
But do you really need Redis in this case?
I am using redis for get data from server.
I am creating stream for following which is having structure like this
Sadd user_following_key user1_id, user2_id, user3_id
#data which i want to access is stored like this
zadd user1_id_key data1
zadd user2_id_key data2
#And data1 data2 ... are set which is stored like this
Set data1 some_data
set data2 some_data
I want to access data1 data2 ... and some_data without running loop over user_following_key return data and also if two or more key are same then only one should come.
Put differently, for each user_id in user_following_key, you want to access the relevant user_id key, retrieve the data key name from it and then fetch the value of the data key. In that case, you must loop over the contents of user_following_key (and over every multi-valued key) in this dereference chain and have multiple calls, per depth/length.
Note: in similar cases Lua could be used to reduce network traffic and latency, but since you'll be actually programmatically dealing with key names inside the script (i.e.g. user_id1), it will make the script unsafe in terms of Redis cluster and against best practices.
What is a zset in a redis database.
I have a redis database with some data. In order to get the values
KEYS *apple*
1) "compleet-index:products:apple"
2) "compleet-index:brands:apple"
after to get the key
GET compleet-index:productos:apple
and I had the response
(error) WRONGTYPE Operation against a key holding the wrong kind of value
I get the type
TYPE compleet-index:productos:iphone
zset
When I make
DUMP compleet-index:productos:iphone
I obtain an exas codes.
Short answer: Use ZRANGE compleet-index:products:apple 0 -1 WITHSCORES
ZSET is a short name for Redis Sorted Set, a Redis data type documented here. Each key in a sorted set has multiple values inside, associated with a floating value score.
I have a Set named 'Projects' with many key-value pairs and I want to retrieve one of the values by providing its key. I checked the redis doc but I only found how to retrieve the entire Set. Is it possible to just retrieve one value by providing its key?
Your concept of Set does not match Redis'.
All members of a set in Redis are stored in a single key. Therefore you can't access members individually by a key.
You should use hashes: http://www.redis.io/commands#hash
HSET key field value does what you are looking for.