Is there anyway to distinguish which database an invalidation applies to?
example:
Tracking socket:
CLIENT ID // 77
PSUBSCRIBE __redis__:*
Main socket:
CLIENT TRACKING on REDIRECT 77 OPTIN
SELECT 1
SET MYKEY VALUE1
CLIENT CACHING YES
GET MYKEY //VALUE1
SELECT 2
SET MYKEY VALUE2
GET MYKEY //VALUE2
SELECT 1
GET MYKEY //VALUE1
The issue i have is that the tracking socket receives a: redis:invalidate 1) MYKEY when MYKEY is set in database 2. However the key I wanted to track is in database 1.
Short of redesigning the application to avoid key collisions across databases or creating a socket per database+tracking, how can i use tracking in a meaningful way?
Edit: Redis 6.0.8 stand alone install
Found the answer in Redis documentation:
"There is a single keys namespace, not divided by database numbers. So if a client is caching the key foo in database 2, and some other client changes the value of the key foo in database 3, an invalidation message will still be sent. This way we can ignore database numbers reducing both the memory usage and the implementation complexity."
Say Two users are updating/putting the key in same hash , how read/write locks are acquired here ?
hmset user:A:address city NY // First user
hmset user:A:address city SEATLE // Second user
For Example in above case :-
First user will acquire the lock on complete hashmap user:A:address and Second user will be blocked till first user is done ?
What if users are working on separate keys under same hashmap
hmset user:A:address county test_county // First user
hmset user:A:address zip 01001 // Second user
Redis processes these commands in a single thread, and doesn't need any lock.
It doesn't matter whether the two users are setting the same field of the hash or different fields of the hash, these commands are processed one-by-one. When Redis is processing a command, it blocks other commands, since Redis is (mostly) single-threaded.
It seems that Redis has no any entity corresponding to "table" in relational database.
For instance, I have to store:
(token, user_id)
(cart_id, token, [{product_id, count}])
If it doesn't separate store those two, the get method would search from both, which would cause chaos.
By the way, (cart_id, token, [{product_id, count}]) is a shopping cart, how to design such data structure in redis?
It seems that Redis has no any entity corresponding to "table" in relational database.
Right, because it is not a relational database. It is a data structure server which is very different and requires a different approach to be used well.
Ultimately to use Redis in the way it is intended you need to not think in relational terms, but think of the data structures you use in the code. More specifically, how do you need the data when you want to consume it? That will be the most likely way to store it in Redis.
In this case there are a few options, but the hash method works incredibly well for this one so I'll detail it here.
First, create a hash, call it users:to:tokens. Store as the key in the hash the user id, and the value the token. Next create the inverse, a hash called 'tokens:to:users'. You will probably be wanting both of these - the ability to look one up from the other - and this foundation will provide that.
Next, for your carts. This, too, will be a hash: carts:cart_id. In this hash you have the product_id and the count.
Finally up is your third hash token:to:cart which builds an index of tokens to cart id. I'd go a step further and do user:to:cart to be able to pull carts by user as well.
Now as to whether to store the keynote in the map or not, I tend to go with "no". By just storing the ID you can easily build the Redis cart key and not store the key's full path in the data store as well the saving memory usage.
Indeed, if you can do so use integers for all of your IDs. By using integers you can take advantage of Redis' integer storage optimizations to keep memory usage down. Hashes storing integers are quite efficient and very fast.
If needed you can use Redis to build your IDs. You can use the INCR command to build a counter for each data type such as userid:counter, cartid:counter, and tokenid:counter. As INCR returns the new value you make a single call to increment and get the new ID and get cartid:counter will always give you the largest ID if you wanted to quickly see how many carts have been created. Kinda neat , IMO.
Now, where it gets tricky is if you want to use expiration to automatically expire carts as opposed to leaving them to "lie around" until you want to clean things up. By setting an expiration on the cart hash (which has the product,count mapping) your carts will automatically expire. However, their references will still be hanging out in the token:to:cart hash. Removing that is a simple periodic task which treats over the members of token:to:cart and does an exists check on the cart's key. If it doesn't exist delete it from the hash.
Redis is a key-value storage. From redis.io:
Redis is an open source (BSD licensed), in-memory data structure
store, used as database, cache and message broker. It supports data
structures such as strings, hashes, lists, sets, sorted sets with
range queries, bitmaps, hyperloglogs and geospatial indexes with
radius queries.
So if you want to store two diffetent types (tokens and carts) you will need to store two keys for different datatypes. For example:
127.0.0.1:6379> hset tokens.token_id#123 user user123
(integer) 1
127.0.0.1:6379> hget tokens.token_id#123 user
"user123"
Where tokens is a namespace for tokens only. It is stored as Redis-Hash:
Redis Hashes are maps between string fields and string values, so they
are the perfect data type to represent objects
To store lists I would do the following:
127.0.0.1:6379> hmset carts.cart_1 token token_id#123 cart_contents cart_contents_key1
OK
127.0.0.1:6379> hmget carts.cart_1 token cart_contents
1) "token_id#123"
2) "cart_contents_key1" # cart_contents is a list of receipts.
cart_contents are represented as a Redis-List:
127.0.0.1:6379> rpush cart_contents.cart_contents_key1 receipt_key1
(integer) 1
127.0.0.1:6379> lrange cart_contents.cart_contents_key1 0 -1
1) "receipt_key1"
Receipt is Redis-Hash for a tuple (product_id, count):
127.0.0.1:6379> hmset receipts.receipt_key1 product_id 43 count 2
OK
127.0.0.1:6379> hmget receipts.receipt_key1 product_id count
1) "43" # Your final product id.
2) "2"
But do you really need Redis in this case?
I am using the new geospatial features on Redis.
I know that behind the scene it's using ZSET.
I am adding new entries this way:
GEOADD" "report-geo-set" "4.78335244" "32.07223969" "jossef"
How could I add an expire to a specific records(in my case: "jossef")
on my set?
If the API doesnt provide it is there any workaround for this?
Thanks,
ray.
Regrettably no - Redis expires entire keys and not the values in their respective data structures. Geo Hashes are implemented on top Sorted Sets and expiration of individual members isn't supported.
What you could do is maintain an additional Sorted Set and for each member in it store the expiration timestamp as score. Then, periodically, fetch the members that need to be expired from it based on ZRANGEBYSCORE and "manually" ZREM the respective members from your Geo Hash.
I use redis to store user sessions by a guid i generate when the log in. I use that as the key in their session object.
If i was to lock a user's account I currently have to go through all sessions and check if they are the user's then delete them.
Is there a way to also query by the user id? Should I be using the sorted set insured of just standard key value?
Going through all keys is probably not the best idea. What you could do is store every user sessions' guids in another key - the set data type seems to be the best choice for that - and add/remove from it as the user opens/closes a session. So, when a user opens a new session you will:
SET session:<guid> <session_object>
SADD user_sessions:<user_id> <session_guid>
and when the session is closed, you'll do:
DEL session:<guid>
SREM user_sessions:<user_id> <session_guid>
To find which session guids are a user's, e.g. for an account lock down, do:
SMEMBERS sessions:<user_id>