I am really confused with redis sorted set.
Lets say I have N number of items in the sorted set. I assumed whenever I add an item, it will be sorted in the set.
If I need the lowest score item, I have to use this.
ZRANGE myitem 0 0
But why is the performance is O(log(N)). Should it not be O(1) if it is already sorted & to get the lowest score item?
https://redis.io/commands/zrange
As noted in the documentation, Redis uses a skip list to implement sorted sets. Skip lists have a lookup time of O(log n).
There are other answers here that go into more detail on the use of this data structure, see here for example.
Related
I am looking to have a set which will store elements and that I can get the cardinality after. I noticed I could use the commands SADD or PFADD then use SCARD or PFCOUNT. What is the difference between these two? What are the advantages/disadvantages?
When using SADD, you store data in a SET.
When using PFADD, you store data in an Hyperloglog, which is a different kind of data structure.
A SET is used to store unique values, when you have to access again these values.
An HyperLogLog allows to get an approximate count of the number of unique values in the data added using PFADD. It is useful when you have a great number of distinct values and don't need to get them back. It may be used by example to get the number of unique visitors for a given day for a given page on a high traffic web site (you just add the unique visitor IDs to the HLL).
SADD and SCARD are for "Set".
PFADD and PFCOUNT are for "HyperLogLog".
Advantage of "HyperLogLog":
"HyperLogLog" takes much less memory than "Set".
This video below explains about "HyperLogLog" precisely in about 5 minutes.
https://youtu.be/UAL2dxl1fsE
I've been using sinter which does intersection of unordered integer sets. Is there any faster way of doing intersection given that I wouldn't mind sorting beforehand (or performing any other preprocessing)?
EDIT:
Found some info [here][1]
EDIT2:
Bounty for specific answer: is zinterstore faster than sinter? Benchmarking would be cool too.
Fast answer
In theory the intersection of lists has complexity O(N) where N is the cardinality of the smallest set.
Use SET (SINTER/SINTERSTORE) if have sparsed data / should keep RAM low (O(N*M)) and use BITSET(SETBIT/BITOP) in all other cases (O(N). Like in your edit one info.
BITSET
The Redis BIT key operations have complexity O(N), there N is the cardinality of the smallest key. And the bitops has very best execution speed based on CPU cache (look at bitops.c sources). So this can be the absolute winner in you have not sparsed data or if memory not important for you (here is more about strings in Redis).
ZSET vs SET (zinterstore vs sinter)
Do not use ZSET (zinterstore) is you have a plain list of integers and want to intersect them. Sorted set in Redis is a complex structure their keys stored in ziplist or skiplist internal encodings. The last one used to store sorted scores but keys stored in other structures. In case of ZSET intersection always much complicated with comparison SET:
ZSET intersection: O(N * K) + O(M * log(M)) worst case with N being the smallest input sorted set, K being the number of input sorted sets and M being the number of elements in the resulting sorted set.
SET intersection: O(N * M) worst case where N is the cardinality of the smallest set and M is the number of sets. Actually math-based minimum in theory.
SET uses dict / intset data structures to store data and in your case (unordered integer sets) intset would be used. Intset the most memory save the structure in Redis. And have the best read speed with comparison to ziplist (doubly linked list, more about this data structure internals here).
I'm trying to get the best of both worlds. Using redis sets as well as having determinism. Is this possible?
I'm storing a set using the usual commands:
SADD myset "foo"
SADD myset "bar"
But then if I request an element to process
SRANDMEMBER myset
I'd like the element that is returned to be deterministic. I do not actually care which one it is, but I want the path in my code to be reproducible, or else debugging becomes considerably harder.
Is it possible to set the seed in redis, or use some other technique, to ensure that the three operations above, in sequence, will always return the same element?
No - SRANDMEMBER's behavior isn't controllable.
If you need order, even if for the sake of debugging only, I suggest you use a different approach. Sorted Sets are the natural candidate for this task and behave pretty much as Sets - use 0 score so the members are ordered lexicographically and replace the calls to SRANDMEMBER with ZRANGE, passing to it your own client-generated random rank or a deterministic one.
I wonder why Redis has no command to increment an element in the list?
You can increment a key's value with INCR, you can use HINCRBY to increment an item in the hash set and you can use ZINCRBY to inrement an element of the sorted set. But not in the list.
This puzzles me. Why not?
What was the thinking behind this decision? Lists are "not supposed to be used like this", then why? They work in a very different way from sets? Then what's the big difference?
The big difference is there is no possibility of accessing a given item efficiently in a Redis list. They are implemented as double-linked lists (for big lists) or completely serialized (ziplist optimization, for small lists). By comparison hash and sorted set are implemented using a hash table which allows O(1) amortized complexity for item accesses.
So if such incrementation command would exist for lists, its complexity would be O(n). Not very interesting for just an incrementation.
Note that if you need such feature, you can easily implement it yourself with a server-side Lua script by calling LINDEX and LSET.
We have questions based where in home page we were showing 2 list
Questions by date modified
Question have bigger views and ans count. And in this both listing if question have same views or ans count then sorting is based on date.
Previously i am directly quiring to MySQL database and fetching the values so it's easy.
But each page request hitting to MySQL it's bit expensive then start doing caching.
I started using Redis. Following is the cases when i use redis cache
Issues is On second listing i have to display questions by votes and not answered combine.
How can i stored this type of data in redis to load faster with sorting based by 2 conditions votes with time and ans count with time?
You can use sorted sets in redis. Your view or answer count can be the score. create a key based on timestamp. Sorted set method zrevrangebyscore will give you the correct order.
you can set your member of sorted set as:
'YEAR_MONTH_DATE_HOUR_MINUTE_SECONDS:question_id'
This way if you sort, questions with same score, will be returned in lexicographical order. That way question which came later will be placed higher if you use zrevrangebyscore.
You can create a hash map to map timestamp and question_id. for faster lookup
I asked a similar question, where I also purposed a solution. I want something different but it will do exactly what you want.
Redis zrevrangebyscore, sorting other than lexicographical order