How can i union 'n' number of sorted sets by using * in Keyname (key-1-1, key-1-,2, key-1-3,..key-1-n) of ZUNIONSTORE command. Is there any other similar option as i have around 55k sortedsets to be joined.
Set operations require explicit key names, not patterns. The only way to do it is the specify each of your 55K keys in the command.
That said, if you're unioning 55K sets, you're probably using it wrong.
Related
Is there a way to search by a parent part of the key in Redis?
For example: X:Y = [1,2] and X:Z = [4,6]
Both keys have a key subpart of X.
Can I run some sort of operation to get X = [1,2,4,6]?
Redis has no built-in ability to do that, but you can use it to build it.
Yes, you can search for keys in Redis according to their name, but it would be inefficient in terms of performance. Refer to SCAN for more information.
A more performant way is to index your keys, so searching is done in sub-linear time. Refer to Secondary Indexing with Redis for some pointers.
Once you've retrieved the names of your keys, it appears that you want the union of their values. One candidate data type that supports this functionality is the Redis Set via the SUNION command.
An alternative approach entirely to scanning/indexing, sets and unions is to use a single data type for all the "keys" sharing the same prefix ("X"). The Redis Hash can do that for you, and while it doesn't offer the equivalent of the union operation on its fields, it can be implemented by a Lua script (or even the application).
Other than these two approached, I'm confident that there are more ways to use Redis to achieve what you're trying to do. Choosing the right one is a matter of understanding all the requirements, but I'm afraid that information is lacking from the question.
In case you're not clear what a set is it allows a user to insert a value, check if a value is in the list and find an intersect (if value exist in two sets). Also a set will only hold a value once.
How do I use a set in SQL? I'm using sqlite but will be moving to postgresql. The most obvious option to me was have a Set table of SetID, SetValue with those two values being the primary key. However I have a feeling this wouldn't be efficient as SetID would be repeated many times. The other option is to have a table int SetId, blob SetArray but I'll need to implement set logic by hand.
What's the best way to do sets in SQL? For either sqlite or postgresql?
A couple options:
Trigger to see of value exists and if it does, return NULL (accomplishes exactly what you are doing)
Just use unique constraints and handle errors
Both these move the table from bags to sets but they do so in different ways, one by giving it more select-like semantics (for a slight performance cost) and the other, more traditional SQL way, to force data validation on entry.
If you want to select as a set, use select distinct instead.
In order to be able to implement queries such as
"select * from table1 where id in [some set of elements]"
I am using Oracle arrays. I found the performance to be quite good. Let's say the ids are ints. So I will create an array type:
create type number_varray as varray([size]) of int;
Then I can use a prepared statement
select * from table1 where id in (select * from Table(?))
setting the parameter to an Array type.
I have a few questions:
If I make size really large is there a performance hit if the number of elements in the query is small?
What is the limit to size?
When I call conn.createOracleArray(String name, Obj[] elements) does it hit the server or does that happy after executing a query?
Is there a more efficient way to implement such queries in Oracle?
Actually it is more than one question, but I'll give it a try:
Performance is not really going to be bad if you look at answer 4
According to this article limit is 65536, but I believe if you exceed this number you do something wrong. Use temporary tables or anything similar instead
This is not really connected to Oracle arrays in my opinion...
Use member of, this is much more efficient way:
select * from table1 where id member of my_number_varray;
I always hear from SQL specialists that it is not efficient to use the '*' sign in SELECT statement and it is better to list all the field names instead.
But I don't find it efficient for me personally when it comes to adding new fields to a table and then updating all the stored procedures accordingly.
So what are the pros and cons in using '*' ?
Thanks.
In general, the use of SELECT * is not a good idea.
Pros:
When you add/remove columns, you don't have to make changes where you did use SELECT *
It is shorter to write
Also see the answers to: Can select * usage ever be justified?
Cons:
You are returning more data than you need. Say you add a VARBINARY column that contains 200k per row. You only need this data in one place for a single record - using SELECT * you can end up returning 2MB per 10 rows that you don't need
Explicit about what data is used
Specifying columns means you get an error when a column is removed
The query processor has to do some more work - figuring out what columns exist on the table (thanks #vinodadhikary)
You can find where a column is used more easily
You get all columns in joins if you use SELECT *
You can't use ordinal referencing (though using ordinal references for columns is bad practice in itself)
Also see the answers to: What is the reason not to use select *?
Pros:
when you really need all the columns, it's shorter to write select *
Cons:
most of the time, you don't need all the columns, but only some of them. It's more efficient to only retrieve what you want
you have no guarantee of the order of the retrieved columns (or at least, the order is not obvious from the query), which forbids accessing columns by index (only by name). But the names are also far from obvious
when joining multiple tables having potentially columns with the same name, you can define aliases for these columns
instead of SELECT * FROM mytable, i would like to select all fields EXCEPT one (namely, the 'serialized' field, which stores a serialized object). this is because i think that losing that field will speed up my query by a lot. however, i have so many fields and am quite the lazy guy. is there a way to say...
`SELECT ALL_ROWS_EXCEPT(serialized) FROM mytable`
?
thanks!
No, there is no convention in SQL to get all but one (or a number of designated) column(s).
Being explicit about what column(s) are being returned, preferably using a table alias (even if only for one table), is ideal.