I have two queries which look at separate database tables and find items from a JSONB column in each table that are in the format ["tag1","tag2","tag3"] etc. The purpose of the queries are to populate a list for a predictive dropdown i.e. if the list contains "dog" and the user types "d", "dog" should be returned. Each of these queries works individually and I can easily combine them into a single JOOQ query?
final Field<String> value = field(name("A", "value"), String.class);
final Result<Record1<String>> res1 = sql.dsl()
.selectDistinct(value)
.from(CAMPAIGN,lateral(table("jsonb_array_elements_text({0})", CAMPAIGN.TAGS)).as("A"))
.where(CAMPAIGN.STORE_KEY.equal(campaign.getStoreKey()))
.and(CAMPAIGN.CAMPAIGN_KEY.notEqual(campaignKey))
.and(value.like(search + "%%"))
.fetch();
final Result<Record1<String>> res2 = sql.dsl()
.selectDistinct(value)
.from(STOREFRONT, lateral(table("jsonb_array_elements_text({0})", STOREFRONT.TAGS)).as("A"))
.where(STOREFRONT.STORE_KEY.equal(campaign.getStoreKey()))
.and(value.like(search + "%%")).fetch();
Sure! In SQL, "combining" two queries is mostly implemented using UNION [ ALL ] (where ALL indicates that you want to maintain duplicates). In your case, write the following:
final Result<Record1<String>> result =
sql.dsl()
.select(value)
.from(
CAMPAIGN,
lateral(table("jsonb_array_elements_text({0})", CAMPAIGN.TAGS)).as("A"))
.where(CAMPAIGN.STORE_KEY.equal(campaign.getStoreKey()))
.and(CAMPAIGN.CAMPAIGN_KEY.notEqual(campaignKey))
.and(value.like(search + "%%"))
.union(
select(value)
.from(
STOREFRONT,
lateral(table("jsonb_array_elements_text({0})", STOREFRONT.TAGS)).as("A"))
.where(STOREFRONT.STORE_KEY.equal(campaign.getStoreKey()))
.and(value.like(search + "%%")))
.fetch();
Note that I have replaced selectDistinct() by select(), because the UNION operation already removes duplicates, so there's no need to remove duplicates in each individual union subquery.
Related
I have a JSON API payload containing tablename, columnlist - how to build a SELECT query from it using pypika?
So far I have been able to use a string columnlist, but not able to do advanced querying using functions, analytics etc.
from pypika import Table, Query, functions as fn
def generate_sql (tablename, collist):
table = Table(tablename)
columns = [str(table)+'.'+each for each in collist]
q = Query.from_(table).select(*columns)
return q.get_sql(quote_char=None)
tablename = 'customers'
collist = ['id', 'fname', 'fn.Sum(revenue)']
print (generate_sql(tablename, collist)) #1
table = Table(tablename)
q = Query.from_(table).select(table.id, table.fname, fn.Sum(table.revenue))
print (q.get_sql(quote_char=None)) #2
#1 outputs
SELECT "customers".id,"customers".fname,"customers".fn.Sum(revenue) FROM customers
#2 outputs correctly
SELECT id,fname,SUM(revenue) FROM customers
You should not be trying to assemble the query in a string by yourself, that defeats the whole purpose of pypika.
What you can do in your case, that you have the name of the table and the columns coming as texts in a json object, you can use * to unpack those values from the collist and use the syntax obj[key] to get the table attribute with by name with a string.
q = Query.from_(table).select(*(table[col] for col in collist))
# SELECT id,fname,fn.Sum(revenue) FROM customers
Hmm... that doesn't quite work for the fn.Sum(revenue). The goal is to get SUM(revenue).
This can get much more complicated from this point. If you are only sending column names that you know to belong to that table, the above solution is enough.
But if you have complex sql expressions, making reference to sql functions or even different tables, I suggest you to rethink your decision of sending that as json. You might end up with something as complex as pypika itself, like a custom parser or wathever. than your better option here would be to change the format of your json response object.
If you know you only need to support a very limited set of capabilities, it could be feasible. For example, you can assume the following constraints:
all column names refer to only one table, no joins or alias
all functions will be prefixed by fn.
no fancy stuff like window functions, distinct, count(*)...
Then you can do something like:
from pypika import Table, Query, functions as fn
import re
tablename = 'customers'
collist = ['id', 'fname', 'fn.Sum(revenue / 2)', 'revenue % fn.Count(id)']
def parsed(cols):
pattern = r'(?:\bfn\.[a-zA-Z]\w*)|([a-zA-Z]\w*)'
subst = lambda m: f"{'' if m.group().startswith('fn.') else 'table.'}{m.group()}"
yield from (re.sub(pattern, subst, col) for col in cols)
table = Table(tablename)
env = dict(table=table, fn=fn)
q = Query.from_(table).select(*(eval(col, env) for col in parsed(collist)))
print (q.get_sql(quote_char=None)) #2
Output:
SELECT id,fname,SUM(revenue/2),MOD(revenue,COUNT(id)) FROM customers
I have a table with a text column and I would like to select all rows that match the list of search parameters that were provided by the user:
select * from value where value.text in ('Mary', 'Steve', 'Walter');
In addition, I want to notify the user if any of his search terms could not be found. Let's say 'Steve' does not exist in the value.text column, how can I write a query that will show 'Steve'? As that information does not exist in any table, I have no idea how it could be done using a SQL query.
The actual Hibernate code looks like this:
List<String> searchItemList = new ArrayList<>();
searchItemList.add("Mary");
searchItemList.add("Steve");
searchItemList.add("Walter");
Query query = em.createQuery("select v from Value as v where v.text in ( :searchitemlist )");
query.setParameter("searchitemlist", searchItemList);
List result = query.getResultList();
log.info("{}", result.size());
log.info("{}", result);
The searchItemList is a list of all search terms provided by the user. Can be a few hundreds lines long. The current workaround is to search the value table once for each searchItem and note all queries that return 0 rows. That is rather inefficient, surely there is a better approach? Please advise.
You can use the following query to get an array of search items that exist in the database
SELECT DISTINCT value.text from value where value.text in ('Mary', 'Steve', 'Walter');
after running this query, If we assume that the answer is stored in an array called result, notExistSearchListItems will give you the final result
IEnumerable<string> notExistSearchListItems = searchItemList.Except(result);
I want to convert an SQL query into a JSONiq Query, is there already an implementation for this, if not, what do I need to know to be able to create a program that can do this ?
I am not aware of an implementation, however, it is technically feasible and straightforward. JSONiq has 90% of its DNA coming from XQuery, which itself was partly designed by people involved in SQL as well.
From a data model perspective, a table is mapped to a collection and each row of the table is mapped to a flat JSON object, i.e., all fields are atomic values, like so:
{
"Name" : "Turing",
"First" : "Alan",
"Job" : "Inventor"
}
Then, the mapping is done by converting SELECT-FROM-WHERE queries to FLWOR expressions, which provide a superset of SQL's functionality.
For example:
SELECT Name, First
FROM people
WHERE Job = "Inventor"
Can be mapped to:
for $person in collection("people")
where $person.job eq "Inventor"
return project($person, ("Name", "First"))
More complicated queries can also be mapped quite straight-forwardly:
SELECT Name, COUNT(*)
FROM people
WHERE Job = "Inventor"
GROUP BY Name
HAVING COUNT(*) >= 2
to:
for $person in collection("people")
where $person.job eq "Inventor"
group by $name := $person.name
where count($person) ge 2
return {
name: $name,
count: count($person)
}
Actually, if for had been called from and return had been called select, and if these keywords were written uppercase, the syntax of JSONiq would be very similar to that of SQL: it's only cosmetics.
{"create_channel":"1","update_comm":"1","channels":"*"}
This is the database field which I want to query.
What would my query look like if I wanted to select all the records that have a "create_channel": "1" and a "update_comm": "1"
Additional question:
View the field below:
{"create_channel":"0","update_comm":"0","channels":[{"ch_id":"33","news":"1","parties":"1","questions ":"1","cam":"1","edit":"1","view_subs":"1","invite_subs":"1"},{"ch_id":"18","news":"1","parties":"1","questions ":"1","cam":"1","edit":"1","view_subs":"1","invite_subs":"1"}]}
How would I go about finding out all those that are subadmins in the News, parties, questions and Cams sections
You can use the ->> operator to return a member as a string:
select *
from YourTable
where YourColumn->>'create_channel' = '1' and
YourColumn->>'update_comm' = '1'
To find a user who has news, parties, questions and cam in channel 33, you can use the #> operator to check if the channels array contains those properties:
select *
from YourTable
where YourColumn->'channels' #> '[{
"ch_id":"33",
"news":"1",
"parties":"1",
"questions ":"1",
"cam":"1"
}]';
I had a scenario in Oracle where i need to match a substring part of column with a list of values. i was using sqlfunction projection for applying the substring on the required column, and added that projection as part of an In Clause Restriction. Below is the simplified criteria i wrote for that.
ICriteria criteriaQuery = session.CreateCriteria<Meeting>()
.Add(Restrictions.In(
Projections.SqlFunction(
"substring",
NHibernateUtil.String,
Projections.Property("Code"),
Projections.Constant(1),
Projections.Constant(3)),
new string[] { "D01", "D02" }))
.Add(Restrictions.In("TypeId", meetingTypes));
The problem that i had with this was that the generated SQL was wrong, where the number of parameters registered for the statement are more than what the statement actually uses and some parameters are repeated even though they are not used. This causes the statement to fail with the message - ORA-01036: illegal variable name/number.
Generated Query
SELECT this_.Meeting_id as Meeting1_0_2_, .....
WHERE substr(this_.Mcs_Main, :p0, :p1) in (:p2, :p3)
and this_.Meeting_Type_Id in (:p4, :p5);
:p0 = 1, :p1 = 3, :p2 = 1, :p3 = 3, :p4 = 'D02', :p5 = 'D03', :p6 = 101, :p7 = 102
p2 and p3 are generated again and are duplicates of p0, p1 because of which the entire query is failing.
I was able to temporarily resolve this by mapping a a new property with a formula, but i don't think that is the right approach since the formula will be executed always even when i don't need the substring to be evaluated.
Any suggestions on whether projections work fine when used with the combination of In clause, the same projection works fine when i use Equal Restriction and not In.
This bug is fixed in 3.0.0.GA version.