How to query from JSON arrays from two different tables? - sql

I have a table name doc_definition with a json column definition which has a format like:
[{
"id":"0",
"name:"Ques1"
},{
"id":"1",
"name:"Ques2"
},{
"id":"2",
"name:"Ques3"
}]
and another table doc which has another json column def_val with format as:
{
"0":{
"value":"Ans1"
},
"1":{
"value":"Ans2"
},
"2":{
"value":"Ans3"
}
}
I want to create a Postgres SQL query that gives the value of a corresponding question by matching the id field inside both json.
SO far I've come up with this:
SELECT json_array_elements(def.definition) ->> 'name' AS json_test
FROM document_definitions AS def
INNER JOIN documents AS doc
ON doc.document_definition_id = def.id
WHERE doc.id = 892 AND json_array_elements(def.definition) ->> 'name' = 'Ques2'
But this throws exception:
ERROR: argument of AND must not return a set
When i put the json_array_elements(def.definition) ->> 'name' = 'Ques2' in SELECT clause, it returns boolean value.
I dont know whats wrong then. Plz Help?

How about that query :
SELECT json_array_elements(def.definition) ->> 'name' AS json_test
FROM document_definitions AS def
INNER JOIN documents AS doc
ON doc.document_definition_id = def.id
WHERE doc.id = 892 AND (json_array_elements(def.definition) ::json->> 'name') = 'Ques2'

After many trial and errors, I've come up with this.
Plz let me know if I can improve this in any way.
SELECT definition_json.definition_value -> definition_json.def_value_id ->> 'value' AS definition_value
FROM (
SELECT json_array_elements(def.definition) ->> 'name' = 'Ques2' AS is_definition_present,
json_array_elements(def.definition) ->> 'id' AS def_value_id,
doc.definition_value, doc.id AS document_id, def.id AS definition_id
FROM document_definitions AS def
INNER JOIN documents AS doc
ON doc.document_definition_id = def.id
) AS definition_json
WHERE definition_json.is_definition_present = 't' AND definition_json.document_id = 892

Related

How to count number of user id in a list without duplicate in sequelize [duplicate]

I am trying to get a distinct count of a particular column using sequelize. My initial attempt is using the 'count' method of my model, however it doesn't look like this is possible.
The DISTINCT feature is needed because I am joining other tables and filtering the rows of the parent based on the related tables.
here's the query I would like:
SELECT COUNT(DISTINCT Product.id) as `count`
FROM `Product`
LEFT OUTER JOIN `Vendor` AS `vendor` ON `vendor`.`id` = `Product`.`vendorId`
WHERE (`vendor`.`isEnabled`=true );
using the following query against my Product model:
Product.count({
include: [{model: models.Vendor, as: 'vendor'}],
where: [{ 'vendor.isEnabled' : true }]
})
Generates the following query:
SELECT COUNT(*) as `count`
FROM `Product`
LEFT OUTER JOIN `Vendor` AS `vendor` ON `vendor`.`id` = `Product`.`vendorId`
WHERE (`vendor`.`isEnabled`=true );
UPDATE: New version
There are now separate distinct and col options. The docs for distinct state:
Apply COUNT(DISTINCT(col)) on primary key or on options.col.
You want something along the lines of:
MyModel.count({
include: ...,
where: ...,
distinct: true,
col: 'Product.id'
})
.then(function(count) {
// count is an integer
});
Original Post
(As mentioned in the comments, things have changed since my original post, so you probably want to ignore this part.)
After looking at Model.count method in lib/model.js, and tracing some code, I found that when using Model.count, you can just add any kind of aggregate function arguments supported by MYSQL to your options object. The following code will give you the amount of different values in MyModel's someColumn:
MyModel.count({distinct: 'someColumn', where: {...}})
.then(function(count) {
// count is an integer
});
That code effectively generates a query of this kind: SELECT COUNT(args) FROM MyModel WHERE ..., where args are all properties in the options object that are not reserved (such as DISTINCT, LIMIT and so on).
The Sequelize documentation on count links to a count method that doesn't let you specify which column to get the count of distinct values:
Model.prototype.count = function(options) {
options = Utils._.clone(options || {});
conformOptions(options, this);
Model.$injectScope(this.$scope, options);
var col = '*';
if (options.include) {
col = this.name + '.' + this.primaryKeyField;
expandIncludeAll.call(this, options);
validateIncludedElements.call(this, options);
}
Utils.mapOptionFieldNames(options, this);
options.plain = options.group ? false : true;
options.dataType = new DataTypes.INTEGER();
options.includeIgnoreAttributes = false;
options.limit = null;
options.offset = null;
options.order = null;
return this.aggregate(col, 'count', options);
};
Basically SELECT COUNT(DISTINCT(*)) or SELECT COUNT(DISTINCT(primaryKey)) if you've got a primary key defined.
To do the Sequelize equivalent of SELECT category, COUNT(DISTINCT(product)) as 'countOfProducts' GROUP BY category, you'd do:
model.findAll({
attributes: [
'category',
[Sequelize.literal('COUNT(DISTINCT(product))'), 'countOfProducts']
],
group: 'category'
})
Looks like this is now supported in Sequelize versions 1.7.0+.
the count and findAndCountAll methods of a model will give you 'real' or 'distinct' count of your parent model.
I was searching for SELECT COUNT(0) query for sequelize, below is the answer for that.
let existingUsers = await Users.count({
where: whereClouser,
attributes: [[sequelize.fn('COUNT', 0), 'count']]
});
This helped me to get distinct count from another table rows,
dataModel.findAll({
attributes: {
include: [[Sequelize.literal("COUNT(DISTINCT(history.data_id))"), "historyModelCount"]]
},
include: [{
model: historyModel, attributes: []
}],
group: ['data.id']
});
Ref 1, Ref 2.
With respect to your question in order to get the distinct counts of products based on the id of product
you just need to pass the key 'distinct' with value 'id' to your count object , Here is the example
To generate this sql query as you asked
SELECT COUNT(DISTINCT(`Product`.`id`)) as `count`
FROM `Product`
LEFT OUTER JOIN `Vendor` AS `vendor` ON `vendor`.`id` = `Product`.`vendorId`
WHERE (`vendor`.`isEnabled`=true );
Add 'distinct' key in your Sequelize query
Product.count({
include: [{model: models.Vendor, as: 'vendor'}],
where: [{ 'vendor.isEnabled' : true }],
distinct: 'id' // since count is applied on Product model and distinct is directly passed to its object so Product.id will be selected
});
This way of using 'distinct' key to filter out distinct counts or rows , I tested in Sequelize Version 6.
Hope this will help you or somebody else!

SELECT DISTINCT ON expressions must match initial ORDER BY expressions in typeorm query

I have a query that I want get rows with unique item_id and sort them according to timestamp.
But I get this error everytime from this code
const boxPreviewActivities = await this.activityLogEntryRepository
.createQueryBuilder('activityLogEntry')
.where("activityLogEntry.activity = 'BOX_PREVIEW'")
.andWhere('activityLogEntry.projectId = :projectId', { projectId })
.andWhere('activityLogEntry.userSub = :userSub', { userSub: user?.sub })
.andWhere((qb) => {
const subQuery = qb
.subQuery()
.select(
"DISTINCT activityLogEntry.originalData ::jsonb -> 'source' ->> 'item_id'"
)
.from(ActivityLogEntryEntity, 'activityLogEntry')
.where("activityLogEntry.activity = 'BOX_DELETE'")
.getQuery();
return `activityLogEntry.originalData ::jsonb -> 'source' ->> 'item_id' NOT IN (${subQuery})`;
})
.distinctOn([
"activityLogEntry.originalData ::jsonb -> 'source' ->> 'item_id' ",
])
.orderBy('activityLogEntry.timestamp', 'DESC')
.limit(limitNumber)
.getMany();
return boxPreviewActivities;
}
** ERROR [ExceptionsHandler] SELECT DISTINCT ON expressions must match initial ORDER BY expressions
QueryFailedError: SELECT DISTINCT ON expressions must match initial ORDER BY expressions**
if I add this two order functions at the and, I dont get error but my query not sorting according to timestamp, instead sort only with item_id. Actually I only want to sort according to timestamp. How can I refactor my query to make it working ?
.orderBy(
"activityLogEntry.originalData ::jsonb -> 'source' ->> 'item_id'",
'DESC'
)
.addOrderBy('activityLogEntry.timestamp', 'DESC')

How to make multiple SELECT queries at once in PostgreSQL?

Update: See the "Update" section below for the latest.
I have been working with Knex.js to build SQL queries in Node.js, and have the following code. This code works on a sort of graph data model (nodes and links), where there is a links table which has everything (links link to links). Given this code, I am wondering how I can make it one query instead of one query per attribute which is how it is now. The getTableName() function returns a string_links table for string values, and <x>_links tables for the other datatypes, while the "basic" links table is just called links.
Essentially how this works is, first query the top level where the parent_id is equal to some "type" ID, say we are querying "user" objects, the type would be "user". So let instance = ... is getting all the instance links from this user type. Then we go through each field of a query (a query for now is just boolean-valued map, like { email: true, name: true }). For each field of the query, we make a query to find all those nodes, linked off the instance, as so-called property links.
There are two types of properties, but don't need to go into too much detail on that. Essentially there are complex properties with audit trails and simple properties without audit trails. That is what is meant by the interactive branch in the logic.
How can I make this into one SQL query? The SQL query it prints out for an example is like this:
select "id" from "links" where "parent_id" = '47c1956bz31330c' and "name" = 'link' limit 1
select "value" from "string_links" where "parent_id" = (select "value" from "links" where "parent_id" = '47c1956bz31330cv' and "name" = 'name' limit 1) and "name" = 'value' limit 1
select "value" from "text_links" where "parent_id" = (select "value" from "links" where "parent_id" = '47c1956bz31330cv' and "name" = 'website' limit 1) and "name" = 'value' limit 1
select "value" from "integer_links" where "parent_id" = (select "value" from "links" where "parent_id" = '47c1956bz31330cv' and "name" = 'revenue' limit 1) and "name" = 'value' limit 1
select "value" from "boolean_links" where "parent_id" = '47c1956bz31330' and "name" = 'verified' limit 1
The original Node.js for Knex.js is here, but really I'm just concerned with how to write this as one regular SQL query, and I can figure out how to make it in Knex.js from there:
async function selectInteractiveInstance(user, name, query) {
const type = model.types[name]
const typeId = await baseSchemaController.selectType(name)
let instance = await knex.from(`links`)
.select('id')
.where('parent_id', typeId)
.where('name', 'instance')
.first()
// { id: 123, props: { ... } }
instance.props = {}
for (let field in query) {
let data = query[field]
let attrSchema = type[field]
const tableName = baseSchemaController.getTableName(attrSchema.type)
if (attrSchema.interactive) {
const query1 = knex
.from(`links`)
.select('value')
.where('parent_id', instance.link)
.where('name', field)
.first()
const record = await knex
.from(tableName)
.select('value')
.where('home', query1)
.where('name', 'value')
.first()
if (record) {
instance.props[field] = record.value
}
} else {
const record = await knex
.from(tableName)
.select('value')
.where('parent_id', instance.id)
.where('name', field)
.first()
if (record) {
instance.props[field] = record.value
}
}
}
return instance
}
The reason for asking is because the number of queries of this function is equal to the number of properties on the object, and I would like to avoid that, but not really that great at SQL yet. I don't see a straightforward or clear path on how to make this into one query, or know if it's possible.
It's also an issue for the following reason. If I want to grab 100 links, and their "fields" (in the primitive link tables), such that the primitive link values match a certain value, then you need to query all field tables simultaneously to see if the query can be satisfied.
Update
I finally landed on a query that works in the optimistic case:
select
"x"."id" as "id",
"s1"."value" as "name",
"s2"."value" as "inc_id",
"s3"."value" as "website",
"s4"."value" as "revenue",
"s5"."value" as "verified"
from "links" as "x"
inner join "links" as "c1" on "c1"."parent_id" = "x"."id"
inner join "string_links" as "s1" on "s1"."parent_id" = "c1"."value"
inner join "links" as "c2" on "c2"."parent_id" = "x"."id"
inner join "string_links" as "s2" on "s2"."parent_id" = "c2"."value"
inner join "links" as "c3" on "c3"."parent_id" = "x"."id"
inner join "text_links" as "s3" on "s3"."parent_id" = "c3"."value"
inner join "links" as "c4" on "c4"."parent_id" = "x"."id"
inner join "integer_links" as "s4" on "s4"."parent_id" = "c4"."value"
inner join "boolean_links" as "s5" on "s5"."parent_id" = "x"."id"
where "x"."parent_id" = '47c1956bz31330'
and "x"."name" = 'link'
and "c1"."name" = 'name'
and "s1"."name" = 'value'
and "c2"."name" = 'inc_id'
and "s2"."name" = 'value'
and "c3"."name" = 'website'
and "s3"."name" = 'value'
and "c4"."name" = 'revenue'
and "s4"."name" = 'value'
and "s5"."name" = 'verified'
This returns an object similar to what I am looking for, joining the same table several times, along with the primitive tables.
However, if any of the values are not linked (are socalled "null" in this context), then the inner join will fail and it will return nothing. How can I still have it return a subset of the object properties, whatever it can find? Is there anything like optional inner joins or anything like that?
Use LEFT JOIN and move possibly unsatisfied predicates to ON clause. Kind of
select
"x"."id" as "id",
"s1"."value" as "name",
"s2"."value" as "inc_id",
"s3"."value" as "website",
"s4"."value" as "revenue",
"s5"."value" as "verified"
from "links" as "x"
left join "links" as "c1" on "c1"."parent_id" = "x"."id" and "c1"."name" = 'name'
left join "string_links" as "s1" on "s1"."parent_id" = "c1"."value" and "s1"."name" = 'value'
left join "links" as "c2" on "c2"."parent_id" = "x"."id" and "c2"."name" = 'inc_id'
left join "string_links" as "s2" on "s2"."parent_id" = "c2"."value" and "s2"."name" = 'value'
left join "links" as "c3" on "c3"."parent_id" = "x"."id" and "c3"."name" = 'website'
left join "text_links" as "s3" on "s3"."parent_id" = "c3"."value" and "s3"."name" = 'value'
left join "links" as "c4" on "c4"."parent_id" = "x"."id" and "c4"."name" = 'revenue'
left join "integer_links" as "s4" on "s4"."parent_id" = "c4"."value" and "s4"."name" = 'value'
left join "boolean_links" as "s5" on "s5"."parent_id" = "x"."id" and "s5"."name" = 'verified'
where "x"."parent_id" = '47c1956bz31330'
and "x"."name" = 'link'

Symfony Doctrine Query returns Array instead of Object

I've found a lot of questions going the other way but none that explain why I'm getting an Array when I use inner joins.
My Query:
$entityManager = $this->getEntityManager();
$query = $entityManager->createQuery(
'SELECT sp, p
FROM App\Entity\Portal\StaticPage sp
INNER JOIN sp.portal p
WHERE p.id = :id
AND sp.name = :name'
)->setParameters(array('name'=> $name, 'id'=> $id));
return $query->getOneOrNullResult();
This gives the following error: Notice: Undefined index: staticpage
But if I do this:
return $query->getArrayResult();
I get the following output:
array[
0 => array [ Object in array form ]
]
Any ideas how to get the object instead of an array with all the values in?

ActiveRecord has_many with multiple conditions

I'm trying to find an object by checking for several of its relations.
Loan.joins(:credit_memo_attributes)
.where(credit_memo_attributes: {name: 'pr2_gtx1_y', value: '2014'})
.where(credit_memo_attributes: {name: 'pr1_gtx1_y', value: '2013'})
.where(credit_memo_attributes: {name: 'tx1_y', value: '2014'})
Calling to_sql on that gives:
"SELECT `loans`.* FROM `loans` INNER JOIN `credit_memo_attributes`
ON `credit_memo_attributes`.`loan_id` = `loans`.`id`
WHERE `credit_memo_attributes`.`name` = 'pr2_gtx1_y' AND `credit_memo_attributes`.`value` = '2014'
AND `credit_memo_attributes`.`name` = 'pr1_gtx1_y' AND `credit_memo_attributes`.`value` = '2013'
AND `credit_memo_attributes`.`name` = 'tx1_y' AND `credit_memo_attributes`.`value` = '2014'"
So, I'm checking for Loans that have credit_memo_attributes with all of those attributes. I know at least 1 of our 20,000 loans meets this criteria, but this query returns an empty set. If I only use 1 of the where clauses, it returns several, as I'd expect, but once I add even 1 more, it's empty.
Any idea where I'm going wrong?
Update:
Based on comments I believe you want multiple joins in your criteria. You can do that like this:
attr_1 = {name: 'pr2_gtx1_y', value: '2014'}
attr_2 = {name: 'pr1_gtx1_y', value: '2013'}
attr_3 = {name: 'tx1_y', value: '2014'}
Loan.something_cool(attr_1, attr_2, attr_3)
class Loan < ActiveRecord::Base
...
def self.something_cool(attr_1, attr_2, attr_3)
joins(sanitize_sql(["INNER JOIN credit_memo_attributes AS cma1 ON cma1.loan_id = loans.id AND cma1.name = :name AND cma1.value = :value", attr_1]))
.joins(sanitize_sql(["INNER JOIN credit_memo_attributes AS cma2 ON cma2.loan_id = loans.id AND cma2.name = :name AND cma2.value = :value", attr_2]))
.joins(sanitize_sql(["INNER JOIN credit_memo_attributes AS cma3 ON cma3.loan_id = loans.id AND cma3.name = :name AND cma3.value = :value", attr_3]))
end
If you look at the SQL generated (that you included in your question, thank you) you'll see that all those conditions are being ANDed together. There are NO rows for which name = 'pr2_gtx1_y' AND name = 'pr1_gtx1_y' (and so forth). So you are getting the result I would expect (no rows).
You can put all names and values into array like ids and years and pass those into where clause like this. Active Record will query all the values in the array.
Loan.joins(:credit_memo_attributes)
.where(credit_memo_attributes: {name: ids, value: years})
Personally I'm still learning active record, in this concern i don't think active record supports multiple where clauses.
Notice how the SQL version is returning your code: it is joining the requirements with an AND.
"SELECT `loans`.* FROM `loans` INNER JOIN `credit_memo_attributes`
ON `credit_memo_attributes`.`loan_id` = `loans`.`id`
WHERE `credit_memo_attributes`.`name` = 'pr2_gtx1_y' AND `credit_memo_attributes`.`value` = '2014'
AND `credit_memo_attributes`.`name` = 'pr1_gtx1_y' AND `credit_memo_attributes`.`value` = '2013'
AND `credit_memo_attributes`.`name` = 'tx1_y' AND `credit_memo_attributes`.`value` = '2014'"
Now, this is next to impossible. An Object.name can never be all pr2_gtx1_y, pr1_gtx1_y, and tx1_y. Same goes for the value attributes.
What you need here is an OR as opposed to the AND.
To this effect, try to change your query to the following:
Loan.joins(:credit_memo_attributes)
.where(
"credit_memo_attributes.name = ? and credit_memo_attributes.value = ?
OR credit_memo_attributes.names = ? and credit_memo_attributes.value = ?
OR credit_memo_attributes.name = ? and credit_memo_attributes.value = ?",
'pr2_gtx1_y', '2014',
'pr1_gtx1_y', '2013',
'tx1_y', '2014'
)