SequelizeJS Order by renamed attribute - orm

I came across an exception, cannot find solution anywhere. It looks like a bug of sequelizeJs.
Here is the question:
I have two model
module.exports = db.sequelize.define('user', {
// id is added automatically
createdAt: {
type: Sequelize.DATE,
field: 'created_at'
}
})
module.exports = db.sequelize.define('question', {
// id is added automatically
userId: {
type: Sequelize.BIGINT,
field: 'user_id'
}
})
User.hasMany(Question,{foreignKey:'userId',as:'questions'})
then I have a query:
models.User.findAll({
limit:limit,
order:[['createdAt','DESC']],
include : [{
{
model:models.Question,
as:"questions"
},]
})
I got an exception saying: "SequelizeDatabaseError: Unknown column 'user.created_at' in 'order clause'"
Here is the generated sql code:
SELECT ***** FROM (SELECT user.id, user.created_at AS createdAt FROM user AS user ORDER BY user.created_at DESC LIMIT 10) AS user LEFT OUTER JOIN question AS questions ON user.id = questions.user_id ORDER BY user.created_at DESC
It seems that order by is applied twice. At the second time, the 'created_at' has been renamed as 'createdAt'. That's why db complains 'cannot find user.created_at'.
My question is how to fix this problem so that I can run my query. I cannot find any available solution. And why Sequelize apply 'order by' twice? it looks like unnecessary.

This is an open issue it seems
See GitHub
However, you could try explicitly stating the attributes you want to query for on both models.

There is no issue with sequelize ,
Issue in below code : there one extra { inside the include block , remove that first
models.User.findAll({
limit:limit,
order:[['createdAt','DESC']],
include : [{
{
model:models.Question,
as:"questions"
},]
})
Query is 100% correct, I have executed the same way and its working fine from both side (sequelize and sql).
Issue you may have is :
your user table must have createdAt field named created_at
inside models.User.findAll() you might have used attributes and
forgot to add created_at
Try to run query inside your sql, and check for the error
SELECT *****
FROM (SELECT user.id, user.created_at AS createdAt FROM user AS user ORDER BY user.created_at DESC LIMIT 10)
AS user
LEFT OUTER JOIN question AS questions ON user.id = questions.user_id
ORDER BY user.created_at DESC
And you will figure it out whats wrong with the query

Related

Sequelize Query : How to do parent and child fetch query when condition on 1 of the child is valid?

I have a parent Table "Orders" and child table "Items". They have one to many relationships. I need to do the query using the "Orders" table. I need the Items in the "include". I want to query the order which has a particular Item. But if it also has other items those too should get selected.
A toned-down version of my query looks something like this:
let term = 'phone';
Orders.findAll({
include: [
model: Item,
where: {itemName: { [Op.like]: `%${term}%` }}
]
})
The query happens but if an order has multiple items and if the item name doesn't have the 'term' in it then that particular item gets excluded. Only the item with the term is returned with the order record.
I want if the where condition is true for 1 of the items, all the other items should also get included.
For example, let's say I search for the term 'phone', I want to get the orders which looks something like the below:
Order No: 123ABC
Order Items:
Android Phone
Smart Watch
In my case, the smartwatch doesn't come with the "include" option.
I will be grateful if anyone can help me find a solution.
Thank You.
If this sql can help you..
select *
from Order o
inner join item i on i.orderId = o.id and i.itemName like %term%
inner join item i2 on i2.orderId = o.id
you can use sequelize like this!
let term = 'phone';
Orders.findAll({
include: [
{
model: Item,
as : 'searchItem',
attributes : [],
where: {itemName: { [Op.like]: `%${term}%` }},
{
model: Item,
}
]
})

Sequelize Query - Count associated tables and count all for pagination

this is my first question on stackoverflow, never used it before but this issue is making me tear my hair out.
I'm building an infinite scroll component for a react app I'm working on a I'm trying to make a Postgres DB query work.
I have 2 tables - Challenges, and UserChallenges.
Challenges have many User Challenges.
I need to get a subsection of Challenges (from start to end) with each Challenge having a count of the number of "participants" (number of associated UserChallenges), and also a count of all challenges.
Something like this:
{
rows: [Challenge, Challenge, Challenge],
count: n
}
Where each challenge includes the total number of userChallenges as "participants" and count is a count of all challenges.
Here is the query:
let json_query = {
attributes: {
include: [[Sequelize.fn("COUNT", Sequelize.col("user_challenges.id")), "participants"]]
},
include: [{
model: UserChallenge, attributes: []
}],
order: [['timestamp', 'DESC']],
offset: start,
limit: end
}
The start and end quantities are the start and end of the pagination.
I'm running this query as follows:
var challengeInstances = await Challenge.findAndCountAll(json_query)
This results in the following error:
name: 'SequelizeDatabaseError',
parent: error: missing FROM-clause entry for table "user_challenges"
and this is the sql it's saying it's running:
`SELECT "challenge".* FROM (SELECT "challenge"."id", "challenge".*, COUNT("user_challenges"."id"), "challenge"."participants" FROM "challenges" AS "challenge" GROUP BY "challenge"."id" ORDER BY "challenge"."end_date" DESC LIMIT '4' OFFSET '0') AS "challenge" LEFT OUTER JOIN "user_challenges" AS "user_challenges" ON "challenge"."id" = "user_challenges"."challenge_id" ORDER BY "challenge"."end_date" DESC;`,
Sequelize or raw queries are both good.
Do let me know if you need any more information and thank you so so much.
you can use sequelize literal like this & remove object from attributes just paste this code for attributes .
attributes: [
[
sequelize.literal(`(
SELECT COUNT(id)
FROM user_challenges
WHERE
// your condition of foreign key like (user_challenges.participants_id = participants.id)
)`),
'numberOfParticipants'
]
]

complex couchbase query using metadata & group by

I am new to Couchbase and kind a stuck with the following problem.
This query works just fine in the Couchbase Query Editor:
SELECT
p.countryCode,
SUM(c.total) AS total
FROM bucket p
USE KEYS (
SELECT RAW "p::" || ca.token
FROM bucket ca USE INDEX (idx_cr)
WHERE ca._class = 'backend.db.p.ContactsDo'
AND ca.total IS NOT MISSING
AND ca.date IS NOT MISSING
AND ca.token IS NOT MISSING
AND ca.id = 288
ORDER BY ca.total DESC, ca.date ASC
LIMIT 20 OFFSET 0
)
LEFT OUTER JOIN bucket finished_contacts
ON KEYS ["finishedContacts::" || p.token]
GROUP BY p.countryCode ORDER BY total DESC
I get this:
[
{
"countryCode": "en",
"total": 145
},
{
"countryCode": "at",
"total": 133
},
{
"countryCode": "de",
"total": 53
},
{
"countryCode": "fr",
"total": 6
}
]
Now, using this query in a spring-boot application i end up with this error:
Unable to retrieve enough metadata for N1QL to entity mapping, have you selected _ID and _CAS?
adding metadata,
SELECT
meta(p).id AS _ID,
meta(p).cas AS _CAS,
p.countryCode,
SUM(c.total) AS total
FROM bucket p
trying to map it to the following object:
data class CountryIntermediateRankDo(
#Id
#Field
val id: String,
#Field
#NotNull
val countryCode: String,
#Field
#NotNull
val total: Long
)
results in:
Unable to execute query due to the following n1ql errors:
{“msg”:“Expression must be a group key or aggregate: (meta(p).id)“,”code”:4210}
Using Map as return value results in:
org.springframework.data.couchbase.core.CouchbaseQueryExecutionException: Query returning a primitive type are expected to return exactly 1 result, got 0
Clearly i missed something important here in terms of how to write proper Couchbase queries. I am stuck between needing metadata and getting this key/aggregate error that relates to the GROUP BY clause. I'd be very thankful for any help.
When you have a GROUP BY query, everything in the SELECT clause should be either a field used for grouping or a group aggregate. You need to add the new fields into the GROUP by statement, sort of like this:
SELECT
_ID,
_CAS,
p.countryCode,
SUM(p.c.total) AS total
FROM testBucket p
USE KEYS ["foo", "bar"]
LEFT OUTER JOIN testBucket finished_contacts
ON KEYS ["finishedContacts::" || p.token]
GROUP BY p.countryCode, meta(p).id AS _ID, meta(p).cas AS _CAS
ORDER BY total DESC
(I had to make some changes to your query to work with it effectively. You'll need to retrofit the advice to your specific case.)
If you need more detailed advice, let me suggest the N1QL forum https://forums.couchbase.com/c/n1ql . StackOverflow is great for one-and-done questions, but the forum is better for extended interactions.

get JOIN table as array of results with PostgreSQL/NodeJS

I'm creating an app where users are able to create questions, and others can upvote/downvote them.
The following is a part of my sql schema:
CREATE TABLE "questions" (
id SERIAL,
content VARCHAR(511) NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
CONSTRAINT pk_question PRIMARY KEY (id)
);
CREATE TABLE "votes" (
id SERIAL,
value INT,
question_id INT NOT NULL,
CONSTRAINT pk_vote PRIMARY KEY (id),
CONSTRAINT fk_question_votes FOREIGN KEY (question_id) REFERENCES questions (id) MATCH SIMPLE ON UPDATE CASCADE ON DELETE CASCADE
);
What I would like to have is Postgres giving me each question with an array of votes, like that:
[{ // a question
id: 1,
content: 'huh?',
votes: [{ // a vote
id: 1,
value: 1
}, { // another vote
id: 2,
value: -1
}]
}, { /*another question with votes*/ }]
I looked at aggregate functions (like array_agg()) but it gave me only the values. A JOIN gave me a question joined with a vote, and would force me to do server side operations, which I would prefer not to.
Is there any way to do that? Is my reasoning regarding what I want to obtain wrong?
Thanks for your time.
This is easy to do with pg-promise:
function buildTree(t) {
const v = q => t.any('SELECT id, value FROM votes WHERE question_id = $1', q.id)
.then(votes => {
q.votes = votes;
return q;
});
return t.map('SELECT * FROM questions', undefined, v).then(a => t.batch(a));
}
db.task(buildTree)
.then(data => {
console.log(data); // your data tree
})
.catch(error => {
console.log(error);
});
The same as above, but using ES7 async/await syntax:
await db.task(async t => {
const questions = await t.any('SELECT * FROM questions');
for(const q of questions) {
q.votes = await t.any('SELECT id, value FROM votes WHERE question_id = $1', [q.id]);
}
return questions;
});
// method "task" resolves with the correct data tree
API: map, any, task, batch
Related questions:
Get a parents + children tree with pg-promise
Conditional task with pg-promise
And if you want to use just a single query, then using PostgreSQL 9.4 and later syntax you can do the following:
SELECT json_build_object('id', q.id, 'content', q.content, 'votes',
(SELECT json_agg(json_build_object('id', v.id, 'value', v.value))
FROM votes v WHERE q.id = v.question_id))
FROM questions q
And then your pg-promise example would be:
const query =
`SELECT json_build_object('id', q.id, 'content', q.content, 'votes',
(SELECT json_agg(json_build_object('id', v.id, 'value', v.value))
FROM votes v WHERE q.id = v.question_id)) json
FROM questions q`;
const data = await db.map(query, [], a => a.json);
And you definitely will want to keep such complex queries in external SQL files. See Query Files.
Conclusion
The choice between the two approaches presented above should be based on the performance requirements of your application:
The single-query approach is faster, but is somewhat difficult to read or extend, being fairly verbose
The multi-query approach is easier to understand and to extend, but it is not great for performance, due to dynamic number of queries executed.
UPDATE-1
The following related answer offers more options, by concatenating child queries, which will give a much improved performance: Combine nested loop queries to parent result pg-promise.
UPDATE-2
Another example added, using ES7 async/await approach.
Please think simple way, May be I am right, I use knex js
let allpost = knex
.select([
'questions.id',
'question.content',
knex.raw('json_agg(v.*) as votes')
])
.from('questions')
.leftJoin('votes as v', 'questions.id', 'v.question_id')
.groupBy('questions.id');
sql-toolkit does exactly this. It's a node library built for pg-promise which allows you to write regular native SQL and receive back properly structured (nested) pure business objects, without either having to split up the query or rewrite it with json_build_object.
For example:
class Article extends BaseDAO {
getBySlug(slug) {
const query = `
SELECT
${Article.getSQLSelectClause()},
${Person.getSQLSelectClause()},
${ArticleTag.getSQLSelectClause()},
${Tag.getSQLSelectClause()}
FROM article
JOIN person
ON article.author_id = person.id
LEFT JOIN article_tags
ON article.id = article_tags.article_id
LEFT JOIN tag
ON article_tags.tag_id = tag.id
WHERE article.slug = $(slug);
`;
return this.one(query, { slug });
// OUTPUT: Article {person: Person, tags: Tags[Tag, Tag, Tag]}
}
The select clause uses the business object "getSQLSelectClause" methods to save tedium in typing the columns, as well as ensure no collisions of names (nothing magical going on, and could just be written out instead).
The this.one is a call into sql-toolkits base DAO class. It is responsible for structuring the flat result records into a nice nested structure.
(Also notice that it is "one" which matches our mental model for the SQL. The DAO methods for one, oneOrNone, many, and any ensure their count against the number of generated top level business objects - not the number of rows the sql expression returns!)
Check out the repository for details on how to set it up on top of pg-promise. It's strictly an enhancement, and does not seek to abstract out pg-promise (you still set up pg-promise and can use it directly). (Disclamer, I am the author of sql-toolkit.)

ZF2 how to avoid sql query limit to add quotes in subquery

I'm trying to set up a subquery in ZendFramework 2 and I got an issue with the limit function for a Select object. Whatever I do, numeric value is put between quotes and makes my query fails : I should get LIMIT 1 and instead I get LIMIT '1'.
Seems this is not the first time this issue has been encountered, I saw some have asked about this issue before (like 8 months ago) but without getting any proper answer.
I also saw this issue has been marker as resolved in 2012 (https://github.com/zendframework/zf2/pull/2775) so I really don't understand what's happening there.
Here's my code in ZF2 :
$resultSet = $this->tableGateway->select( function (Select $select) use ($params) {
$sub = new Select();
$sub->from(array('temp' => 'scores'))
->columns(array(new \Zend\Db\Sql\Expression("id AS id")))
->where(array('temp.glitch' => array('None', 'Glitch')))
->where('temp.zone=scores.zone')
->order('temp.multi DESC, temp.score DESC')
->limit(1);
$select->join('players', 'player=players.id', array('player_name' => 'name', 'player_url' => 'name_url'))
->join('countries', 'players.country=countries.id', array('country_name' => 'name', 'country_iso' => 'iso'))
->join('cars', 'car=cars.id', array('car_name' => 'name'), 'left')
->join('zones', 'zone=zones.id', array('zone_name' => 'name'));
$select->where(array('scores.id' => $sub));
$select->order('scores.zone ASC');
print_r($select->getSqlString());
});
This should render the following query (which I get right except LIMIT '1' instead of LIMIT 1) :
SELECT "scores".*, "players"."name" AS "player_name", "players"."name_url" AS "player_url", "countries"."name" AS "country_name", "countries"."iso" AS "country_iso", "cars"."name" AS "car_name", "zones"."name" AS "zone_name"
FROM "scores" INNER JOIN "players" ON "player"="players"."id"
INNER JOIN "countries" ON "players"."country"="countries"."id"
LEFT JOIN "cars" ON "car"="cars"."id"
INNER JOIN "zones" ON "zone"="zones"."id"
WHERE "scores"."id" = (SELECT id AS id FROM "scores" AS "temp" WHERE "temp"."glitch" IN ('None', 'Glitch')
AND temp.zone=scores.zone ORDER BY "temp"."multi" DESC, "temp"."score" DESC LIMIT 1)
ORDER BY "scores"."zone" ASC
Since this doesn't seem to work this way, is there another way I could proceed to get my limit (using Mysql 5 database) ?
EDIT :
Thanks for your help. Finally I figured out a way to get things done the way I want and to remove the quotes by simply remove the subquery construction and to write it directly in the where function :
$select->where('scores.id = (SELECT id FROM scores AS lookup WHERE lookup.zone = scores.zone ORDER BY multi DESC , score DESC LIMIT 1)');
Although I can continue my dev with this, I feel more like using a poor trick to get rid of this issue and so I will let this question unanswered until someone comes with a real solution there.
Anyway there might be no solution at all, since it might be an issue in ZF2 core itself.
Change the line -
$select->where(array('scores.id' => $sub));
with
$select->where(array('scores.id' => new \Zend\Db\Sql\Expression("({$sub->getSqlString($this->tableGateway->adapter->getPlatform())})"));
Try with just above change.
And if it still doesn't work then make changes to the core Select class file located at -
PROJECT_FOLDER/vendor/zendframework/zendframework/library/Zend/Db/Sql/Select.php
Line No. 921 -
Change $sql = $platform->quoteValue($limit); with $sql = $limit;
Line No. 940 -
Change return array($platform->quoteValue($offset)); with return array($offset);
I have come across the issue from github and wondered as why it is still not working with the latest ZF2 files. I know the solution given above doesn't look like the proper one but I had to somehow make it work. I have tried it and it works.
Its only a quick fix before the actual solution comes into picture.