Sequelize Query - Count associated tables and count all for pagination - sql

this is my first question on stackoverflow, never used it before but this issue is making me tear my hair out.
I'm building an infinite scroll component for a react app I'm working on a I'm trying to make a Postgres DB query work.
I have 2 tables - Challenges, and UserChallenges.
Challenges have many User Challenges.
I need to get a subsection of Challenges (from start to end) with each Challenge having a count of the number of "participants" (number of associated UserChallenges), and also a count of all challenges.
Something like this:
{
rows: [Challenge, Challenge, Challenge],
count: n
}
Where each challenge includes the total number of userChallenges as "participants" and count is a count of all challenges.
Here is the query:
let json_query = {
attributes: {
include: [[Sequelize.fn("COUNT", Sequelize.col("user_challenges.id")), "participants"]]
},
include: [{
model: UserChallenge, attributes: []
}],
order: [['timestamp', 'DESC']],
offset: start,
limit: end
}
The start and end quantities are the start and end of the pagination.
I'm running this query as follows:
var challengeInstances = await Challenge.findAndCountAll(json_query)
This results in the following error:
name: 'SequelizeDatabaseError',
parent: error: missing FROM-clause entry for table "user_challenges"
and this is the sql it's saying it's running:
`SELECT "challenge".* FROM (SELECT "challenge"."id", "challenge".*, COUNT("user_challenges"."id"), "challenge"."participants" FROM "challenges" AS "challenge" GROUP BY "challenge"."id" ORDER BY "challenge"."end_date" DESC LIMIT '4' OFFSET '0') AS "challenge" LEFT OUTER JOIN "user_challenges" AS "user_challenges" ON "challenge"."id" = "user_challenges"."challenge_id" ORDER BY "challenge"."end_date" DESC;`,
Sequelize or raw queries are both good.
Do let me know if you need any more information and thank you so so much.

you can use sequelize literal like this & remove object from attributes just paste this code for attributes .
attributes: [
[
sequelize.literal(`(
SELECT COUNT(id)
FROM user_challenges
WHERE
// your condition of foreign key like (user_challenges.participants_id = participants.id)
)`),
'numberOfParticipants'
]
]

Related

Cannot parse SQL result count from Logic App

I run this simple query in Logic App using the "Execute a SQL query (V2)" connector to find out if a number exists in my table.
select count(*) from users where user_number='724-555-5555';
If the number exist, I get this JSON , but somehow I cant parse it.
[
{
"": 1
}
]
Any idea how to simply retrieve 0 or 1 ?
Thanks
David
You need to add an explicit column name:
SELECT
count(*) AS cnt
FROM
users
WHERE
user_number = '724-555-5555';
That will give you this result:
[ { "cnt": 1 } ]
...which is valid JSON.

Select documents from Fauna collection between two dates AND that satisfy another criteria

I am able to use the following code to retrieve documents from a Fauna collection that have a date which falls between start and end dates:
Paginate(Range(Match(Index("orders_by_date")) , start, end))
Is it possible to add another criteria to this statement to retrieve not only, in this case, orders between two dates but also have the field status = "completed".
Thank you
You can create an index that way:
CreateIndex(
{
name:'orders_by_date_status',
source:Collection("orders"),
terms: [{field:['data','status']}],
values:[{field:['data','order_date']},{field:['ref']}]
}
)
and query your collection with a query like this:
Paginate(
Range(
Match('orders_by_date_status','completed'),
[Date("2020-03-20")],
[Date("2020-06-20")]
)
)
to get back something like this:
{
data: [
[Date("2020-05-20"), Ref(Collection("orders"), "285246145700037121")],
[Date("2020-06-20"), Ref(Collection("orders"), "285246152717107713")]
]
}
Hope this answers your question.
Luigi

complex couchbase query using metadata & group by

I am new to Couchbase and kind a stuck with the following problem.
This query works just fine in the Couchbase Query Editor:
SELECT
p.countryCode,
SUM(c.total) AS total
FROM bucket p
USE KEYS (
SELECT RAW "p::" || ca.token
FROM bucket ca USE INDEX (idx_cr)
WHERE ca._class = 'backend.db.p.ContactsDo'
AND ca.total IS NOT MISSING
AND ca.date IS NOT MISSING
AND ca.token IS NOT MISSING
AND ca.id = 288
ORDER BY ca.total DESC, ca.date ASC
LIMIT 20 OFFSET 0
)
LEFT OUTER JOIN bucket finished_contacts
ON KEYS ["finishedContacts::" || p.token]
GROUP BY p.countryCode ORDER BY total DESC
I get this:
[
{
"countryCode": "en",
"total": 145
},
{
"countryCode": "at",
"total": 133
},
{
"countryCode": "de",
"total": 53
},
{
"countryCode": "fr",
"total": 6
}
]
Now, using this query in a spring-boot application i end up with this error:
Unable to retrieve enough metadata for N1QL to entity mapping, have you selected _ID and _CAS?
adding metadata,
SELECT
meta(p).id AS _ID,
meta(p).cas AS _CAS,
p.countryCode,
SUM(c.total) AS total
FROM bucket p
trying to map it to the following object:
data class CountryIntermediateRankDo(
#Id
#Field
val id: String,
#Field
#NotNull
val countryCode: String,
#Field
#NotNull
val total: Long
)
results in:
Unable to execute query due to the following n1ql errors:
{“msg”:“Expression must be a group key or aggregate: (meta(p).id)“,”code”:4210}
Using Map as return value results in:
org.springframework.data.couchbase.core.CouchbaseQueryExecutionException: Query returning a primitive type are expected to return exactly 1 result, got 0
Clearly i missed something important here in terms of how to write proper Couchbase queries. I am stuck between needing metadata and getting this key/aggregate error that relates to the GROUP BY clause. I'd be very thankful for any help.
When you have a GROUP BY query, everything in the SELECT clause should be either a field used for grouping or a group aggregate. You need to add the new fields into the GROUP by statement, sort of like this:
SELECT
_ID,
_CAS,
p.countryCode,
SUM(p.c.total) AS total
FROM testBucket p
USE KEYS ["foo", "bar"]
LEFT OUTER JOIN testBucket finished_contacts
ON KEYS ["finishedContacts::" || p.token]
GROUP BY p.countryCode, meta(p).id AS _ID, meta(p).cas AS _CAS
ORDER BY total DESC
(I had to make some changes to your query to work with it effectively. You'll need to retrofit the advice to your specific case.)
If you need more detailed advice, let me suggest the N1QL forum https://forums.couchbase.com/c/n1ql . StackOverflow is great for one-and-done questions, but the forum is better for extended interactions.

SequelizeJS Order by renamed attribute

I came across an exception, cannot find solution anywhere. It looks like a bug of sequelizeJs.
Here is the question:
I have two model
module.exports = db.sequelize.define('user', {
// id is added automatically
createdAt: {
type: Sequelize.DATE,
field: 'created_at'
}
})
module.exports = db.sequelize.define('question', {
// id is added automatically
userId: {
type: Sequelize.BIGINT,
field: 'user_id'
}
})
User.hasMany(Question,{foreignKey:'userId',as:'questions'})
then I have a query:
models.User.findAll({
limit:limit,
order:[['createdAt','DESC']],
include : [{
{
model:models.Question,
as:"questions"
},]
})
I got an exception saying: "SequelizeDatabaseError: Unknown column 'user.created_at' in 'order clause'"
Here is the generated sql code:
SELECT ***** FROM (SELECT user.id, user.created_at AS createdAt FROM user AS user ORDER BY user.created_at DESC LIMIT 10) AS user LEFT OUTER JOIN question AS questions ON user.id = questions.user_id ORDER BY user.created_at DESC
It seems that order by is applied twice. At the second time, the 'created_at' has been renamed as 'createdAt'. That's why db complains 'cannot find user.created_at'.
My question is how to fix this problem so that I can run my query. I cannot find any available solution. And why Sequelize apply 'order by' twice? it looks like unnecessary.
This is an open issue it seems
See GitHub
However, you could try explicitly stating the attributes you want to query for on both models.
There is no issue with sequelize ,
Issue in below code : there one extra { inside the include block , remove that first
models.User.findAll({
limit:limit,
order:[['createdAt','DESC']],
include : [{
{
model:models.Question,
as:"questions"
},]
})
Query is 100% correct, I have executed the same way and its working fine from both side (sequelize and sql).
Issue you may have is :
your user table must have createdAt field named created_at
inside models.User.findAll() you might have used attributes and
forgot to add created_at
Try to run query inside your sql, and check for the error
SELECT *****
FROM (SELECT user.id, user.created_at AS createdAt FROM user AS user ORDER BY user.created_at DESC LIMIT 10)
AS user
LEFT OUTER JOIN question AS questions ON user.id = questions.user_id
ORDER BY user.created_at DESC
And you will figure it out whats wrong with the query

Rails ActiveRecord Join Query With conditions

I have following SQL Query:
SELECT campaigns.* , campaign_countries.points, offers.image
FROM campaigns
JOIN campaign_countries ON campaigns.id = campaign_countries.campaign_id
JOIN countries ON campaign_countries.country_id = countries.id
JOIN offers ON campaigns.offer_id = offers.id
WHERE countries.code = 'US'
This works perfectly well. I want its rails active record version some thing like:
Campaign.includes(campaign_countries: :country).where(countries: {code: "US"})
Above code runs more or less correct query (did not try to include offers table), issue is returned result is collection of Campaign objects so obviously it does not include Points
My tables are:
campaigns --HAS_MANY--< campaign_countries --BELONGS_TO--< countries
campaigns --BELONGS_TO--> offers
Any suggestions to write AR version of this SQL? I don't want to use SQL statement in my code.
I some how got this working without SQL but surely its poor man's solution:
in my controller I have:
campaigns = Campaign.includes(campaign_countries: :country).where(countries: {code: country.to_s})
render :json => campaigns.to_json(:country => country)
in campaign model:
def points_for_country country
CampaignCountry.joins(:campaign, :country).where(countries: {code: country}, campaigns: {id: self.id}).first
end
def as_json options={}
json = {
id: id,
cid: cid,
name: name,
offer: offer,
points_details: options[:country] ? points_for_country(options[:country]) : ""
}
end
and in campaign_countries model:
def as_json options={}
json = {
face_value: face_value,
actual_value: actual_value,
points: points
}
end
Why this is not good solution? because it invokes too many queries:
1. It invokes query when first join is performed to get list of campaigns specific to country
2. For each campaign found in first query it will invoke one more query on campaign_countries table to get Points for that campaign and country.
This is bad, Bad and BAD solution. Any suggestions to improve this?
If You have campaign, You can use campaign.campaign_countries to get associated campaign_countries and just get points from them.
> campaign.campaign_countries.map(&:points)
=> [1,2,3,4,5]
Similarly You will be able to get image from offers relation.
EDIT:
Ok, I guess now I know what's going on. You can use joins with select to get object with attached fields from join tables.
cs = Campaign.joins(campaign_countries: :country).joins(:offers).select('campaigns.*, campaign_countries.points, offers.image').where(countries: {code: "US"})
You can than reference additional fields by their name on Campaign object
cs.first.points
cs.first.image
But be sure, that additional column names do not overlap with some primary table fields or object methods.
EDIT 2:
After some more research I came to conclusion that my first version was actually correct for this case. I will use my own console as example.
> u = User.includes(:orders => :cart).where(:carts => { :id => [5168, 5167] }).first
> u.orders.length # no query is performed
=> 2
> u.orders.count # count query is performed
=> 5
So when You use includes with condition on country, in campaign_countries are stored only campaign_countries that fulfill Your condition.
Try this:
Campaign.joins( [{ :campaign_countries => :countries}, :offers]).where('`countries`.`code` = ?', "US")