In sails I need function to get rows by referenced table. Something like
Child.find({ parent.age: 30 })
or in SQL language
SELECT child.* FROM child JOIN parent ON child.parent_id = parent.id WHERE parent.age = 30
So far I have written uggly function which does the work in 3 steps:
get_children_in_celebration: function(req, res) {
// 1. Get parend ids
Parent.find({ select: ['id'], age: 30}).exec(function(err_parents, res_parents) {
// 2. Collect them to array suitable for next query
var parent_ids = [];
for(var i = 0; i < res_parents.length; i++) {
parent_ids[i] = res_parents[i].id;
}
// Get their children
Child.find({ parent_id: parent_ids }).exec(function(err_children, res_children) {
return res.json(res_children);
});
});
},
The first query returns ~5000 parents and therefore together it means heavy load for db. Does sails offer any nicer solution?
No, Sails does not support filtering on associated table. I had the similar issue, had used raw queries for the same.
Model.query("select * from table", function (error, response) {})
Related
I am having multiple nested where conditions and want to generate them without too much code duplication with typeORM.
The SQL where condition should be something like this:
WHERE "Table"."id" = $1
AND
"Table"."notAvailable" IS NULL
AND
(
"Table"."date" > $2
OR
(
"Table"."date" = $2
AND
"Table"."myId" > $3
)
)
AND
(
"Table"."created" = $2
OR
"Table"."updated" = $4
)
AND
(
"Table"."text" ilike '%search%'
OR
"Table"."name" ilike '%search%'
)
But with the FindConditions it seems not to be possible to make them nested and so I have to use all possible combinations of AND in an FindConditions array. And it isn't possible to split it to .where() and .andWhere() cause andWhere can't use an Object Literal.
Is there another possibility to achieve this query with typeORM without using Raw SQL?
When using the queryBuilder I would recommend using Brackets
as stated in the Typeorm doc: https://typeorm.io/#/select-query-builder/adding-where-expression
You could do something like:
createQueryBuilder("user")
.where("user.registered = :registered", { registered: true })
.andWhere(new Brackets(qb => {
qb.where("user.firstName = :firstName", { firstName: "Timber" })
.orWhere("user.lastName = :lastName", { lastName: "Saw" })
}))
that will result with:
SELECT ...
FROM users user
WHERE user.registered = true
AND (user.firstName = 'Timber' OR user.lastName = 'Saw')
I think you are mixing 2 ways of retrieving entities from TypeORM, find from the repository and the query builder. The FindConditions are used in the find function. The andWhere function is use by the query builder. When building more complex queries it is generally better/easier to use the query builder.
Query builder
When using the query build you got much more freedom to make sure the query is what you need it to be. With the where you are free to add any SQL as you please:
const desiredEntity = await connection
.getRepository(User)
.createQueryBuilder("user")
.where("user.id = :id", { id: 1 })
.andWhere("user.date > :date OR (user.date = :date AND user.myId = :myId)",
{
date: specificCreatedAtDate,
myId: mysteryId,
})
.getOne();
Note that depending on your used database the actual SQL that you use here needs to be compatible. With that could also come a possible draw back of using this method. You will tie your project to a specific database. Make sure to read up about the aliases for tables you can set if you are using relations this would be handy.
Repository
You already saw that this is much less comfortable. This is because the find function or more specific the findOptions are using objects to build the where clause. This makes is harder to implement a proper interface to implement nested AND and OR clauses side by side. There for (I assume) they have chosen to split AND and OR clauses. This makes the interface much more declarative and means the you have to pull your OR clauses to the top:
const desiredEntity = await repository.find({
where: [{
id: id,
notAvailable: Not(IsNull()),
date: MoreThan(date)
},{
id: id,
notAvailable: Not(IsNull()),
date: date
myId: myId
}]
})
I cannot imagin looking a the size of the desired query that this code would be very performant.
Alternatively you could use the Raw find helper. This would require you to rewrite your clause per field, since you will only get access to the one alias at a time. You could guess the column names or aliases but this would be very poor practice and very unstable since you cannot directly control this easily.
if you want to nest andWhere statements if a condition is meet here is an example:
async getTasks(filterDto: GetTasksFilterDto, user: User): Promise<Task[]> {
const { status, search } = filterDto;
/* create a query using the query builder */
// task is what refer to the Task entity
const query = this.createQueryBuilder('task');
// only get the tasks that belong to the user
query.where('task.userId = :userId', { userId: user.id });
/* if status is defined then add a where clause to the query */
if (status) {
// :<variable-name> is a placeholder for the second object key value pair
query.andWhere('task.status = :status', { status });
}
/* if search is defined then add a where clause to the query */
if (search) {
query.andWhere(
/*
LIKE: find a similar match (doesn't have to be exact)
- https://www.w3schools.com/sql/sql_like.asp
Lower is a sql method
- https://www.w3schools.com/sql/func_sqlserver_lower.asp
* bug: search by pass where userId; fix: () whole addWhere statement
because andWhere stiches the where class together, add () to make andWhere with or and like into a single where statement
*/
'(LOWER(task.title) LIKE LOWER(:search) OR LOWER(task.description) LIKE LOWER(:search))',
// :search is like a param variable, and the search object is the key value pair. Both have to match
{ search: `%${search}%` },
);
}
/* execute the query
- getMany means that you are expecting an array of results
*/
let tasks;
try {
tasks = await query.getMany();
} catch (error) {
this.logger.error(
`Failed to get tasks for user "${
user.username
}", Filters: ${JSON.stringify(filterDto)}`,
error.stack,
);
throw new InternalServerErrorException();
}
return tasks;
}
I have a list of
{
date: specificCreatedAtDate,
userId: mysteryId
}
My solution is
.andWhere(
new Brackets((qb) => {
qb.where(
'userTable.date = :date0 AND userTable.type = :userId0',
{
date0: dates[0].date,
userId0: dates[0].type,
}
);
for (let i = 1; i < dates.length; i++) {
qb.orWhere(
`userTable.date = :date${i} AND userTable.userId = :userId${i}`,
{
[`date${i}`]: dates[i].date,
[`userId${i}`]: dates[i].userId,
}
);
}
})
)
That will produce something similar
const userEntity = await repository.find({
where: [{
userId: id0,
date: date0
},{
id: id1,
userId: date1
}
....
]
})
Hello I have and Array with objects, each object have atributes that I need for an select:
In this case it is the result from another consult with typeorm
" const CompaniesRelation: Array = await getRepository(CompanyRelation).find({ where:{ UserId: data.UserId, IsActive: true} });"
Companies: Array = [{CompanyId="a"}{CompanyId="b"}{CompanyId="c"}];
I need to make an select of all the data that matches with the Ids that are into Companies so for that I need to make an SQL like it:
const CompanyData: Array = SELECT *
FROM Company
INNER JOIN Company.CompanyId = CompaniesRelation[].CompanyId;
but it throw me error in typing, ¿how can I acces to each objetc into the array for make that match?
At the final I should traduce it sql to typeOrm, but I new and solving first in SQL it should help me to traduce to typeorm
Okay great, let us consider what we have to work with right:
So first we have a statement that gets a list of companies like so:
const CompaniesRelation: Array = await getRepository(CompanyRelation).find({
where: {
UserId: data.UserId,
IsActive: true
}
});
which ends up with something like this:
[ { CompanyId: 'a' }, { CompanyId: 'b' }, { CompanyId: 'c' } ]
Now we want to get a list of companies from an SQL DB with these Company IDs.
So the query should look like this:
// so first we re map the relation to an array of strings...
const ids: Array<string> = CompaniesRelation.map(c => c.CompanyId);
// then use it in the query, note the string interpolation for the query
const query: string = `SELECT * FROM Company WHERE CompanyId IN(${JSON.stringify(ids).slice(1, -1)});`;
I don't think this will cover the scope of the problem you have, I hope it helps though...feel free to ask
I've been trying to structure the relations in my database for more efficient querying and joins but after following the guides for '.belongsToMany', '.through' and '.belongsTo' I'm now getting empty results.
I've got a Sound model and a Keyword model which I want to model with a many-to-many relationship (each Sound can have multiple Keywords, and each Keyword can be related to multiple sounds). Based on the documentation '.belongsToMany' would be the relation to use here.
I've set up my models as follows, using a 'sound_keyword' relational table/SoundKeyword relational model (where each entry has it's own unique 'id', a 'soundID', and a 'keywordID'):
var Sound = bookshelf.Model.extend({
tableName: 'sounds',
keywords: function () {
return this.belongsToMany(Keyword, 'sound_keyword', 'id', 'id').through(SoundKeyword, 'id', 'soundID');
},
});
var Keyword = bookshelf.Model.extend({
tableName: 'keywords',
sounds: function () {
return this.belongsToMany(Sound, 'sound_keyword', 'id', 'id').through(SoundKeyword, 'id', 'keywordID');
}
});
where:
var SoundKeyword = bookshelf.Model.extend({
tableName: 'sound_keyword',
sound: function () {
return this.belongsTo(Sound, 'soundID');
},
keyword: function () {
return this.belongsTo(Keyword, 'keywordID');
}
});
From what I've read in the docs and the BookshelfJS GitHub page the above seems to be correct. Despite this when I run the following query I'm getting an empty result set (the Sound in question is related to 3 Keywords in the DB):
var results = await Sound
.where('id', soundID)
.fetch({
withRelated: ['keywords']
})
.then((result) => {
console.log(JSON.stringify(result.related('keywords')));
})
Where am I going wrong with this? Are the relationships not set up correctly (Possibly wrong foreign keys?)? Am I fetching related models incorrectly?
Happy to provide the Knex setup as needed.
UPDATED EDIT:
I had been using the Model-Registry Plugin from the start and had forgotten about it. As it turns out, while the below syntax is correct, it prefers syntax similar to the following (i.e. lowercase 'model', dropping the '.extends' and putting model names in quotes):
var Sound = bookshelf.model('Sound',{
tableName: 'sounds',
keywords: function () {
return this.belongsToMany('Keyword', 'sound_keyword', 'soundID', 'keywordID');
},
});
var Keyword = bookshelf.model('Keyword',{
tableName: 'keywords',
sounds: function () {
return this.belongsToMany('Sound', 'sound_keyword', 'keywordID', 'soundID');
}
});
Hope this can be of help to others.
Seems like removing the '.through' relation and changing the IDs in the '.belongsToMany' call did the trick (as below), though I'm not entirely sure why (the docs seem to imply belongsToMany and .through work well together - possibly redundant?)
var Sound = bookshelf.Model.extend({
tableName: 'sounds',
keywords: function () {
return this.belongsToMany(Keyword, 'sound_keyword', 'soundID', 'keywordID');
},
});
var Keyword = bookshelf.Model.extend({
tableName: 'keywords',
sounds: function () {
return this.belongsToMany(Sound, 'sound_keyword', 'keywordID', 'soundID');
}
});
I did try my original code with soundID and keywordId instead of 'id' (as below), but without the .through relation and that gave the same empty results.
We have an asp.net-core website which handles users search as follows:
public async Task<ICollection<UserSearchResult>> SearchForUser(string name, int page)
{
return await db.ApplicationUsers.Where(u => u.Name.Contains(name) && !u.Deleted && u.AppearInSearch)
.OrderByDescending(u => u.Verified)
.Skip(page * recordsInPage)
.Take(recordsInPage)
.Select(u => new UserSearchResult()
{
Name = u.Name,
Verified = u.Verified,
PhotoURL = u.PhotoURL,
UserID = u.Id,
Subdomain = u.Subdomain
}).ToListAsync();
}
The query translates to something similar to the following:
SELECT [t].[Name], [t].[Verified], [t].[PhotoURL], [t].[Id], [t].[Subdomain] FROM (SELECT [u0].* FROM [AspNetUsers] AS [u0] WHERE (((CHARINDEX('khaled', [u0].[Name]) > 0) OR ('khaled' = N'')) AND ([u0].[Deleted] = 0)) AND ([u0].[AppearInSearch] = 1) ORDER BY [u0].[Verified] DESC OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY ) AS [t]
In Client-Side we use typeahead and bloodhound as follows:
engine = new Bloodhound({
identify: function (user) {
return user.UserID;
},
queryTokenizer: Bloodhound.tokenizers.whitespace,
datumTokenizer: Bloodhound.tokenizers.obj.whitespace('name'),
dupDetector: function (a, b) { return a.UserID === b.UserID; },
remote: {
cache: false,
url: '/account/Search?name=%QUERY&page=0',
wildcard: '%QUERY'
}
});
and we configure typeahead as follows:
$('#demo-input').typeahead(
{
hint: $('.Typeahead-hint'),
menu: $('.Typeahead-menu'),
minLength: 3,
classNames:
{
open: 'is-open',
empty: 'is-empty',
cursor: 'is-active',
suggestion: 'Typeahead-suggestion',
selectable: 'Typeahead-selectable'
}
},
{
source: engineWithDefaults,
displayKey: 'name',
templates:
{
suggestion: template,
empty: empty,
footer: all
},
limit: 5
})
The search works just find on localhost and the query runs great as a sql query.
I have also created an index on Verified and cut the speed to 1 second or less.
Our website has millions of registered users and the problem is that as soon as we make search available for all users the DTU percentage on Azure goes to 100% and the queries timeout.
We also have a redis cache to speed-up similar queries but this didn't help us with this issue.
Your support is appreciated :)
It's quite likely to be u.Name.Contains(name) i.e. CHARINDEX('khaled', [u0].[Name]) > 0 which will have to scan the entire table or, at best, the index. That will be slow and there's not much you can do about it.
If you have a large bias to deleted or appearInSearch you might be able to use a conditional index but these types of searches are notoriously slow. You will need some special constructs to make this work.
with CouchDB is possible do queries "like" SQL. http://guide.couchdb.org/draft/cookbook.html says that
How you would do this in SQL:
SELECT field FROM table WHERE value="searchterm"
How you can do this in CouchDB:
Use case: get a result (which can be a record or set of records) associated with a key ("searchterm").
To look something up quickly, regardless of the storage mechanism, an index is needed. An index is a data structure optimized for quick search and retrieval. CouchDB’s map result is stored in such an index, which happens to be a B+ tree.
To look up a value by "searchterm", we need to put all values into the key of a view. All we need is a simple map function:
function(doc) {
if(doc.value) {
emit(doc.value, null);
}
}
This creates a list of documents that have a value field sorted by the data in the value field. To find all the records that match "searchterm", we query the view and specify the search term as a query parameter:
/database/_design/application/_view/viewname?key="searchterm"
how can I do this with PouchDB? the API provide methods to create temp view, but how I can personalize the get request with key="searchterm"?
You just add your attribute settings to the options object:
var searchterm = "boop";
db.query({map: function(doc) {
if(doc.value) {
emit(doc.value, null);
}
}, { key: searchterm }, function(err, res) { ... });
see http://pouchdb.com/api.html#query_database for more info
using regex
import PouchDB from 'pouchdb';
import PouchDBFind from 'pouchdb-find';
...
PouchDB.plugin(PouchDBFind)
const db = new PouchDB(dbName);
db.createIndex({index: {fields: ['description']}})
....
const {docs, warning} = await db.find({selector: { description: { $regex: /OVO/}}})