I've got an Azure web app serving as a restful API. This back end defines the models being used by my system: Users, Teams, Projects, Items. A User can join many Teams, and the Team-->Project-->Item structure is a Grandparent-->Parent-->Child relationship.
I want to set up a daily email digest of updates for each User. I want this to be run in an Azure web job in the background, so as not to burden the customer facing API server.
When writing code in the API's controllers, I can write easy queries like this to get all the items that belong to a certain team (because the models and their hierarchical relationships are defined in the API):
var items = await (from x in db.Items
where x.Project.Team.TeamId == teamId
select x).ToListAsync();
However, in the web job, I can't do that - there's no models to speak of. I see a couple options:
Option A) Somehow use (or re-create) the models (and the DB context?) in the web job. If this is possible (and not a terrible idea), I think this is what I want.
Option B) Make peace with many, verbose SQL queries in the web job. I can query the User table for what Teams they're part of. Then, for each Team, I can query the Project table for a set of Projects. Then, for each Project, I can query the Items table for the Items in each project.
Is there a way to make a magical SQL query that works more efficiently than Option B's many calls to the database? I've tried things like the following, but haven't found a working scheme:
SELECT * FROM Items WHERE Projects WHERE Teams = teamId;
The best way from my point of view is Opition C) - refactor your solution and create separate project that will contain data context, models and all migrations. After that you can reference this project from your main app and from WebJob. You can also move some core database-related functions into this project to make them accessible from other projects.
Well, it appears what I wanted was a JOIN: What's the difference between INNER JOIN, LEFT JOIN, RIGHT JOIN and FULL JOIN?
I've really only ever used LINQ to pull from SQL databases, now I know how to do it the old fashioned, more verbose, less intuitive way :)
Here's the code that I'm using to pull all the items that belong to a given team:
SqlCommand command = new SqlCommand();
command.CommandText = #"SELECT Items.* FROM Items
JOIN Projects
ON Projects.ProjectId = Items.ProjectId
JOIN Teams
ON Teams.TeamId = Projects.TeamId
WHERE Teams.TeamId = '" + teamId.ToString() + "'";
Related
I want to retrieve Asset Hierarchy of Twin Graph in Azure Digital Twins using REST API, is there any way to do so?
The situation you described in your question (and the comments) can be solved by writing multiple queries. To get the full graph as a hierarchy, you will first need to query the top-level items (in your case organisations) and then you can either run a query for every organisation or fit it all into one query. I'm making some assumptions about your hierarchy in these examples, you will need to adapt them to your ontology.
Getting all the organisations:
SELECT ROOM FROM DIGITALTWINS DT WHERE IS_OF_MODEL(DT, 'dtmi:example:organisation;1', exact)
Getting the hierarchy for one organisation:
SELECT Organisation, Factory, ProductionLine, Equipment
FROM DIGITALTWINS Organisation
JOIN Factory RELATED Organisation.contains
JOIN ProductionLine RELATED Factory.contains
JOIN Equipment RELATED ProductionLine.contains
WHERE Organisation.$dtId = 'Organisation1'
Getting the hierarchy for all previously found organisations:
SELECT Organisation, Factory, ProductionLine, Equipment
FROM DIGITALTWINS Organisation
JOIN Factory RELATED Organisation.contains
JOIN ProductionLine RELATED Factory.contains
JOIN Equipment RELATED ProductionLine.contains
WHERE Organisation.$dtId IN ['Organisation1', 'Organisation2', 'Organisation3', 'etc...']
There is a limitation of 5 JOIN statements to take into account. And both queries will return a row for every Equipment twin, so you will need to transform the result a bit.
In both the SDK and REST API, you can achieve this through the Query operation.
Consider a simple situation of users and subjects related with a pivot table named user_subjects, now if I have to fetch all the users who have opted for a certain subject. What is the best way to achieve this in AdonisJs 5. Following is the solution I found.
const subject = await Subject.find(req.params.id);
const users = await subject.related('users').query();
There is nothing wrong with the above approach but I think there is an extra query to fetch the entity and then load the relationship. Is there a more optimised way to achieve above?
Also to take the above example a bit further, if suppose users are related to classes through a pivot table user_classes, can that not be achieved using 1 query on Lucid Models and not using Database class?
This is the situation:
Say I have an application in which two entity types exist:
Company
Person
Moreover, Person has a reference to Company via Person.employer, which denotes the company a person is employed at.
In my application I am using repositories to separate the database operations from my business-model related services: I have a PersonRepository.findOne(id) method to retrieve a Person entity and a CompanyRepository.findOne(id) method to retrieve a Company. So far so good.
This is the dilemma:
Now if I make a call to PersonRepository.findOne(id) to fetch a Person entity, I also need to have a fully resolved Company included inline via the Person.employer property – and this is where I am facing the dilemma of having two implementation options that are both suboptimal:
Option A) Redundant queries throughout my repositories but less database round trips:
Within the PersonRepository I can build a query which selects the user and also selects the company in a single query – however, the select expression for the company is difficult and includes some joins in order to assemble the company correctly. The CompanyRepository already contains this logic to select the company and rewriting it in the UserRepository is redundant. Hence, ideally I only want the CompanyRepository to take care of the company selection logic in order to avoid having to code the same query expression redundantly in two repositories.
Option B): Separation of concerns without query-code redundancy but at the price of additional db roundtrips and repo-dependencies:
Within the PersonRepository I could reference the CompanyRepository to take care of fetching the Company object and then I would add this entity to the Person.employer property in the PersonRepository. This way, I kept the logic to query the company encapsulated inside the CompanyRepository by which a clean separation of concerns is achieved. The downside of this is that I make additional round trips to the database as two separate queries are executed by two repositories.
So generally speaking, what is the preferred way to deal with this dilemma?
Also, what is the preferred way to handle this situation in ASP.NET Core and EF Core?
Edit: To avoid opinion based answers I want to stress: I am not looking for a pros and cons of the two options presented above but rather striving for a solution that integrates the good parts of both options – because maybe I am just on the wrong track here with my two listed options. I am also fine with an answer that explains why there is no such integrative solution, so I can sleep better and move on.
In order to retrieve a company by ID you need to read Person's data, and fetch company ID from it. Hence if you would like to keep company-querying logic in a single place, you would end up with two round-trips - one to get company ID (along with whatever other attributes a Person has) and one more to get the company itself.
You could reuse the code that makes a company from DbDataReader, but the person+company query would presumably require joining to "forward" person's companyId to the Company query, so the text of these queries would have to be different.
You could have it both ways (one roundtrip, no repeated queries) if you move querying logic into stored procedures. This way your person_sp would execute company_sp, and return you all the relevant data. If necessary, your C# code would be able to harvest multi-part result set using reader.NextResult(). Now the "hand-off" of the company ID would happen on RDBMS side, eliminating the second round-trip. However, this approach would require maintaining stored procedures on RDBMS side, effectively shipping some repository logic out of your C# code base.
Is there a method to automatically join tables that have primary to foreign relationship rather then designate joining on those values?
The out and out answer is "no" - no RDBMS I know of will allow you to get away with not specifying columns in an ON clause intended to join two tables in a non-cartesian fashion, but it might not matter...
...because typically multi tier applications these days are built with data access libraries that DO take into account the relationships defined in a database. Picking on something like entity framework, if your database exists already, then you can scaffold a context in EF from it, and it will make a set of objects that obey the relationships in the frontend code side of things
Technically, you'll never write an ON clause yourself, because if you say something to EF like:
context.Customers.Find(c => c.id = 1) //this finds a customer
.Orders //this gets all the customer's orders
.Where(o => o.date> DateTIme.UtcNow.AddMonths(-1)); //this filters the orders
You've got all the orders raised by customer id 1 in the last month, without writing a single ON clause yourself... EF has, behind the scenes, written it but in the spirit of your question where there are tables related by relation, we've used a framework that uses that relation to relate the data for the purposes thtat the frontend put it to.. All you have to do is use the data access library that does this, if you have an aversion to writing ON clauses yourself :)
It's a virtual certaintythat there will be some similar ORM/mapping/data access library for your front end language of choice - I just picked on EF in C# because it's what I know. If you're after scouting out what's out there, google for {language of choice} ORM (if you're using an OO language) - you mentioned python,. seems SQLAlchemy is a popular one (but note, SO answers are not for recommending particular softwares)
If you mean can you write a JOIN at query time that doesn't need an ON clause, then no.
There is no way to do this in SQL Server.
I am not sure if you are aware of dbForge; it may help. It recognises joinable tables automatically in following cases:
The database contains information that specifies that the tables are related.
If two columns, one in each table, have the same name and data type.
Forge Studio detects that a search condition (e.g. the WHERE clause) is actually a join condition.
I'm using django for my project, its all going well except its table handling.
Autocreated tables are great for people who don't want to make raw queries and are not interested in adding and deleting columns. But for me, I want to have complete control over my database.
class Partner(models.Model):
software = models.ManyToManyField(Software)
This creates:
partner_partner
partner_partner_software
Second table to handle manytomany relation.
This kind of behaviour makes it complicated for me to observe and understand my table structure.
I think this allows me to do it:
partner = Partner.objects.get(pk=1)
software = partner.software
if I don't use manytomany like relations and instead of above usage,
partner = Partner.objects.get(pk=1)
software = Software.objects.get(name = partner.software)
You think using this kind of technique is the best choice in this case?
I used Django and I have complete control over my tables. I just set it up in a way that it lets Django handle the CMS part and CodeIgniter handles the site.