I'm trying to determine if it's possible to write rules for Postgres to dynamically alter queries at runtime.
Essentially, I have a multitenancy solution that uses a mapping table to determine if a row is owned by a tenant. I've simplified the examples below for easy reading.
In these examples, "car" table is scoped to a particular tenant by a carmapping table which has foreign keys scoped_model_id pointing at the car table and scoping_model_pk pointing at the tenant/customer/whatever model that can generically be considered a "scoping model" used to scope various tables in the application.
Example scoped query:
SELECT "car"."id" FROM "car" WHERE
("car"."id" IN (SELECT U0."scoped_model_id" FROM "carmapping" U0 WHERE U0."scoping_model_pk" IN (1));
Original query:
SELECT "car"."id" FROM "car";
Is there a way to use rules to dynamically add these WHERE clauses?
It would need to be capable of determining what scoping_model_pk is active in the current request - whether that is passed to Postgres as an additional piece of metadata for all db cursor connections or embedded as a noop string somehow within the SQL statement itself. And will also need to determine what the mapping table is for the model currently targeted by the SELECT statement - in this case, carmapping.
We can do this easily at the application level within the ORM, but it would be best to do at the db level for consistency (because we have some code not using the ORM and don't want to have to remember to update any raw SQL code).
Related
Is there a method to automatically join tables that have primary to foreign relationship rather then designate joining on those values?
The out and out answer is "no" - no RDBMS I know of will allow you to get away with not specifying columns in an ON clause intended to join two tables in a non-cartesian fashion, but it might not matter...
...because typically multi tier applications these days are built with data access libraries that DO take into account the relationships defined in a database. Picking on something like entity framework, if your database exists already, then you can scaffold a context in EF from it, and it will make a set of objects that obey the relationships in the frontend code side of things
Technically, you'll never write an ON clause yourself, because if you say something to EF like:
context.Customers.Find(c => c.id = 1) //this finds a customer
.Orders //this gets all the customer's orders
.Where(o => o.date> DateTIme.UtcNow.AddMonths(-1)); //this filters the orders
You've got all the orders raised by customer id 1 in the last month, without writing a single ON clause yourself... EF has, behind the scenes, written it but in the spirit of your question where there are tables related by relation, we've used a framework that uses that relation to relate the data for the purposes thtat the frontend put it to.. All you have to do is use the data access library that does this, if you have an aversion to writing ON clauses yourself :)
It's a virtual certaintythat there will be some similar ORM/mapping/data access library for your front end language of choice - I just picked on EF in C# because it's what I know. If you're after scouting out what's out there, google for {language of choice} ORM (if you're using an OO language) - you mentioned python,. seems SQLAlchemy is a popular one (but note, SO answers are not for recommending particular softwares)
If you mean can you write a JOIN at query time that doesn't need an ON clause, then no.
There is no way to do this in SQL Server.
I am not sure if you are aware of dbForge; it may help. It recognises joinable tables automatically in following cases:
The database contains information that specifies that the tables are related.
If two columns, one in each table, have the same name and data type.
Forge Studio detects that a search condition (e.g. the WHERE clause) is actually a join condition.
Context
I have three tables in my SQL Server database: 1) School, 2) College, 3) University.
Then I have another table: Tags.
Each of the three tables (School, College, University) can have Tags associated with them. For which purpose I have three association tables: SchoolTags, CollegeTags, UniversityTags.
Problem
I am trying to create a user-defined function that will take the name of association table as parameter (i.e. 'SchoolTags') and the Id of the entity (school/college/university) and will return a list of tags associated with that entityId.
The issue I am having is I have got to join Tags with a table whose name will come in as parameter. For that I am creating a dynamic query. And we can not run dynamic queries in SQL Server user-defined functions.
Question
Any idea how can that be acheived?
Note: I want separate association tables as I have created and do not want to convert them into a generic association table and I do not want to add If-Else based on table names in my function so that if a new association table is created, I do not need to update my function.
I am using Microsoft SQL Server.
Whatever language you are using, you would probably use if:
begin
if table = 'school' then
begin
. . .
end;
else if table = 'college' then
. . .
end;
The exact syntax depends on the scripting language for the database you are using.
What you desire is impossible. You cannot pass a table name as a parameter to a UDF and use dynamic sql in the UDF to then create and execute a statement that is specific to the table passed as the argument. You already know that you have no choice but to use if-else statements in your UDF to achieve your goal - it is your pipe-dream of "never having to update (or verify) your code when the schema changes" (yes - I rephrased it to make your issue more obvious) that is a problem.
There are likely to be other ways of implementing some useful functionality - but I suggest that you are thinking too far ahead and trying to implement generic functions without a clear purpose. And that is a very difficult and trouble-prone path that requires sophisticated tsql skills.
And to re-iterate the prior responses, you have a schema problem. You purposely created three different entities - and now you want a common function to use with any of them. So before you spend much time on this particular aspect, you should take some time to think carefully about how you intend to use (i.e., write queries against) these tables. If you find yourself using unions frequently to combine these entities into a common resultset, then you have might have a mismatch between your actual business and your model (schema) of it.
Consider normalizing your database in related, logical groupings for one EducationInstitution table and one JoinEducTags table. Those tables sound like they maintain the same structure but of different typology and hence should be saved in one table with different Type field for School, College, University, etc.
Then, add necessary constraints, primary/foreign keys for the one-to-many relationship between all three sets:
You never want to keep restructuring your schema (i.e., add tables) for each new type. With this approach, your user-defined function would just need to receive value parameters not identifiers like tables to be run in dynamic querying. Finally, this approach scales better with efficient storage. And as you will see normalization saves on complex querying.
I have multiple database files which exist in multiple locations with exactly similar structure. I understand the attach function can be used to connect multiple files to one database connection, however, this treats them as seperate databases. I want to do something like:
SELECT uid, name FROM ALL_DATABASES.Users;
Also,
SELECT uid, name FROM DB1.Users UNION SELECT uid, name FROM DB2.Users ;
is NOT a valid answer because I have an arbitrary number of database files that I need to merge. Lastly, the database files, must stay seperate. anyone know how to accomplish this?
EDIT: an answer gave me the idea: would it be possible to create a view which is a combination of all the different tables? Is it possible to query for all database files and which databases they 'mount' and then use that inside the view query to create the 'master table'?
Because SQLite imposes a limit on the number of databases that can be attached at one time, there is no way to do what you want in a single query.
If the number can be guaranteed to be within SQLite's limit (which violates the definition of "arbitrary"), there's nothing that prevents you from generating a query with the right set of UNIONs at the time you need to execute it.
To support truly arbitrary numbers of tables, your only real option is to create a table in an unrelated database and repeatedly INSERT rows from each candidate:
ATTACH DATABASE '/path/to/candidate/database' AS candidate;
INSERT INTO some_table (uid, name) SELECT uid, name FROM candidate.User;
DETACH DATABASE candidate;
Some cleverness in the schema would take care of this.
You will generally have 2 types of tables: reference tables, and dynamic tables.
Reference tables have the same content across all databases, for example country codes, department codes, etc.
Dynamic data is data that will be unique to each DB, for example time series, sales statistics,etc.
The reference data should be maintained in a master DB, and replicated to the dynamic databases after changes.
The dynamic tables should all have a column for DB_ID, which would be part of a compound primary key, for example your time series might use db_id,measurement_id,time_stamp. You could also use a hash on DB_ID to generate primary keys, use same pk generator for all tables in DB. When merging these from different DBS , the data will be unique.
So you will have 3 types of databases:
Reference master -> replicated to all others
individual dynamic -> replicated to full dynamic
full dynamic -> replicated from reference master and all individual dynamic.
Then, it is up to you how you will do this replication, pseudo-realtime or brute force, truncate and rebuild the full dynamic every day or as needed.
I have a scenario where I want to persist document info record to a table specific to the typo of document, rather than a generic table for all records.
For example, records for Invoices will be stored in dbo.Doc_1000 and records for Receipts will be stored in dbo.Doc_2000 where 1000 and 2000 are id autogenerate and store in well-known table (dbo.TypeOfDoc.
Furthermore each dbo.Doc.xxx table have a group of system column (always the same) and could have a group of dynamic column (metadata).
Tables dbo.Doc.xxx and eventually dynamic column are clearly created at runtime.
If this is possible with NHibernate???
Thanks.
hope that I got your point. I am currently looking for a solution for a problem that looks similar. I want to integrate a feature in my application where the admin user can design an entity at runtime.
As far as I know, once the SessionFactory is configured and ready to use, there is no way to modify the mapping used by nhibernate. If you want to use a customized table structure that is configured, created and modified at runtime, you should have a place where a corresponding mapping lives, e.g. as a nhibernate mapping xml file and you have to set up a new SessionFactory each time you change the database model to reflect these changes.
I'm designing this collection of classes and abstract (MustInherit) classes…
This is the database table where I'm going to store all this…
As far as the Microsoft SQL Server database knows, those are all nullable ("Allow Nulls") columns.
But really, that depends on the class stored there: LinkNode, HtmlPageNode, or CodePageNode.
Rules might look like this...
How do I enforce such data integrity rules within my database?
UPDATE: Regarding this single-table design...
I'm still trying to zero in on a final architecture.
I initially started with many small tables with almost zero nullalbe fields.
Which is the best database schema for my navigation?
And I learned about the LINQ to SQL IsDiscriminator property.
What’s the best way to handle one-to-one relationships in SQL?
But then I learned that LINQ to SQL only supports single table inheritance.
Can a LINQ to SQL IsDiscriminator column NOT inherit?
Now I'm trying to handle it with a collection of classes and abstract classes.
Please help me with my .NET abstract classes.
Use CHECK constraints on the table. These allow you to use any kind of boolean logic (including on other values in the table) to allow/reject the data.
From the Books Online site:
You can create a CHECK constraint with
any logical (Boolean) expression that
returns TRUE or FALSE based on the
logical operators. For the previous
example, the logical expression is:
salary >= 15000 AND salary <= 100000.
It looks like you are attempting the Single Table Inheritance pattern, this is a pattern covered by the Object-Relational Structural Patterns section of the book Patterns of Enterprise Application Architecture.
I would recommend the Class Table Inheritance or Concrete Table Inheritance patterns if you wish to enforce data integrity via SQL table constraints.
Though it wouldn't be my first suggestion, you could still use Single Table Inheritance and just enforce the constraints via a Stored Procedure.
You can set up some insert/update triggers. Just check if these fields are null or notnull, and reject insert/update operation if needed. This is a good solution if you want to store all the data in the same table.
You can create also create a unique table for each classes as well.
Have a unique table for each type of node.
Why not just make the class you're building enforce the data integrity for its own type?
EDIT
In that case, you can either a) use logical constraints (see below) or b) stored procedures to do inserts/edits (a good idea regardless) or c) again, just make the class enforce data integrity.
A mixture of C & B would be the course of events I take. I would have unique stored procedures for add/edits for each node type (i.e. Insert_Update_NodeType) as well as make the class perform data validation before saving data.
Personally I always insist on putting data integrity code on the table itself either via a trigger or a check constraint. The reason why is that you cannot guarantee that only the user interface will update insert or delete records. Nor can you guarantee that someone might not write a second sp to get around the constraints in the orginal sp without understanding the actual data integrity rules or even write it because he or she is unaware of the existence of the sp with the rules. Tables are often affected by DTS or SSIS packages, dynamic queries from the user interface or through Query analyzer or the query window, or even by scheduled jobs that run code. If you do not put the data integrity code at the table level, sooner or later your data will not have integrity.
It's probably not the answer you want to hear, but the best way to avoid logical inconsistencies, you really want to look at database normalisation
Stephen's answer is the best. But if you MUST, you could add a check constraint the HtmlOrCode column and the other columns which need to change.
I am not that familiar with SQL Server, but I know with Oracle you can specify Constraints that you could use to do what you are looking for. I am pretty sure you can define constraints in SQL server also though.
EDIT: I found this link that seems to have a lot information, kind of long but may be worth a read.
Enforcing Data Integrity in Databases
Basically, there are four primary types of data integrity: entity, domain, referential and user-defined.
Entity integrity applies at the row level; domain integrity applies at the column level, and referential integrity applies at the table level.
Entity Integrity ensures a table does not have any duplicate rows and is uniquely identified.
Domain Integrity requires that a set of data values fall within a specific range (domain) in order to be valid. In other words, domain integrity defines the permissible entries for a given column by restricting the data type, format, or range of possible values.
Referential Integrity is concerned with keeping the relationships between tables synchronized.
#Zack: You can also check out this blog to read more details about data integrity enforcement, here- https://www.bugraptors.com/what-is-data-integrity/
SQL Server doesn't know anything about your classes. I think that you'll have to enforce this by using a Factory class that constructs/deconstructs all these for you and makes sure that you're passing the right values depending upon the type.
Technically this is not "enforcing the rules in the database" but I don't think that this can be done in a single table. Fields either accept nulls or they don't.
Another idea could be to explore SQL Functions and Stored Procedures that do the same thing. BUt you cannot enforce a field to be NOT NULL for one record and NULL for the next one. That's your Business Layer / Factory job.
Have you tried NHibernate? It's much more matured product than Entity Framework. It's free.