Using unique index for cross table constraint - sql

I'm pretty new to Oracle and database design in general but I have a question. I have two basic tables called employee and department. The employee table holds a salary property. Now I need to implement a new business rule: The total of all employee salaries in one department (an employee is part of a department) may not exceed 200k. I know this can be done with a materialized view, but can it also be done with unique indexes? If so, how would one approach this particular situation?


No, you can't use a unique index to enforce this sort of rule.
You could try to enforce this sort of rule via triggers but that tends to get rather difficult in a multiuser environment. You'd need to do things like lock the department row to ensure that only one session can be modifying employee information for a particular department at a time which tends to introduce significant scalability issues. It also tends to involve quite a bit of code to handle all the potential cases.


This can't be done with any indexes. But it is can be done using a trigger.
If you add a column maxDeptSalary to your departments table, then you can create a trigger on Insert and Update in which you
query your maxDeptSalary and get the value
run Sum on existing records in your employee table for that department
compare sum + new.value <= maxDeptSalary
Reject your record if criteria is not met
But the reality is, this is should be done on application level, not the database

Related

How do I enforce a total participation constraint on 1:N relationship in a relational database?

Employee_Department_ER
In this example, entity Department is managed by many employees and each Department must participate in such relationship (double lines indicate total participation). Entity Employee can manage one Department and each Employee may or may not participate in such relationship.
I'm wondering what is the correct way to map this into a relational schema in a database. I've done some search on Google and other posts but their answers still confuse me. One approach is to use
Employee(<attrs_for_employee>, department_id) // department_id is a foreign key that refers to Department's primary key
Department(<attrs_for_department>)
This approach clearly shows the 1:N relationship, but I don't see any total participation in the Department (Department might or might not be managed by an Employee).
If we extends Department instead,
Department(<attrs_for_department>, employee_id)
Employee(<attrs_for_employee>
This approach shows the total participation in the Department but loses the 1:N relationship.
Is there a correct way to show both the 1:N and total participation in relational schema? Is it necessary to create a new schema, Employee_Department that merges both of the primary keys like in a M:N relationship?

Use a database trigger. The trick here is that "a department must be managed by one or more employees" can't be directly represented in the model itself. It has to be in logic. The best logic is in a trigger. The exact code for your vendor product will vary, but the logic is this:
Put a BEFORE INSERT trigger on DEPARTMENT.
In that trigger, get the EMPLOYEE_ID of the incoming INSERT.
Go find that in the EMPLOYEE table.
If it is there, INSERT.
If it is not, throw a message saying that one EMPLOYEE must be added first, and that such an employee must be used in the record to be inserted in DEPARTMENT.
Put a BEFORE DELETE trigger on EMPLOYEE. This is to ensure you don't later orphan the DEPARTMENT.
In that trigger, ensure that DEPARTEMENT has at least two EMPLOYEEs, so that the one you're about to delete is not the last one.
If two or more, let the DELETE pass.
If just one, throw an error saying you can't DELETE the last EMPLOYEE.
Categorically, there are many rules that must be enforced in code that cannot be enforced in the model. Triggers are the way to go.
If you're on a platform that doesn't do triggers well or at all, you have to write this either in stored procedures run on a schedule, or in stand-alone code (e.g. Python, Java, etc.) and run on a schedule. Triggers are magical for data rules!

design database model and related sql queries

Develop the rdbms for the administrative structure of an organisation. Each employee belong to a certain dept and is associated with multiple projects. each manager is an employee who manages several projects as well as several employee. Each project executes for a certain duration. employees stay in organisations for a certain duration.
Queries:
find no of employees who has worked is every project
find max no of employees working at a time in project 'x'
find the unproductive managers who manages less than 5 project in last 1 year
find the dept whose employees handled maximum project in last 1 year
edit:
I am not able to decide how to deal with the time constraint in the last 3 queries.
I have made 3 tables:
EMPLOYEE with attributes: emp_id,name,dept,manager_id where emp_id is primary key and manager_id is self referential foreign key
PROJECT with p_id,p_name,manager_id where p_id is primary key
ALLOTMENT with emp_id,p_id where both attributes make a composite primary key
The above helps me answer the first query but how do I add the time constraints to answer the rest of the queries. Do I need date-time attribute or a simple duration attribute will work or something else is required here? please help.

ALLOTMENT is the intersection between EMPLOYEE and PROJECT. It records the employees working on a project.
However, employees join projects and leave projects. Projects grow and shrink in their resourcing. So clearly ALLOTMENT needs columns to indicate the time span of a particular assignment, say START_DATE and END_DATE.
Once you add those columns you will be able to answer the remaining questions. Some of them will remain tricky (especially 2) but at least you will have the information required.
Incidentally, you probably ought to have a DEPARTMENT table but you can write those queries without it. Also, in real life a PROJECT would have an initiation date and (we hope) a completion date. However they too are not required for the queries you have to write.

In the project table consider adding two columns start_date and duration| End_date of the project. I think this will be sufficient for you to work through the last 3 queries.
You can consider having another column No_of_Employees_under_project in project table, which is normalized and will dynamically reflect employees joining or leaving the project. This column should be added only after measuring the gain against the cost of normalization.

SQL table design advice

I am building a community site where logon will be by email and members will be able to change their name/nick name.
Do you think I should keep member name/nick name in my members table with other properties of member or create another table, write member name/nick name on that table and associate member’s id.
I am in favour of second option because, I think it would be faster to pull members name from it.
Is it right/better way?
Update: reason is for other table is that I need to pull username for different sections. For example forums. Wouldn't it be faster to query a small table for each username for each post in a from topic?

I would keep it one table and set a unique constraint on Email in that table.
I can't see a single advantage in adding another table.

Why do you think the second option would be faster?
If nickname is a required one-to-one relation to member ID the appropriate place to store them is in the same table. This is still a indexed single-record search so it should be more-or-less as fast as your other option.
In fact, this solution would probably be faster, since you could get the nickname in the same SELECT as you get the other information.
Update to answer the update to the question:
The second table isn't any smaller in terms of the number of rows. The main factors in a SQL search are 1) number of records in the table and 2) number of possible matches from the indexed part of the search.
In this case, the number of records in your smaller table would be exactly the same as the larger table. And the number of possible matching records returned by the index will always be 1 because the member ID is unique.
The number of columns in the table you're searching is generally irrelevant to the time taken to return the data (the number of column you actually list in the SELECT statement can have an effect, but that's the same no matter which table you're searching).
SQL databases are very, very good at finding data. Structure your data correctly and let the database worry about getting it back to. Premature optimization is, as they say, the root of all evil.

Go with the first option: keep the name/nick name in the members table. There's no need to introduce an additional table, and the overhead of a join that goes with it, in this case.

Yes, associating member's ID to the other properties is the right way to go.
You can simply create an index on name to speed up your queries.

Database design....storing relationship in one table, and the data in another table?

I'm looking at this company's database design, and would like to know the purpose of their design, ie store relationship in one table and the data in another, why do this?
They have this,
EMPLOYEE
Id (PK)
DepartmentId
EMPLOYEE_DATA
EmployeeId (PK)
First Name
Last Name
Position
etc...
Rather than this...
EMPLOYEE
Id (PK)
DepartmentId
First Name
Last Name
Position
etc...
...OR this...(employee can belong to many departments)
EMPLOYEE
Id (PK)
First Name
Last Name
etc...
EMPLOYEE_DEPARTMENT
Id
EmployeeId
DepartmentId
Position

That's a link table, or join table, or cross table.. lots of different names.
How would you assign an employee to two different departments with your design? You can't. You can only assign them to one.
With their design, they can assign the same ID to multiple departments by creating multiple records with the employee ID and different department ID's.
EDIT:
You need to be more specific about what you're asking. Your first question seemed to be asking what the purpose of mapping table was. Then you changed it, then you changed it again.. none of which makes much sense.
It seems now that you are asking what the better design is, which is a totally different question than what you originally asked. Please state specifically what question you want answered so we don't have to guess.
EDIT2:
Upon re-reading, if this is the actual design, then no.. It does not support multiple department id's. Such a design makes little sense, except for one situation. If the original design did not include a department, this would allow them to add a department ID without modifying the original EMPLOYEE_DATA table.
They may not have wanted to update legacy code to support the Employee id, so they added it this way.

Purpose of design is determined by business rules.
Business rules dictate entity (logical model perspective) / table (physical model perspective) design. No design is "bad" if it is built according to the requirements that were determined based on business rules. Those rules can however change over time -- foreseeing such changes and building to accommodate/future-proof the data model can really save time, effort and ultimately money.
The first and third example are the same -- the third example has an extraneous column (EMPLOYEE_DEPARTMENT.id). ORMs don't like composite keys, so a single column is used in place of.
The second example is fine if:
employees will never work for more than one department
there's no need for historical department tracking
Conclusion:
The first/third example is more realistic for the majority of real-world situations, and can be easily customized to provide more value without major impact (re-writing entire table structure). It uses what is most commonly referred to as a many-to-many relationship to allow many employees to relate to many departments.

If an employee can be in more than one department, then you would need a mapping table but I'd do it like the following:
EMPLOYEE
Id (PK)
First Name
Last Name
DEPARTMENT
Id (PK)
Name
EMPLOYEE_DEPARTMENT
EmployeeId_fk (PK)
DepartmentId_fk (PK)
Position
This would allow for multiple positions in multiple departments.

You would do this if an employee can be a member of multiple departments. With the latter table, each employee can only belong to one department.

The only remotely good reason for doing this is to implement an extension model where the master table identifying unique customers does not include all the data for customers that is not always necessary. Instead, you create one core table with the core employee data and and extension table with all the supplementary fields. I've seen people take this approach to avoid creating large tables with many columns that are rarely needed. However, in my experience it's typically premature optimization, and I wouldn't recommend it.
In contrast to many responses, the model included does not support multiple departments per employee - it is not a many to many mapping approach.

Database Design for Revisions?

We have a requirement in project to store all the revisions(Change History) for the entities in the database. Currently we have 2 designed proposals for this:
e.g. for "Employee" Entity
Design 1:
-- Holds Employee Entity
"Employees (EmployeeId, FirstName, LastName, DepartmentId, .., ..)"
-- Holds the Employee Revisions in Xml. The RevisionXML will contain
-- all data of that particular EmployeeId
"EmployeeHistories (EmployeeId, DateModified, RevisionXML)"
Design 2:
-- Holds Employee Entity
"Employees (EmployeeId, FirstName, LastName, DepartmentId, .., ..)"
-- In this approach we have basically duplicated all the fields on Employees
-- in the EmployeeHistories and storing the revision data.
"EmployeeHistories (EmployeeId, RevisionId, DateModified, FirstName,
LastName, DepartmentId, .., ..)"
Is there any other way of doing this thing?
The problem with the "Design 1" is that we have to parse XML each time when you need to access data. This will slow the process and also add some limitations like we cannot add joins on the revisions data fields.
And the problem with the "Design 2" is that we have to duplicate each and every field on all entities (We have around 70-80 entities for which we want to maintain revisions).

I think the key question to ask here is 'Who / What is going to be using the history'?
If it's going to be mostly for reporting / human readable history, we've implemented this scheme in the past...
Create a table called 'AuditTrail' or something that has the following fields...
[ID] [int] IDENTITY(1,1) NOT NULL,
[UserID] [int] NULL,
[EventDate] [datetime] NOT NULL,
[TableName] [varchar](50) NOT NULL,
[RecordID] [varchar](20) NOT NULL,
[FieldName] [varchar](50) NULL,
[OldValue] [varchar](5000) NULL,
[NewValue] [varchar](5000) NULL
You can then add a 'LastUpdatedByUserID' column to all of your tables which should be set every time you do an update / insert on the table.
You can then add a trigger to every table to catch any insert / update that happens and creates an entry in this table for each field that's changed. Because the table is also being supplied with the 'LastUpdateByUserID' for each update / insert, you can access this value in the trigger and use it when adding to the audit table.
We use the RecordID field to store the value of the key field of the table being updated. If it's a combined key, we just do a string concatenation with a '~' between the fields.
I'm sure this system may have drawbacks - for heavily updated databases the performance may be hit, but for my web-app, we get many more reads than writes and it seems to be performing pretty well. We even wrote a little VB.NET utility to automatically write the triggers based on the table definitions.
Just a thought!

Do not put it all in one table with an IsCurrent discriminator attribute. This just causes problems down the line, requires surrogate keys and all sorts of other problems.
Design 2 does have problems with schema changes. If you change the Employees table you have to change the EmployeeHistories table and all the related sprocs that go with it. Potentially doubles you schema change effort.
Design 1 works well and if done properly does not cost much in terms of a performance hit. You could use an xml schema and even indexes to get over possible performance problems. Your comment about parsing the xml is valid but you could easily create a view using xquery - which you can include in queries and join to. Something like this...
CREATE VIEW EmployeeHistory
AS
, FirstName, , DepartmentId
SELECT EmployeeId, RevisionXML.value('(/employee/FirstName)[1]', 'varchar(50)') AS FirstName,
RevisionXML.value('(/employee/LastName)[1]', 'varchar(100)') AS LastName,
RevisionXML.value('(/employee/DepartmentId)[1]', 'integer') AS DepartmentId,
FROM EmployeeHistories

The History Tables article in the Database Programmer blog might be useful - covers some of the points raised here and discusses the storage of deltas.
Edit
In the History Tables essay, the author (Kenneth Downs), recommends maintaining a history table of at least seven columns:
Timestamp of the change,
User that made the change,
A token to identify the record that was changed (where the history is maintained separately from the current state),
Whether the change was an insert, update, or delete,
The old value,
The new value,
The delta (for changes to numerical values).
Columns which never change, or whose history is not required, should not be tracked in the history table to avoid bloat. Storing the delta for numerical values can make subsequent queries easier, even though it can be derived from the old and new values.
The history table must be secure, with non-system users prevented from inserting, updating or deleting rows. Only periodic purging should be supported to reduce overall size (and if permitted by the use case).

Avoid Design 1; it is not very handy once you will need to for example rollback to old versions of the records - either automatically or "manually" using administrators console.
I don't really see disadvantages of Design 2. I think the second, History table should contain all columns present in the first, Records table. E.g. in mysql you can easily create table with the same structure as another table (create table X like Y). And, when you are about to change structure of the Records table in your live database, you have to use alter table commands anyway - and there is no big effort in running these commands also for your History table.
Notes
Records table contains only lastest revision;
History table contains all previous revisions of records in Records table;
History table's primary key is a primary key of the Records table with added RevisionId column;
Think about additional auxiliary fields like ModifiedBy - the user who created particular revision. You may also want to have a field DeletedBy to track who deleted particular revision.
Think about what DateModified should mean - either it means where this particular revision was created, or it will mean when this particular revision was replaced by another one. The former requires the field to be in the Records table, and seems to be more intuitive at the first sight; the second solution however seems to be more practical for deleted records (date when this particular revision was deleted). If you go for the first solution, you would probably need a second field DateDeleted (only if you need it of course). Depends on you and what you actually want to record.
Operations in Design 2 are very trivial:
Modify
copy the record from Records table to History table, give it new RevisionId (if it is not already present in Records table), handle DateModified (depends on how you interpret it, see notes above)
go on with normal update of the record in Records table
Delete
do exactly the same as in the first step of Modify operation. Handle DateModified/DateDeleted accordingly, depending on the interpretation you have chosen.
Undelete (or rollback)
take highest (or some particular?) revision from History table and copy it to the Records table
List revision history for particular record
select from History table and Records table
think what exactly you expect from this operation; it will probably determine what information you require from DateModified/DateDeleted fields (see notes above)
If you go for Design 2, all SQL commands needed to do that will be very very easy, as well as maintenance! Maybe, it will be much much easier if you use the auxiliary columns (RevisionId, DateModified) also in the Records table - to keep both tables at exactly the same structure (except for unique keys)! This will allow for simple SQL commands, which will be tolerant to any data structure change:
insert into EmployeeHistory select * from Employe where ID = XX
Don't forget to use transactions!
As for the scaling, this solution is very efficient, since you don't transform any data from XML back and forth, just copying whole table rows - very simple queries, using indices - very efficient!

We have implemented a solution very similar to the solution that Chris Roberts suggests, and that works pretty well for us.
Only difference is that we only store the new value. The old value is after all stored in the previous history row
[ID] [int] IDENTITY(1,1) NOT NULL,
[UserID] [int] NULL,
[EventDate] [datetime] NOT NULL,
[TableName] [varchar](50) NOT NULL,
[RecordID] [varchar](20) NOT NULL,
[FieldName] [varchar](50) NULL,
[NewValue] [varchar](5000) NULL
Lets say you have a table with 20 columns. This way you only have to store the exact column that has changed instead of having to store the entire row.

If you have to store history, make a shadow table with the same schema as the table you are tracking and a 'Revision Date' and 'Revision Type' column (e.g. 'delete', 'update'). Write (or generate - see below) a set of triggers to populate the audit table.
It's fairly straightforward to make a tool that will read the system data dictionary for a table and generate a script that creates the shadow table and a set of triggers to populate it.
Don't try to use XML for this, XML storage is a lot less efficient than the native database table storage that this type of trigger uses.

Ramesh, I was involved in development of system based on first approach.
It turned out that storing revisions as XML is leading to a huge database growth and significantly slowing things down.
My approach would be to have one table per entity:
Employee (Id, Name, ... , IsActive)
where IsActive is a sign of the latest version
If you want to associate some additional info with revisions you can create separate table
containing that info and link it with entity tables using PK\FK relation.
This way you can store all version of employees in one table.
Pros of this approach:
Simple data base structure
No conflicts since table becomes append-only
You can rollback to previous version by simply changing IsActive flag
No need for joins to get object history
Note that you should allow primary key to be non unique.

The way that I've seen this done in the past is have
Employees (EmployeeId, DateModified, < Employee Fields > , boolean isCurrent );
You never "update" on this table (except to change the valid of isCurrent), just insert new rows. For any given EmployeeId, only 1 row can have isCurrent == 1.
The complexity of maintaining this can be hidden by views and "instead of" triggers (in oracle, I presume similar things other RDBMS), you can even go to materialized views if the tables are too big and can't be handled by indexes).
This method is ok, but you can end up with some complex queries.
Personally, I'm pretty fond of your Design 2 way of doing it, which is how I've done it in the past as well. Its simple to understand, simple to implement and simple to maintain.
It also creates very little overhead for the database and application, especially when performing read queries, which is likely what you'll be doing 99% of the time.
It would also be quite easy to automatic the creation of the history tables and triggers to maintain (assuming it would be done via triggers).

Revisions of data is an aspect of the 'valid-time' concept of a Temporal Database. Much research has gone into this, and many patterns and guidelines have emerged. I wrote a lengthy reply with a bunch of references to this question for those interested.

I'm going to share with you my design and it's different from your both designs in that it requires one table per each entity type. I found the best way to describe any database design is through ERD, here's mine:
In this example we have an entity named employee. user table holds your users' records and entity and entity_revision are two tables which hold revision history for all the entity types that you will have in your system. Here's how this design works:
The two fields of entity_id and revision_id
Each entity in your system will have a unique entity id of its own. Your entity might go through revisions but its entity_id will remain the same. You need to keep this entity id in you employee table (as a foreign key). You should also store the type of your entity in the entity table (e.g. 'employee'). Now as for the revision_id, as its name shows, it keep track of your entity revisions. The best way I found for this is to use the employee_id as your revision_id. This means you will have duplicate revision ids for different types of entities but this is no treat to me (I'm not sure about your case). The only important note to make is that the combination of entity_id and revision_id should be unique.
There's also a state field within entity_revision table which indicated the state of revision. It can have one of the three states: latest, obsolete or deleted (not relying on the date of revisions helps you a great deal to boost your queries).
One last note on revision_id, I didn't create a foreign key connecting employee_id to revision_id because we don't want to alter entity_revision table for each entity type that we might add in future.
INSERTION
For each employee that you want to insert into database, you will also add a record to entity and entity_revision. These last two records will help you keep track of by whom and when a record has been inserted into database.
UPDATE
Each update for an existing employee record will be implemented as two inserts, one in employee table and one in entity_revision. The second one will help you to know by whom and when the record has been updated.
DELETION
For deleting an employee, a record is inserted into entity_revision stating the deletion and done.
As you can see in this design no data is ever altered or removed from database and more importantly each entity type requires only one table. Personally I find this design really flexible and easy to work with. But I'm not sure about you as your needs might be different.
[UPDATE]
Having supported partitions in the new MySQL versions, I believe my design also comes with one of the best performances too. One can partition entity table using type field while partition entity_revision using its state field. This will boost the SELECT queries by far while keep the design simple and clean.

If indeed an audit trail is all you need, I'd lean toward the audit table solution (complete with denormalized copies of the important column on other tables, e.g., UserName). Keep in mind, though, that bitter experience indicates that a single audit table will be a huge bottleneck down the road; it's probably worth the effort to create individual audit tables for all your audited tables.
If you need to track the actual historical (and/or future) versions, then the standard solution is to track the same entity with multiple rows using some combination of start, end, and duration values. You can use a view to make accessing current values convenient. If this is the approach you take, you can run into problems if your versioned data references mutable but unversioned data.

If you want to do the first one you might want to use XML for the Employees table too. Most newer databases allow you to query into XML fields so this is not always a problem. And it might be simpler to have one way to access employee data regardless if it's the latest version or an earlier version.
I would try the second approach though. You could simplify this by having just one Employees table with a DateModified field. The EmployeeId + DateModified would be the primary key and you can store a new revision by just adding a row. This way archiving older versions and restoring versions from archive is easier too.
Another way to do this could be the datavault model by Dan Linstedt. I did a project for the Dutch statistics bureau that used this model and it works quite well. But I don't think it's directly useful for day to day database use. You might get some ideas from reading his papers though.

How about:
EmployeeID
DateModified
and/or revision number, depending on how you want to track it
ModifiedByUSerId
plus any other information you want to track
Employee fields
You make the primary key (EmployeeId, DateModified), and to get the "current" record(s) you just select MAX(DateModified) for each employeeid. Storing an IsCurrent is a very bad idea, because first of all, it can be calculated, and secondly, it is far too easy for data to get out of sync.
You can also make a view that lists only the latest records, and mostly use that while working in your app. The nice thing about this approach is that you don't have duplicates of data, and you don't have to gather data from two different places (current in Employees, and archived in EmployeesHistory) to get all the history or rollback, etc).

If you want to rely on history data (for reporting reasons) you should use structure something like this:
// Holds Employee Entity
"Employees (EmployeeId, FirstName, LastName, DepartmentId, .., ..)"
// Holds the Employee revisions in rows.
"EmployeeHistories (HistoryId, EmployeeId, DateModified, OldValue, NewValue, FieldName)"
Or global solution for application:
// Holds Employee Entity
"Employees (EmployeeId, FirstName, LastName, DepartmentId, .., ..)"
// Holds all entities revisions in rows.
"EntityChanges (EntityName, EntityId, DateModified, OldValue, NewValue, FieldName)"
You can save your revisions also in XML, then you have only one record for one revision. This will be looks like:
// Holds Employee Entity
"Employees (EmployeeId, FirstName, LastName, DepartmentId, .., ..)"
// Holds all entities revisions in rows.
"EntityChanges (EntityName, EntityId, DateModified, XMLChanges)"

We have had similar requirements, and what we found was that often times the user just wants to see what has been changed, not necessarily roll back any changes.
I'm not sure what your use case is, but what we have done was create and Audit table that is automatically updated with changes to an business entity, including the friendly name of any foreign key references and enumerations.
Whenever the user saves their changes we reload the old object, run a comparison, record the changes, and save the entity (all are done in a single database transaction in case there are any problems).
This seems to work very well for our users and saves us the headache of having a completely separate audit table with the same fields as our business entity.

It sounds like you want to track changes to specific entities over time, e.g. ID 3, "bob", "123 main street", then another ID 3, "bob" "234 elm st", and so on, in essence being able to puke out a revision history showing every address "bob" has been at.
The best way to do this is to have an "is current" field on each record, and (probably) a timestamp or FK to a date/time table.
Inserts have to then set the "is current" and also unset the "is current" on the previous "is current" record. Queries have to specify the "is current", unless you want all of the history.
There are further tweaks to this if it's a very large table, or a large number of revisions are expected, but this is a fairly standard approach.