Related
I have to specify that this is for a database assignment. I'm pretty good with SQL code but the diagram aspect of the assignment is killing me, I think that every step I take is wrong.
They have given us This scenario and requirements :
A research team has asked you to create a database for a project on movie production
companies; the project aims to use machine learning, neural networks and other
methods to extract information about the situation of movie production companies in
Europe and the health of this sector for a set of specific countries, including the UK.
The data analytics application resulting from this project – which you DO NOT have to
develop; your job is to develop the central, server-side database that underpins it – has been commissioned by a research institute (which shall remain nameless), and it is
intended to be open source, and therefore available to anyone.
Basically, it is a machine learning application that would run on a database with the aim
to identify the correlation between different aspects of the sector, including funding
opportunities and development of new production companies or studios.
The database records every production company in Europe, including the name of the
company, the address, ZIP code, city, country, type of the company (e.g., non-profit
organisation), number of employees and net worth (calculated as total assets minus
total liabilities). Every production company has its name registered with one and only
one local government authority (for example, Companies House in the UK) on a specific
date; each company can have many shareholders. The authority typically requires
information about all the shareholders, including town of birth, mother’s maiden name,
father’s first name, their personal telephone number (only one), national insurance
number (each country in Europe has a similar unique ID), and passport number. Also,
the registration procedure has a cost associated with it (e.g., 12£ in the UK).
The database also records the employees’ data for each company: each employee is
assumed to work for a single production company. Due to the complex structure of
movie production companies and the need for various skills and professions,
employees are categorised into crew and staff. The crew consists of three main groups:
the actors, the director(s) and those who work on other jobs relevant to the filming
(producers, editors, production designers, costume designers, composer, etc.). All
other employees belong to the staff group, including those responsible for HR,
advertising, etc. Employees are identified by an employee ID, first name, last name and
an optional middle name, date of birth and start date. Also, each employee has their
contact details recorded, whether it is a single phone number or multiple, with a
description associated with each of them. Each employee has a single email address,
too.
Members of the crew are paid hourly, and this is recorded in the database as well as a
bonus that depends on their contract. Actors get a bonus for each day of work and
another bonus for each scene completed; directors get a bonus at the end of the
shooting; crew members that work in other jobs relevant to the filming get a bonus at
the end of the shooting, and they have their role recorded as well (e.g., producer or
costume designer).
Staff members have the monthly salary and the working hours (e.g., full time 9-5).
Furthermore, each staff member belongs to a specific department (e.g., advertising),
which is located in a given building at a given address (both recorded in the database).
The database records all movies from each production company. More specifically, for
each movie the following information is recorded: a universal unique movie code(similar to the ISBN for books), the title of the movie, the year and the first release date
(different release dates are not important and should NOT be recorded).
Also, the database records each member of the crew that is part of the movie, and the
role they have in the movie: each crew member can play a single role or multiple roles
in the same movie, and each role has a description associated with it. For example, in
each movie there can be a single protagonist or more than one, the same actor can play
one or several roles, or even have a cameo.
One of the aims of the project is to provide insights on the impact of funding and grants
within the movie industry. To this end, the database should be able to record all the
funding that each production company receives. This must include the name of the
grant, the funding body (e.g., the government of a given country or European Union
grants such as the ERDF), the maximum amount for that grant and the deadline to
submit a proposal.
Then, for each company the database must record the date of the application to a given
grant, the amount requested, the outcome (successful/unsuccessful).
A grant can be given to a single production company or shared among several. Finally,
once the database is ready, the project will run a set of machine learning algorithms to
perform high level data analysis based on the different grants and their corresponding
impact with the aim to investigate the impacts of such funding against a list of criteria.
No additional information is provided at this stage from the project.
In the spec, the requirements are numerated from 1 to 5, as the scenario was not given
at that time. The details of each requirement are provided in the following:
Each production company may have received one or multiple grants, and grants
can be shared by more than one company.
It is possible for each employee to have more than one telephone number. Each
telephone number has a description associated with it (e.g., personal, or work).
Each production company is registered only once but can have many shareholders.
Each employee can either be a member of the crew OR a staff member. Each crew
member can be an actor OR a director OR have another role. Each staff member
belongs to a department. No duplication of data is allowed.
Each crew member may be part of one or more movies in a single role or many.
Based on that I have created THIS DIAGRAM.
I think I have all the entities,attributes and relationships down but I'm missing the keys. Keys can't be names right? I will use the company entity as an example. So, should I create new attributes like company_id to use as primary keys or just underline the name attributes and use it as Primary Key?
Also, please tell me if there's anything else wrong with the diagram.
Thanks a lot!
I created an er diagram but some entities don't attributes that can be used as primary keys because they are names. I tried using them but I don't think it's right.
The problem with names as primary keys
In your diagram, you have a couple of name used to identify entities: Grant, Production Company, Shareholder (full name), Employee, Movie (Title). You can in theory use them as primary key. However, this is a bad practice:
names can change (e.g. departments and companies can be renamed, movies can have a temporary working title);
names are often not sufficient to distinguish entities (e.g. there may be different people having the same name, e.g. Adam Smith);
names can be spelled differently across source of information , and are also easily misspelled;
although not really noticeable with modern RDBMS, names are more time consuming to search, and consume more memory when used as foreign keys.
How to chose a primary key?
You'd better use a primary key that guarantees uniqueness. You can then decide easily if a same name correspond to a different entity or not.
The next question that you'll then face in you design is surrogate key vs. natural key:
When there's no other unique information, you'll have not choice than using a surrogate key.
When there are other potential unique attributes, you may chose to use either a natural key (e.g. company registration number, national insurance number together with a country code, movie code?) or a surrogate one.
Keep in mind that both have advantages and inconveniences, but the surrogate key is in general more robust, as natural keys sometimes appear to be not as stable as expected.
Other remarks concerns about your ERD
By the way, here some issues and other remarks:
Works in relation does not relate Staff to anything else. From the name, it's obviously not a reflexive relation either. So this is a diagram error. department (name) and building should either be attached to a Department entity or be attributes of Staff.
In several cases you relate attributes to other attributes (actor-extra role, phone number-description) . This is also a diagramming error. Either add the extra attribute to the same entity, or there's a missing relationship with a missing entity.
In one case you relate two entities without a relation between the two (production company- application). This is an inconsistency that must be corrected also.
The following attributes are not real attributes but probably values of an unidentified attribute: producer, composer, actor, editor, xyzzy designer, advertising, HR, janitor.
Government authority is a misleading entity name: nowhere do you refer to data about the authority itself (name of the authority, e.g. "CNC", country of the government, ...). It's only information about the company's registration.
In your diagram you leave the hourly and monthly wage at the level of the Employee. This does not model accurately the requirements.
The link of the relation receive funding and the entity Application with the same attribute outcome seems very ambiguous.
In the name of the entities, stay consistent: either singular or plural. But mixing both will lead to lots of typos.
Better show cardinality in the link between the relation and the entity, than on the top of the relation: this avoids confusion about the direction of reading.
As a side remark, your question provides wealth of interesting details, but that are not really needed for answering the core of your question. Better limit yourself to only the information directly related to your issue in your next questions ;-)
Research or not research, keep in mind that GDPR may apply and that it requires inter alia privacy by design (some information about the shareholder and the employee may require some additional thoughts).
I am trying to understand cardinality in ER diagrams, starting with a simple example of a visitor visiting a city. Thinking about it, you can have many visitors visiting many cities, so I drew the ER diagram as shown. Visitor is an Entity with attributes such as name, Visits is the relationship with the VisitorID and CityID as the primary key and other attributes such as date, and City is an entity, with attributes such as country. This would mean 0 to many visitors can visit 0 to many cities. Does this mean that in a database with this ER diagram, I would have three tables? One for Visitor, one for City, and one for VisitCity? Am I understanding the cardinality correctly?
Yes, you described the relationships correctly. N:M cardinality needs a third table.
Visits could contain other information, too. For example, since a visitor can't be in two places at once, there might be an associated time period.
Visitor could have another relationship to City -- say, born_in with different cardinality, namely N:1, meaning many people were born in the same city (but each is born in only one city!) For that, born_in would be an attribute of Visitor.
yes that is correct. your drawing shows 0 to many visitor(s) can visit 0 to many city(s). so in this case you are correct. as you stated you have named your middle-table visitCity.
but lets say you are required to have a database where visitors can visit 1 city only. you would have one-to-many relationship. this means you don't need to use a middle-table for that. so you will end up with 2 tables only.
just because something logical like "many visitors can visit many cities" doesn't mean is true. the requirements is what makes these rules (project assigned by the client saying what he/she needs. or if u are still in school, this would be the papers assigned to you wich the story in it). so when creating databases you have to listen/read the requirements very carefully to determine when to use what. but from reading what u have said i think u are going in the right track :)
Yes, that's exactly right.
If you wish to keep your DB in 1NF, every time you face yourself with a N x N cardinality, a new table must be created with the keys to both entities.
More info on normalization here.
I just started to work with databases and want to realize the reasons for the concept of adventure-work-db tables design. why do we consider BusinessEntity as a table and we didn't put it in person table? isn't it kind of over-Normalized Table?
A table should generally represent one thing/object/concept. If you were to include the content of BusinessEntity within Person, then the model is saying that a Person is a BusinessEntity and vice versa which I guess is not really true. Although I accept that BusinessEntity does not currently have anything it in other than a reference to a Person.
However the model as it stands is easily extendable. You may wish to add new columns in the future to BusinessEntity (e.g. Company Number, tax registration code). Such attritubes do not belong to a person so would not really belong in the Person table. Hence if your application has a concept of a Business Entity then it is good practice to model it as in your diagram.
I've been learning database design and I'm confused about 1:1 relationships. From what I understand, you can simply add columns to the appropriate table. Can someone provide a real world example of where a 1:1 relationship was either necessary or provided some significant benefit? I.e., where would I use a 1:1 relationship and what would it look like?
I'll give you a real practical example.
In the medical billing world, doctors who want to get paid by medicare handle billing by creating a dictation report for each visit with a patient. This might actually be a recorded audio dictation transcribed by a secretary, but more often it's just a written description of what they did and talked about with the patient, along with history, impressions, and so forth. A licensed medical coder will then read this dictation and decide what the doctor is allowed to bill.
Separate from the dictation, there is demographic information about the patient involved: name, age, billing address, etc. This information must be strictly separate from information about the dictation, to prevent coders from allowing bias to cloud their billing judgements or violating patients' privacy.
This data is often kept well-normalized with a 1:many relationship in the data systems at the point of origin, and only the right parts are displayed to the right people at the right times. However, a significant number of offices out-source their billing function to a third party. This way a small clinic, for example, doesn't have to keep a licensed medical coder on staff; one coder at the billing office can handle the needs of many clinics. When the data is sent from the clinic to the billing office, the patient demographic information and the dictations need to come over as separate pieces, possibly at separate times. At this point, they'll likely be stored in completely separate tables with a 1:1 relationship and a shared ID field to match them up later.
In this case, the 1:1 relationship has very little to do with the data model. You could probably match up the records at the time of import, and as a bill moves through the system eventually the provincial patient information received in the clinic's demographic record will be matched to a real person so the 1:many relationship can be restored. Otherwise you'd get a separate statement on a separate account for each visit to the doctor.
Instead, it has almost everything to do with the systems design. There are likely entirely different people building and using the billing part verses the coding part at our imaginary billing service. This way, each side can each have full control of it's own fiefdom, and you are sure that no one, not even a developer, is breaking any privacy rules.
True one-to-one relationships seldom
occur in the real world. This type of
relationship is often created to get
around some limitation of the database
management software rather than to
model a real-world situation. In
Microsoft Access, one-to-one
relationships may be necessary in a
database when you have to split a
table into two or more tables because
of security or performance concerns or
because of the limit of 255 columns
per table. For example, you might keep
most patient information in
tblPatient, but put especially
sensitive information (e.g., patient
name, social security number and
address) in tblConfidential (see
Figure 3). Access to the information
in tblConfidential could be more
restricted than for tblPatient. As a
second example, perhaps you need to
transfer only a portion of a large
table to some other application on a
regular basis. You can split the table
into the transferred and the
non-transferred pieces, and join them
in a one-to-one relationship.
That's a quote from here: Fundamentals of Relational Database Design
And here's a similar question on SO.
Another reason I can see for using a 1:1 (where I have used it in the past) is if you have a table with a lot of columns, and only a few of them are involved in very intensive and frequent queries which need to be fast, I would break it into two tables that are related 1:1 where I could query the lightweight table and get good performance, but still have the other data related to it easily with a simple join.
I belief tables should be designed with the domain background. So if those columns form two different entities, they should not be mixed in one table. From my experience 1:1 relationships tend to evolve into 1:n relationships over time.
For example you may want to store the postal address of a person. But after some time, you are required to store more than one address per person. Refactoring programs from a 1:1 relationship into 1:n is usually a lot easier than extracting some columns from an old table into a new one.
Many database systems allow defining of access permissions per table in a very easy way. But defining permissions on individual columns is often quite painful.
It's useful if X has a 1:1 relationship with Y and Z also has a 1:1 relationship with Y. Y can be abstracted out into a shared table rather than duplicating in both X and Z.
EDIT: A real world example would be Customers, Companies, and Addresses. There can be a N:N relationship between Customer and Company. But both Customer and Company have 1:1 relationships with Address. Some Address rows could be related to both a Customer and a Person.
First, because they are talking about Access (Jet, Ace, whatever) -- credit to #Richard DesLonde for spotting this -- then they are probably talking about 1:0..1 relationships. I do not believe true 1:1 relationships are workable in Access because it has no mechanism for deferring constraints nor executing multiple statements in a SQL PROCEDURE. Most Access practitioners are satisfied to use a 1:0..1 relationship to model a true 1:1 relationship, so I guess the authors are satisfied to use the term "1:1" informally to refer to both.
Of course, 1:1 and 1:1..0 relationships are common enough in the real world. I rather think they are trying to convey the (valid) point that some 1:1 and 1:1..0 relationships are invented in a data model for business purposes.
Consider a "natural person" (i.e. human) and a "corporation". They have no attributes in common (sure, both have a "name" but their domains are different e.g. "natural person name" has sub atomic domains for "family name", "given name" and "title", etc).
However, in a given data model distinct entity types may play the same role. For example, both a "natural person" and a "corporation" can be the officer of a "corporation". In the data model, we could have two distinct entity types "natural person officers" and "corporate officers" that are likely to have many attributes in common and from the same domains e.g. appointment date, termination date, etc; further, they business rules would be the same e.g. appointment date must be before termination date. Also, both would participate in equivalent relationships e.g. "natural person representing", etc.
The data model could be 'split' at high level, resulting in pairs of very similar tables e.g. "natural person officer" and "corporate officer", "natural person officer natural person representing" and "corporate officer natural person representing", etc.
However, another approach is to model the common attributes and relationships using a fabricated entity type. For example, both a "natural person" and a "corporation" could be considered to be a "legal person" (aside: there is such a concept of "legal person" in law but does this mean the same as existing in the real world?!)
Therefore, we could have a superclass table for "legal persons" and subtype tables for "natural persons" and "corporations" respectively. The "officers" table would reference the "legal persons" table. All subsequent relationship tables could reference the "officers" table, which would half the number of tables from this point down.
There are practical problems to such a 'subclassing' approach. Because a "natural person" and a "corporation" has no attributes in common, they have no common key, therefore the "legal persons" table would need to have an artificial key, with all the problems that this entails, especially if it needs to be exposed in the application. Also, because the relationships between "legal persons", "natural persons" and "corporations" are truly 1:1 some DBMSs, as Access, will lack the necessary functionality to effectively implement them and many will have to settle for making them 1:0..1.
A 1:1 relationship is an abstract concept that you model in your data, but at the database level (assuming RDBMS) doesn't really exist. You always have a foreign key on one table pointing to another, so technically the parent table being pointed to by the FK could have multiple children. This is something you'll want to enforce in your business logic.
A good example of a 1:1 relationship in a modeling sense would be the relationship between employee and person. You have a person with certain data, then you have extra attributes on that same person that you put on an employee. A good way to think of this in OO programming terms is as inherited classes. The Employee class, inherits from Person. In fact may ORM systems will model the 1:1 relationship in the database with each table having a shared primary key.
Every Customer has a physical address and an optional mailing address. What is your preferred way to model this?
Option 1. Customer has foreign key to Address
Customer (id, phys_address_id, mail_address_id)
Address (id, street, city, etc.)
Option 2. Customer has one-to-many relationship to Address, which contains a field
to describe the address type
Customer (id)
Address (id, customer_id, address_type, street, city, etc.)
Option 3. Address information is de-normalized and stored in Customer
Customer (id, phys_street, phys_city, etc. mail_street, mail_city, etc.)
One of my overriding goals is to simplify the object-relational mappings, so I'm leaning towards the first approach. What are your thoughts?
I tend towards first approach for all the usual reasons of normalisation. This approach also makes it easier to perform data cleansing on mailing details.
If you are possibly going to allow multiple addresses (mail, residential, etc) or wish to be able to use effective dates, consider this approach
Customer (id, phys_address_id)
Cust_address_type (cust_id, mail_address_id, address_type, start_date, end_date)
Address (id, street, city, etc.)
One important fact you may need to consider (depending on your problem domain) is that people change addresses, and may want to let you know in advance of their address change; this is certainly true for utility companies, telcos, etc.
In this case you need to have a way to store multiple addresses for the customer with validity dates, so that the address can be set up in advance and automatically switch at the correct point. If this is a requirement, then a variation on (2) is the only sensible way to model it, e.g.
Customer (id, ...)
Address (id, customer_id, address_type, valid_from, valid_to)
On the other hand, if you don't need to cater for this (and you're sure you won't in the future) then probably (1) is simpler to manage because it's much easier to maintain data integrity as there's no issues with ensuring only one address of the same type exists, and the joins become simpler as they're only on one field.
So either (1) or (2) are fine depending on whether you need house-moves, but I'd steer clear of (3) because you're then repeating the definition of what an address is in the table, and you'll have to add multiple columns if you change what an address looks like. It's possibly slightly more performant, but to be honest when you're dealing with properly indexed joins in a relational database there isn't a lot to be gained, and it's likely to be slower in some scenarios where you don't need the address as the record size for a customer will be larger.
We are moving forward with a model like this:
Person (id, given_name, family_name, title, suffix, birth_date)
Address (id, culture_id, line1, line2, city, state, zipCode, province, postalCode)
AddressType (id, descriptiveName)
PersonAddress (person_id, address_id, addressType_id, activeDates)
Most may consider this excessive. However, an undeniable common theme amongst the apps we develop is that they will have some of these fundamental entities - People, Organizations, Addresses, Phone Numbers, etc.. - and they all want to combine them in different ways. So, we're building in some generalization up-front that we are 100% certain we have use cases for.
The Address table will follow a table-per-hierarchy inheritance scheme to differentiate addresses based on culture; so a United States address will have a state and zip field, but Canadian addresses will have a province and postal code.
We use a separate connecting table to "give" a person an address. This keeps our other entities - Person & Address - free from ties to other entities when our experience is this tends to complicate matters down the road. It also makes it far simpler to connect Address entities to many other types of entities (People, Organizations, etc.) and with different contextual information associated with the link (like activeDates in my example).
The second option would probably be the way I would go. And on the off-chance it would let users add additional address' (If you wanted to let them do that), that they could switch between at will for shipping and such.
I'd prefer #1. Good normalization and communicates intent clearly. This model also allows the same address object (row) to be used for both addresses, something I have found to be quite valuable. It's far too easy to get lost in duplicating this information too much.
When answering those kinds of questions I like to use the classifications of DDD. If it's a Entity it should have a separate ID, if it's a value object it should not.
Option 3 is too restrictive, and option 1 cannot be extended to allow for other address types without changing the schema.
Option 2 is clearly the most flexible and therefore the best choice.
In most code I write nowadays every customer has one and only one physical location. This is the legal entity beeing our business partner. Therefore I put street, city etc in the customer object/table. Often this is the possible simplest thing that works and it works.
When an additional mailing address is needed, I put it in a separate object/table to not clutter the customer object to much.
Earlier in my career I normalized like mad having an order referencing a customer which references a shipping address. This made things "clean" but slow and inelegant to use. Nowadays I use an order object which just contains all the address information. I actually consider this more natural since a customer might change his (default?) address, but the address of a shipment send in 2007 should always stay the same - even if the customer moves in 2008.
We currently implement the VerySimpleAddressProtocol in out project to standardize the fields used.
I'd go for the first option. In these situations I'm very weary of YAGNI (you aren't going to need it). I can't count the number of times I've looked at schemas that've had one-to-many tables "just incase" that are many years old. If you only need two, just use the first option; if the requirement changes in the future, change it then.
Like in many cases: It depends.
If your customers deal with multiple addresses then a to-many relationship would be appropriate. You could introduce a flag on address that signals if an address is for shipment or bill, etc. Or you store the different address types in different tables and have multiple to-one relationships on a customer.
In cases where you only need to know one address of a customer why would you model that to-many? A to-one relationship would satisfy your needs here.
Important: Denormalize only if you encounter performance issues.
I would go with option 1. If you want to, you could even modify it a little bit to keep an address history:
Customer (id, phys_address_id, mail_address_id)
Address (id, customer_id, start_dt, end_dt, street, city, etc.)
If the address changes, just end date the current address and add a new record in the Address table. The phys_address_id and mail_address_id always point to the current address.
That way you can keep a history of addresses, you could have multiple mailing addresses stored in the database (with the default in mail_address_id), and if the physical address and mailing address are identical you'll just point phys_address_id and mail_address_id at the same record.
Good thread. I have spent a while contemplating the most suitable schema and I have concluded that quentin-starin's solution is the best except I have added start_date and end_date fields to what would be his PersonAddress table. I have also decided to add notes, active and deleted.
deleted is for soft delete functionality as I think I do not want to lose trace of previous addresses simply by deleting the record from the junction table. I think that is quite wise and something others may want to consider. If not done this way, it could be left to revision of paper or electronic documents to try to trace address information (something best avoided).
notes I think of being something of a requirement but that might just be preference. I've spent time in backfill exercises verifying addresses in databases and some addresses can be very vague (such as rural addresses) that I think it is very useful to at least allow notes about that address to be held in the record address.
One thing i would like to hear opinions on is the unique indexing of the address table (again, referring to the table of the same name in quentin-starin's example. Do you think it should be unique index should be enforced (as a compound index presumably across all not-null/required fields)? This would seem sensible but it might still be hard to stop duplicate data regardless as postal/zip codes are not always unique to a single property. Even if the country, province and city fields are populated from reference data (which they are in my model), spelling differences in the address lines may not match up. The only way to best avoid this might be to run one or a number of DB queries from the incoming form fields to see if a possible duplicate has been found. Another safety measure would be give the user the option of selecting from address in the database already linked to that person and use that to auto-populate. I think this might be a case where you can only be sensible and take precautions to stop duplication but just accept it can (and probably will) happen sooner or later.
The other very important aspect of this for me is future editing of the address table records. Lets say you have 2 people both listed at: -
11 Whatever Street
Whatever City
Z1P C0D3
Should it not be considered dangerous to allow the same address table record to be assigned to different entities (person, company)? Then let's say the user realises one of these people lives at 111 Whatever Street and there is a typo. If you change that address, it will change it for both of the entities. I would like to avoid that. My suggestion would be to have the model in the MVC (in my case, PHP Yii2) look for existing address records when a new address is being created known to be related to that customer (SELECT * FROM address INNER JOIN personaddress ON personaddress.address_id = address.id WHERE personaddress.person_id = {current person being edited ID}) and provide the user the option of using that record instead (as was essentially suggested above).
I feel linking the same address to multiple different entities is just asking for trouble as it might be a case of refusing later editing of the address record (impractical) or risking that the future editing of the record may corrupt data related to other entities outside of the one who's address record is being edited.
I would love to hear people's thoughts.