I need to create a database that, among other things, lets people choose 1 - N zip codes in the US.
Intuitively it seems best to make users a row and zip codes columns.
The problem I am having is that is like 42k columns. Which I am confident is outside most SQL DBs upper bound on columns.
I could have separate tables for each state. And then would have like 500-5K columns / table?
I mean that is doable, but the whole thing just seems a little ridiculous.
All thoughts, critiques, etc. are appreciated.
Also, any know the best place to get a list of zip codes (maybe broken down by state?)? Googling yielded some dated stuff. And so far I have USPS APIs for live verification. But I just need a static list.
Thanks everyone.
In any database -- including BigQuery -- your description suggests a table UserZips with one row per UserId and ZipCode.
Bigquery does not require such a structure. It supports arrays within a row, so you can have an array of the zip codes that a user chooses.
It also supports records within a row, so you can have an array of records. Each record could have a zip code and other information.
In many databases, including BigQuery, you might find a JSON object to be the appropriate representation.
Nonetheless, what first comes to mind is a table with a column for the user and zip code.
A structure like this would allow you to add as many zip codes as needed and link as many zips as you needed to as many users as you have.
http://dbfiddle.uk/?rdbms=sqlserver_2014&fiddle=6d1c3a41cb439d17f4def51a672766e1
CREATE TABLE zipCodes (
zipID int identity
, zipcode varchar(5) NOT NULL
, zipPlusFour varchar(4) DEFAULT '0000'
, CONSTRAINT PK_zipID PRIMARY KEY (zipID)
) ;
CREATE TABLE users (
userID int identity
, username nvarchar(20) NOT NULL
, CONSTRAINT PK_userID PRIMARY KEY (userID)
) ;
CREATE TABLE xref_users_zips (
userID int NOT NULL
, zipID int NOT NULL
, CONSTRAINT FK_userID FOREIGN KEY (userID) REFERENCES users(userID)
, CONSTRAINT FK_zipID FOREIGN KEY (zipID) REFERENCES zipCodes(zipID)
) ;
INSERT INTO zipCodes (zipcode)
VALUES ('00501'), ('00544'), ('00601')
;
INSERT INTO users (username)
VALUES ('johndoe'),('robertbuilder'),('zaphodbeeblebrox')
;
INSERT INTO xref_users_zips (userID, zipID)
VALUES (1,1), (2,2), (3,3)
;
SELECT *
FROM users u
INNER JOIN xref_users_zips xuz ON u.userID = xuz.userID
INNER JOIN zipcodes z ON xuz.zipID = z.zipID
Related
I've heard that having an autogenerated primary key is a convention. However, I'm trying to understand its benefits in the following scenario:
CREATE TABLE countries
(
countryID int(11) PRIMARY KEY AUTO_INCREMENT,
countryName varchar(128) NOT NULL
);
CREATE TABLE students
(
studentID int(11) PRIMARY KEY AUTO_INCREMENT,
studentName varchar(128) NOT NULL,
countryOfOrigin int(11) NOT NULL,
FOREIGN KEY (countryOfOrigin) REFERENCES countries (countryID)
);
INSERT INTO countries (countryName)
VALUES ('Germany'), ('Sweden'), ('Italy'), ('China');
If I want to insert something into the students table, I need to lookup the countryIDs in the countries table:
INSERT INTO students (studentName, countryOfOrigin)
VALUES ('Benjamin Schmidt', (SELECT countryID FROM countries WHERE countryName = 'Germany')),
('Erik Jakobsson', (SELECT countryID FROM countries WHERE countryName = 'Sweden')),
('Manuel Verdi', (SELECT countryID FROM countries WHERE countryName = 'Italy')),
('Min Lin', (SELECT countryID FROM countries WHERE countryName = 'China'));
In a different scenario, as I know that the countryNames in the countries table are unique and not null, I could to the following:
CREATE TABLE countries2
(
countryName varchar(128) PRIMARY KEY
);
CREATE TABLE students2
(
studentID int(11) PRIMARY KEY AUTO_INCREMENT,
studentName varchar(128) NOT NULL,
countryOfOrigin varchar(128) NOT NULL,
FOREIGN KEY (countryOfOrigin) REFERENCES countries2 (countryName)
);
INSERT INTO countries2 (countryName)
VALUES ('Germany'), ('Sweden'), ('Italy'), ('China');
Now, inserting data into the students2 table is simpler:
INSERT INTO students2 (studentName, countryOfOrigin)
VALUES ('Benjamin Schmidt', 'Germany'),
('Erik Jakobsson', 'Sweden'),
('Manuel Verdi', 'Italy'),
('Min Lin', 'China');
So why should the first option be the better one, given that countryNames are unique and are never going to change?
There are two apects involved here:
natural keys vs. surrogate keys
autoincremented values
You are wondering why to have to deal with some arbitrary number for a country, when a country can be uniquely identified by its name. Well, imagine you use the country names in several tables to relate rows to each other. Then at some point you are told that you misspelled a country. You want to correct this, but have to do this in every table the country occurs in. In big databases you usually don't have cascading updates in order to avoid updates that unexpectedly take hours instead of mere minutes or seconds. So you must do this manually, but the foreign key constraints get in your way. You cannot change the parent table's key, because there are child tables using this, and you cannot change the key in the child tables first, because that key has to exist in the parent table. You'll have to work with a new row in the parent table and start from there. Quite some task. And even if you have no spelling issue, at some point someone might say "we need the official country names; you have China, but it must be the People's Republic of China instead" and again you must look up and change that contry in all those tables. And what about partial backups? A table gets totally messed up due to some programming error and must be replaced by last week's backup, because this is the best you have. But suddenly some keys don't match any more. You never want a table's key to change.
You say "country names are unique and are never going to change". Think again :-)
It is easier to have your database use a technical arbitrary ID instead. Then the country name only exists in the country table. And if that name must get changed, you change it just in that one place, and all relations stay intact. This, however, doesn't mean that natural keys are worse than technical IDs. They are not. But it's more difficult with them to set up a database correctly. In case of countries a good natural key would be a country ISO code, because this uniquely identifies a country and doesn't change. This would be my choice here.
With students it's the same. Students usually have a student number or student ID in real world, so you can simply use this number to uniquely identifiy a student in the database. But then, how do we get these unique student IDs? At a large university, two secretaries may want to enter new students at the same time. They ask the system what the last student's ID was. It was #11223, so they both want to issue #11224, which causes a conflict of course, because only one student can be given that number. In order to avoid this, DBMS offer sequences of which numbers are taken. Thus one of the secretaries pulls #11224 and the other #11225. Auto-incremented IDs work this way. Both secretaries enter their new student, the rows get inserted into the student table and result in the two different IDs that get reported back to the secretaries. This makes sequences and auto incrementing IDs a great and safe tool to work with.
Convention can be a useful guide. It isn't necessarily the best option in all situations.
There are usually tradeoffs involved, like space, convenience, etc.
While you showed one method of resolving / inserting the proper country key value, there's a slightly less wordy option supported by standard SQL (and many databases).
INSERT INTO students (studentName, countryOfOrigin)
WITH list (name, country) AS (
SELECT *
FROM (
VALUES ('Benjamin Schmidt', 'Germany')
, ('Erik Jakobsson', 'Sweden')
, ('Manuel Verdi', 'Italy')
, ('Min Lin', 'China')
) AS x
)
SELECT name, countryID
FROM list AS l
JOIN countries AS c
ON c.countryName = l.country
;
and a little less wordy again:
INSERT INTO students (studentName, countryOfOrigin)
WITH list (name, country) AS (
VALUES ('Benjamin Schmidt', 'Germany')
, ('Erik Jakobsson', 'Sweden')
, ('Manuel Verdi', 'Italy')
, ('Min Lin', 'China')
)
SELECT name, countryID
FROM list AS l
JOIN countries AS c
ON c.countryName = l.country
;
Here's a test case with MariaDB 10.5:
Working test case (updated)
I have 3 table Student,Teacher,User.
Student:
CREATE TABLE Student( id INT NOT NULL PRIMARY KEY,name VARCHAR(50) NOT NULL);
INSERT INTO [dbo].[Student]([id],[name]) VALUES(4,'Ram'),(5,'Raman');
Teacher:
CREATE TABLE Teacher( id INT NOT NULL PRIMARY KEY,name VARCHAR(50) NOT NULL);
INSERT INTO [dbo].[Student]([id],[name]) VALUES(1,'Raj'),(2,'Rahul');
User:
CREATE TABLE [dbo].[User](
id INT NOT NULL PRIMARY KEY,
user_id INT NOT NULL,
user_type CHAR(1) NOT NULL,
user_name VARCHAR(10) NOT NULL,
user_password VARCHAR(255) NOT NULL,
CONSTRAINT FOREIGN KEY (user_id) REFERENCES Student (id),
CONSTRAINT FOREIGN KEY (user_id) REFERENCES Teacher (id) );
Now I try to INSERT in User table with below query
INSERT INTO [dbo].[User] ([id] ,[user_id] ,[user_type],[user_name] ,[user_password]) VALUES (1 ,1,'S','Raj_001','********')
It gives me error for violation of foreign key due to
value of user_id is available in Teacher and not in Student
So my question is: How can I achieve that a value of user_id is present in one of those table and data should be inserted in User table.
Your table structure is flawed. A foreign key tells the database that there is definitely one and only one row on one side of the relationship. You can't have a partial match, it's all or nothing. This is before considering how you would ensure that you don't end up with the same id in both the teacher and student table.
It would be better to have two columns in your user table, one for teacher id and one for student id. In fact going further given the only extra data in both student and teacher tables is their name why not just eliminate both and store the name in the user table?
Another option to consider is that your foreign key is pointed in the wrong direction. Perhaps a better approach is reversing it to ensure each student and teacher is a user rather than that a user is either a student or a teacher.
First of all get rid of those key words from table name like [User],user_id etc.
It really is problematic and irritating.
Secondly why 2 key in [User] table,id, user_id ? It is not require.
I will keep only id or user_id.
Thirdly, knowing the real table structure or even purpose of each table help in better data modeling.
From [User] table what it appear is that id and user_type are composite primary key.
It should be. If this is true then you can't define FK constraint, as user_type is not available in either Teacher table and Student Table.
And what is appear that ,for example first data is inserted in Student or Teacher then data is inserted in User table in same Transaction.
So in all above scenario, Instead of Trigger is ideal scenario in this condition.
My script is just demo,
Create Proc spStudentInsert
as
set nocount on
set xact_abort on
begin try
begin tran
--bulk insert or single insert ,no problem
insert into Student
insert into [User]
if (##Trancount>0)
commit
end try
begin catch
if (##Trancount>0)
rollback
end catch
CREATE TRIGGER INSTEADOF_TR_I_User ON [user]
INSTEAD OF INSERT
AS
BEGIN
DECLARE #Flag BIT = 1
IF NOT EXISTS (
SELECT 1
FROM Student S
INNER JOIN inserted i ON i.id = S.id
)
SET #Flag = 0
ELSE IF NOT EXISTS (
SELECT 1
FROM Teacher T
INNER JOIN inserted i ON i.id = T.id
)
AND #Flag = 1
SET #Flag = 0
IF (#Flag = 0)
BEGIN
RAISERROR (
N'Invalid user'
,16
,1
)
RETURN
END
END
In case I am wrong about id, user_type composite PK then you can do other way,
PK of User id is FK in Student table as well as Teacher table.
Also , id are PK in their respective table.
So first you insert in User table then you insert in Student or Teacher table.
So design in this case will be,
CREATE TABLE [dbo].[User](
id INT NOT NULL ,
user_type CHAR(1) NOT NULL,
user_name VARCHAR(10) NOT NULL,
user_password VARCHAR(255) NOT NULL,
CONSTRAINT [PK_user] PRIMARY KEY (id)
)
INSERT INTO [dbo].[User] ([id] ,[user_type],[user_name] ,[user_password])
VALUES (1 ,1,'S','Ram_001','********')
--drop table [User]
--alter table [user]
-- drop constraint PK_user
CREATE TABLE Student( id INT NOT NULL PRIMARY KEY,name VARCHAR(50) NOT NULL);
ALTER TABLE Student
add CONSTRAINT FK_StudentUser FOREIGN KEY (id) REFERENCES [User] (id);
INSERT INTO [dbo].[Student]([id],[name]) VALUES(1,'Ram'),(5,'Raman');
--select * from [Student]
CREATE TABLE Teacher( id INT NOT NULL PRIMARY KEY,name VARCHAR(50) NOT NULL);
ALTER TABLE Teacher
add CONSTRAINT FK_TeacherUser FOREIGN KEY (id) REFERENCES [User] (id);
INSERT INTO [dbo].Teacher([id],[name]) VALUES(1,'Raj'),(2,'Rahul');
So what it appear from your question, I will create Instead of Trigger and go with that model.
There are two ways to do this without re-doing your table schema
Create a 4th table that contains the union of ID from Student and Teacher. Presumably, you would insert to that table whenever you insert into Student and Teacher, and then have the constraint act against that table.
Create a custom function based constraint rather than a foreign key which looks up against a union of both the student and teacher tables.
Neither of these are great/clean solutions, and as others have noted, you probably are dealing with the fact that the schema isn't ideal.
Still, if you're just modifying an existing system (and I assume this is a simplified version of what you're actually dealing with), then one of the two solutions I mentioned id easier than redoing the schema.
Your foreign key definition has some logical problems. It forces the user_id to exists in both tables. The solution here is depended on the business needs and real data.
You can create a Person table with 1-1 relation to the student and the Teacher tables and then use the Person.Id column in the foreign key definition. This solution assumes that the students' and teachers' data may change differently.
As another way (which is explained in other answers), If your student and teachers' data is similar, you can combine both tables, and difference data by one added "Type" column.
SO you want to tell the system that your User must be in one of your tables .
it's not possible in databases logic but you can write a script that have a condition (IF exist) then insert you user data
notice : you have to remove your foreign keys .
its a wrong logic !
you are telling your system that your user is a student and a teacher to !
that is absolutely wrong .
I feel like there were some excellent responses in this thread, but I'm going to take a stab at giving you a different direction. I'll try to be clear on why, and try to acknowledge your situation as I do so.
Student/Teacher Data is Often Messy
As someone with experience normalizing data sets in higher education, the issue you've run into resonated with me. Educational users could be in all three categories (Student, Teacher, and User) or just one of them, depending on the how and why the category was linked. Worse, they can enter from multiple directions and end up with multiple unlinked accounts. More mature institutions and tools have protections against this, but I still see user-created databases and ten year old 'it was temporary' solutions that cause me existential pain.
The Main Stumbling Block
Any database with tables that independently define who is a user based on different criteria have a potential point of failure.
Foreign keys was the right direction to be thinking in for this problem. You want these tables to connect and you want them to stay consistent with one another, regardless of which side of the data gets altered. We just need to add a little extra.
One Table To Rule Them All
Before I go further, I want to say that it is possible to get all of the fields you're tracking into a single table, but having multiple tables with distinct purposes is an easy way to protect against changes later.
The foreign key table must inherit the key from another table, but people often say foreign keys can't be primary keys as well. Why?
Foreign keys are not automatically unique keys in the tables they're in. If there can be multiple fields tied to that same key, the table ends up worthless.
We fix that with the Unique constraint. Applied to a foreign key field, Unique essentially makes it act as a primary key would.
Sample Method
Below is an alternative design for what you seemed to be after, creating a master list of IDs that can link across all tables. I tossed in a few minor tracking fields that can be useful for debugging.
/*Create Tables*/
CREATE TABLE ID(
USER_ID int NOT NULL PRIMARY KEY AUTO_INCREMENT,
USER_CREATED timestamp
);
CREATE TABLE USER(
USER_ID int NOT NULL UNIQUE FOREIGN KEY REFERENCES ID(USER_ID),
USER_LOGIN VARCHAR(10) NOT NULL UNIQUE,
USER_PASSWORD VARCHAR(255) NOT NULL,
USER_NAME VARCHAR(50) NOT NULL
);
CREATE TABLE PERMISSIONS(
USER_ID int NOT NULL UNIQUE FOREIGN KEY REFERENCES ID(USER_ID),
STUDENT CHAR(1),
TEACHER CHAR(1)
);
This creates a flag for student and teacher that could both be true or both be false. If you want the code to force them into only one or the other, you can still have the permissions table do a USER_TYPE field instead. I suggest a null or neither value being possible in either case if you plan to use this for any length of time. Best of luck.
i am actually reading Doctrine Reference: One to Many, Unidirectional with Join table. but this will probably be more of a SQL quesiton. basically, this is supposed to model a one to many, unidirectional relationship. i guess from the PHP code (in that link), its such that 1 user have many phonenumbers.
the question is from the SQL, it seems like 1 user can have many phonenumbers. and 1 phonenumber can only belong to 1 user. am i right?
CREATE TABLE User (
id INT AUTO_INCREMENT NOT NULL,
PRIMARY KEY(id)
) ENGINE = InnoDB;
CREATE TABLE users_phonenumbers (
user_id INT NOT NULL,
phonenumber_id INT NOT NULL,
UNIQUE INDEX users_phonenumbers_phonenumber_id_uniq (phonenumber_id),
PRIMARY KEY(user_id,
phonenumber_id)
) ENGINE = InnoDB;
CREATE TABLE Phonenumber (
id INT AUTO_INCREMENT NOT NULL,
PRIMARY KEY(id)
) ENGINE = InnoDB;
ALTER TABLE users_phonenumbers ADD FOREIGN KEY (user_id) REFERENCES User(id);
ALTER TABLE users_phonenumbers ADD FOREIGN KEY (phonenumber_id) REFERENCES Phonenumber(id);
can't i just simplify the database to ... below ... no need for join tables and what not?
Users (id, name)
Phonenumbers (id, user [FK], number)
Correct, these are two valid approaches to the same problem. And yes, the unique index on users_phonenumbers means that each phone number can belong to only one user.
The design is actually suboptimal.
The idea must have been that there are telephone numbers, users, and that they can be linked many-to-many. Because of the unique index on phonenumberid hoever, each number can only be assigned to one user.
Then the whole users_phonenumbers has become redundant, because they could just have added a userid column on the phonenumbers table and save themselves a join.
BAd table design if you ask me.
Summary
How do I make it easy for non-programmers to write a query such as the following?
select
table_name.*
, foreign_table_1.name
, foreign_table_2.name
from
table_name
left outer join foreign_table foreign_table_1 on foreign_table_1.id = foreign_1_id
left outer join foreign_table foreign_table_2 on foreign_table_2.id = foreign_1_id
;
Context
I have a situation like this:
create table table_name (
id integer primary key autoincrement not null
, foreign_key_1_id integer not null
, foreign_key_2_id integer not null
, some_other_column varchar(255) null
);
create table foreign_table (
id integer primary key autoincrement not null
, name varchar(255) null
);
...in which both foreign_key_1_id and foreign_key_2_id reference foreign_table. (Obviously, this is simplified and abstracted.) To query and get the respective values, I might do something like this:
select
table_name.*
, foreign_table_1.name
, foreign_table_2.name
from
table_name
left outer join foreign_table foreign_table_1 on foreign_table_1.id = foreign_1_id
left outer join foreign_table foreign_table_2 on foreign_table_2.id = foreign_1_id
;
(That is, alias foreign_table in the join to link things up correctly.) This works fine. However, some of my clients want to use SQL Maestro to query the tables. This program uses the foreign key information to link up tables using a fairly straightforward interface ("Visual Query Builder"). For instance, the user can pick multiple tables and SQL Maestro will fill in the joins, like seen here:
(source: sqlmaestro.com)
(That's a diagram from their website, just for illustration.)
This strategy works great as long as foreign keys only reference one table. The multiple-reference situation seems to be confusing it because it gneerates SQL like this:
SELECT
table_name.some_other_column,
foreign_table.name
FROM
table_name
INNER JOIN foreign_table ON (table_name.foreign_key_1_id = foreign_table.id)
AND (table_name.foreign_key_2_id = foreign_table.id)
...because the foreign keys are defined as follows:
create table table_name (
id integer primary key autoincrement not null
, foreign_key_1_id integer not null
, foreign_key_2_id integer not null
, some_other_column varchar(255) null
---------------------------
-- The part that changed:
---------------------------
, foreign key (foreign_key_1_id) references foreign_table(id)
, foreign key (foreign_key_2_id) references foreign_table(id)
);
create table foreign_table (
id integer primary key autoincrement not null
, name varchar(255) not null
);
That's a problem because you only get 1 foreign_table.name value back, whereas there are often 2 separate values.
Question
How would I go about defining foreign keys to handle this situation? (Is it possible or does it make sense to do so? I wouldn't think that it would make a big difference in constraint checking, so I've thought that that is a reason I can't find any information.) My end goal is to make querying this information easy for my clients to do by themselves, and although this situation doesn't happen every day, it's time-consuming / frustrating to have to help people through it every time it comes up.
If there isn't a way of solving my foreign key problem this way, can you suggest any alternatives? I already have some ways of getting people this information though views, but people often need to have more flexibility than that.
It seems to me that your definition is just fine, and SQL Maestro is incorrectly interpreting two foreign keys to the same table as the same foreign key, so I would alert them to that fact so that they can fix it.
I currently have multiple tables in my database which consist of the same 'basic fields' like:
name character varying(100),
description text,
url character varying(255)
But I have multiple specializations of that basic table, which is for example that tv_series has the fields season, episode, airing, while the movies table has release_date, budget etc.
Now at first this is not a problem, but I want to create a second table, called linkgroups with a Foreign Key to these specialized tables. That means I would somehow have to normalize it within itself.
One way of solving this I have heard of is to normalize it with a key-value-pair-table, but I do not like that idea since it is kind of a 'database-within-a-database' scheme, I do not have a way to require certain keys/fields nor require a special type, and it would be a huge pain to fetch and order the data later.
So I am looking for a way now to 'share' a Primary Key between multiple tables or even better: a way to normalize it by having a general table and multiple specialized tables.
Right, the problem is you want only one object of one sub-type to reference any given row of the parent class. Starting from the example given by #Jay S, try this:
create table media_types (
media_type int primary key,
media_name varchar(20)
);
insert into media_types (media_type, media_name) values
(2, 'TV series'),
(3, 'movie');
create table media (
media_id int not null,
media_type not null,
name varchar(100),
description text,
url varchar(255),
primary key (media_id),
unique key (media_id, media_type),
foreign key (media_type)
references media_types (media_type)
);
create table tv_series (
media_id int primary key,
media_type int check (media_type = 2),
season int,
episode int,
airing date,
foreign key (media_id, media_type)
references media (media_id, media_type)
);
create table movies (
media_id int primary key,
media_type int check (media_type = 3),
release_date date,
budget numeric(9,2),
foreign key (media_id, media_type)
references media (media_id, media_type)
);
This is an example of the disjoint subtypes mentioned by #mike g.
Re comments by #Countably Infinite and #Peter:
INSERT to two tables would require two insert statements. But that's also true in SQL any time you have child tables. It's an ordinary thing to do.
UPDATE may require two statements, but some brands of RDBMS support multi-table UPDATE with JOIN syntax, so you can do it in one statement.
When querying data, you can do it simply by querying the media table if you only need information about the common columns:
SELECT name, url FROM media WHERE media_id = ?
If you know you are querying a movie, you can get movie-specific information with a single join:
SELECT m.name, v.release_date
FROM media AS m
INNER JOIN movies AS v USING (media_id)
WHERE m.media_id = ?
If you want information for a given media entry, and you don't know what type it is, you'd have to join to all your subtype tables, knowing that only one such subtype table will match:
SELECT m.name, t.episode, v.release_date
FROM media AS m
LEFT OUTER JOIN tv_series AS t USING (media_id)
LEFT OUTER JOIN movies AS v USING (media_id)
WHERE m.media_id = ?
If the given media is a movie,then all columns in t.* will be NULL.
Consider using a main basic data table with tables extending off of it with specialized information.
Ex.
basic_data
id int,
name character varying(100),
description text,
url character varying(255)
tv_series
id int,
BDID int, --foreign key to basic_data
season,
episode
airing
movies
id int,
BDID int, --foreign key to basic_data
release_data
budget
What you are looking for is called 'disjoint subtypes' in the relational world. They are not supported in sql at the language level, but can be more or less implemented on top of sql.
You could create one table with the main fields plus a uid then extension tables with the same uid for each specific case. To query these like separate tables you could create views.
Using the disjoint subtype approach suggested by Bill Karwin, how would you do INSERTs and UPDATEs without having to do it in two steps?
Getting data, I can introduce a View that joins and selects based on specific media_type but AFAIK I cant update or insert into that view because it affects multiple tables (I am talking MS SQL Server here). Can this be done without doing two operations - and without a stored procedure, natually.
Thanks
Question is quite old but for modern postresql versions it's also worth considering using json/jsonb/hstore type.
For example:
create table some_table (
name character varying(100),
description text,
url character varying(255),
additional_data json
);