I have lots of sql tables. The tables are "dependent" , i.e. constraints on foreign keys are defined between the tables.
I need to transfer the tables from sql to csv. What is correct way to do that:
Define tables exactly as they are defined in sql? (What should I do with the foreign keys?)
Try to generate other tables by joining the existing ones based on foreign keys in order to hide the foreign keys dependencies?
May be there are other options? What are the pros and cons ?
Thanks,
Note:This is need for another application that run some anylitics on the data
I would suggest to create a view in SQL which contains all information from all tables you need in your CSV later.
The view already implements the dependencies (link of two rows from different tables) and linkes all together in one table.
It would be way easier than your second proposal to create a new table because the view will do all the work for you.
I guess you will need your dependencies.
So you should not ignore them.
Here a quick example how they work:
Lets say you have 2 Tables the first one is named persons and the second one is cars. In the persons table you have 3 columns: ID, Name, Age. In the second one you have ID, Car. To see which person has which car you just check which id from the first table has which value for car in the second one.
If you link them together in a view the result is one single table with the columns ID, Person, Age, Car.
Same does the view.
Later you can simply export the view to CSV.
Maybe I can help you better if you define your needs a bit more detailed.
What kind of data is in your tables, how are they linked(what are the primary/secondary keys).
Related
I have different tables in my scheme with different columns, but I want to store data of when was the table modified or when was the data stored, so I added some columns to specify that.
I realized that I had to add the same "modification_date" and "modification_time" columns to all my tables, so I thought about making a new table called DATA_INFO so I won't need to do so, but every table has a different PRIMARY KEY and I don't know which one to add as FOREIGN KEY to the DATA_INFO table.
I don't know if I have to maybe add all of them or is there another way to do what I need.
It's better to have the same "modification_datetime" column in all tables, rather than trying to keep that data in a central table.
That's what we have done at every shop I've worked in.
I want to emphasize that a separate table is not reasonable for this purpose. The lack of an obvious foreign key is a hint.
Unlike Tab Allerman, tables that I create are much less likely to be updated, so I have three additional columns on most tables:
CreatedBy -- the user who created the row
CreatedAt -- when the row was creatd
CreatedOn -- the system where the table was created
The most important point is that this information can -- in many databases -- be implemented using default values rather than triggers. That is a big advantage of working within a single row. The fewer triggers, the better.
Ok I am very new to creating databases with Entity in mind.
I have a Master table which is going to have:
departmentID
functionID
processID
procedureID
Each of those ID's need to point to a specific list of information. Which is name, description and owner of course they link back to each ID in the master table.
My question is, should I make 4 separate tables or create one table since the information is the same in all the tables except one.
The procedureID will actually need to have an extra field for documentID to point to a specific document.
Is it possible and a good idea to make one table and add some inheritance, or is it better to make 4 separate tables?
Splitting data into a number of related tables brings many advantages over one single table. Also by having data held in separate tables, it is simple to add records that are not yet needed but may be in the future. You can also create your corresponding objects for each table in your code. Also it would be more difficult to split the data into separate tables in the future if somehow you need to do that.
I'm trying to create a relationship between two tables in powerpivot. However, my tables don't have any keys. What I would like to do is create a SQL-Unique-Constraint-like relationship, which is based upon multiple values combined, being the key.
For example:
Table1 columns are First, Last, Address, Phone
Table2 columns are the same.
I want to create a relationship in excel that is the equivalent of
select * from Table1 full join Table2 on 1.Fist=2.First and 1.Last=2.Last and 1.Address=2.Address
However, the create relationship dialogue doesn't allow multiple columns to selected. I tried going the route of just creating multiple 1-column relationships. However, relationships also cannot include columns were there are duplicate values in the column.
I have a feeling I may just be approaching accomplishing this from the wrong direction. Any help is appreciated! Thank you.
Zee,
You are right that PowerPivot does not natively support multi-column relationships. There are however 2 work arounds:
Add a key to each table of the respective columns concatenated together and providing this is unique in at least one the relationship can be created. If you have a situation where neither table has unique keys then an intermediate table of unique keys could be created using SQL.
Technically multiple relationships can be created between tables but only one can be active. There is a DAX function called USERELATIONSHIP() which can use inactive relationships. This is an advanced technique.
Your solution may well be to combine the two tables in your source SQL query.
Jacob
If all you want to do is inner join using 2 or more columns, please consider creating a calculated column that concatenates the 2 or 3 columns in each of the 2 tables and then create a relationship between them.
I have had similar cases and used this technique.
I have a problem with finding a way to represent multiple tables hash tables into a single table.
Say I have 3 tables with the format:
Table1(Table1_PK1,Table1_PK2,Table1_PK3,Table1_Hash)
Table2(Table2_PK1,Table2_PK2,Table2_Hash)
Table3(Table3_Pk1,Table3_PK2,Table3_PK3,Table3_PK4,Table3_PK5,Table3_Hash)
Table1_PK1,Table1_PK2,Table1_PK3... are columns and they might have different datatypes (VARCHAR, INT or DATETIME ...).
My question is if there is a way to create a single table (fixed number of columns) that can represent all of these 3 tables (may be more in practical).
I am trying to do this for my database tool. Each table actual a table which contains primary keys and a hash data associating with them.
Since you're apparently building a database tool, not a database, it might make more sense to do this in application code rather than in a database table.
In a different answer, you commented
I am still looking for a dynamic way to do it without knowing how many primary keys a table can have.
A table can have only one primary key. That primary key can consist of more than one column, though. (You already knew this; you were just using the wrong words, which might confuse others.)
A table can also have an arbitrary number of other keys, which will be either declared (as NOT NULL UNIQUE) or "undeclared" (by creating an index that guarantees uniqueness over a set of columns).
You can look all that stuff up at run time in one or both of two ways. (Links go to documentation for PostgreSQL.)
System tables, sometimes called system catalogs
information_schema views
As far as I know, all modern SQL platforms implement at least one of these interfaces. The information_schema views are covered in the SQL standards, but there seems to be some room for interpretation. They don't look quite the same on all platforms.
Why combine the 3 tables into one? Would be really bad db design. But here's a way to do it:
The one table will have a column for each of the 3 tables' columns you want in the final table. I am making the assumption that TableX_Hash is the same type, so that remains as one unique column:
Table_All_in_One (
Table1_PK1,
Table1_PK2,
Table1_PK3,
# space just for clarity of grouping
Table2_PK1,
Table2_PK2,
Table3_PK1,
Table3_PK2,
Table3_PK3,
Table3_PK4,
Table3_PK5,
TableX_Hash # Assuming all the _Hash'es are the same type+length,
# otherwise, add Table1_Hash, Table2_Hash, Table3_Hash
# This can be your new primary key
)
The Primary Keys (PKx) are required to be non-NULL only in their own tables. For this table, they have to allow nulls. The idea is that each row of this new table will only hold the data for one of the tables. The other columns will be empty for that row. If you want to associate the row of one table with another, you can add that to the same row or add FK_Table1_Hash, FK_Table2_Hash and FK_Table3_Hash columns which will refer to the TableX_Hash value of a record.
PS: I wonder if what you are really looking for is a View and not this really bad all-in-one table.
Edit: Combining them into one "without knowing how many primary keys a table can have." as per your comment:
Store all the _PKs concatenated into one column:
Table_All_in_One (
New_PK,
TableX_Hash,
Table1_PKx, # Concatenated PKs of Table1
Table2_PKx, # Concatenated PKs of Table2, etc.
...,
# OR just one
TableX_PKs, # concatenate all the PK's into one VARCHAR field
# Add a pipe `|` between them optionally.
Table_Num # If using just one, then you'll need to store the table number
)
You will not be able to conveniently pick records based on part of their composite primary key. It will always have to be TableX_PKs = CONCAT_WS('|', Table1_PK1, Table1_PK2, ...). So your only dependency is the number of PKs in the original column.
In order to model a bunch of tables you will need 3 tables. An entity table that contains the table names of the tables you wish to set up this way called a factor or entity table. A Factor_detail table that contains all the columns and their associated properties of the tables. A table, factor_detail_value, for storing things like lookup values for lookup tables. I'm trying to learn more about this myself as well because we are using this technique at work as well. Genrate sql on the fly for any table so encoded, and store the data in a repository pertiinant to the data itself. This way if a table changes and you need to add a column or change a datatype, you can add a row to the factor detail table without affecting a database shut down in production. In most businesses a four hour shut down to make a sql data table change can cost thousands of dollars. If you are dealing with insurance for example, each additional state that you sell insurance in has different requirements for being able to seel it and that will result in table changes. We reduced our table count way down from over 700 tables in this manner also we can make changes without database shut down thus avoiding the loss in revenue.
When I create a view I can base it on multiple columns from different tables.
When I want to create a lookup table I need information from one table, for example the foreign key of an order table, to get customer details from another table. I can create a view having parameters to make sure it will get all data that I need. I could also - from what I have been reading - make a lookup table. What is the difference in this case and when should I choose for a lookup table?? I hope this ain't a bad question, I'm not very into db's yet ;).
Creating a view gives you a "live" representation of the data as it is at the time of querying. This comes at the cost of higher load on the server, because it has to determine the values for every query.
This can be expensive, depending on table sizes, database implementations and the complexity of the view definition.
A lookup table on the other hand is usually filled "manually", i. e. not every query against it will cause an expensive operation to fetch values from multiple tables. Instead your program has to take care of updating the lookup table should the underlying data change.
Usually lookup tables lend themselves to things that change seldomly, but are read often. Views on the other hand - while more expensive to execute - are more current.
I think your usage of "Lookup Table" is slightly awry. In normal parlance a lookup table is a code or reference data table. It might consist of a CODE and a DESCRIPTION or a code expansion. The purpose of such tables is to provide a lsit of permitted values for restricted columns, things like CUSTOMER_TYPE or PRIORITY_CODE. This category of table is often referred to as "standing data" because it changes very rarely if at all. The value of defining this data in Lookup tables is that they can be used in foreign keys and to populate Dropdowns and Lists Of Values.
What you are describing is a slightly different scenario:
I need information from one table, for
example the foreign key of an order
table, to get customer details from
another table
Both these tables are application data tables. Customer and Order records are dynamic. Now it is obviously valid to retrieve additional data from the Customer table to display along side the Order data, and in that sense Customer is a "lookup table". More pertinently it is the parent table of Order, because it has the primary key referenced by the foreign key on Order.
By all means build a view to capture the joining logic between Order and Customer. Such views can be quite helpful when building an application that uses the same joined tables in several places.
Here's an example of a lookup table. We have a system that tracks Jurors, one of the tables is JurorStatus. This table contains all the valid StatusCodes for Jurors:
Code: Value
WS : Will Serve
PP : Postponed
EM : Excuse Military
IF : Ineligible Felon
This is a lookup table for the valid codes.
A view is like a query.
Read this tutorial and you may find helpful info when a lookup table is needed:
SQL: Creating a Lookup Table
Just learn to write sql queries to get exactly what you need. No need to create a view! Views are not good to use in many instances, especially if you start to base them on other views, when they will kill performance. Do not use views just as a shorthand for query writing.