I have a generated SQL table which fits nicely in rows/columns, but I'd also like to store some information about that table such as:
When was it generated?
Who generated it?
What script was run to generate it?
Where did the data come from?
A list of key-pair values which didn't fit in the table and really only describe more information about the source
I'm a little new to SQL, but how would one normally store this information?
I would image that after I CREATE TABLE foo, I would need to then CREATE TABLE foo_meta, then each question will be a column and I'll need to INSERT just one row: the answers to those questions. Is that a normal way to handle this?
Some clarifications,
I'm using SQLite3
Each table represents recorded time-history data where each column is a parameter and each row is a time-step.
Each table will have associated metadata
The meta-data contains things like initial conditions and other conditions which weren't recorded in the time-history.
I will add a bunch of test-results to the same database. Each test will have a data table (time-history) and a meta-data table.
Related
I’ve hit sort of a roadblock in a current project I’m working on, I don’t have a lot of web developers in my office and as a matter in fact the only other web dev just went on vacation. Anyway I was wondering if anyone could help me with structuring two of my postgres tables.
The user needs to be able to create custom data tables, one for each specific program (a parent record). The form I’ve setup for these tables allows you to add or remove inputs based on how many fields you need and then specify the name, data_type, etc.
My initial idea was to create a new table in the dB each time a user created one of these custom tables. The other web dev, who has created something similar, said it would be better to create a fields table that stores each custom field information and then have a data table that stores every cell of data tying to a field id.
I understand having the fields table so that I can retrieve just the field information and build my front-end tables and edit forms dynamically, but I’m a little confused on how to get the data into the table. I’m used to having an array of objects and each object relating to an entire row. But with this method it’s storing each cell of data instead of row of data and I don’t know the best way to select and organize it on the backend.
Data for these tables are going to be imported in from CSV files formatted to the custom table structure, below is the current structure I have for my two tables. I got a suggestion on reddit to use JSON to store each rows data, but I'm wondering how I'll be able to do sorting and filtering with this data. My current table structure is listed below, and this is before I got the suggestion to use the json data. I'm guessing if I went that route I would remove the fieldId column and instead use it for
the JSON key name, and store that fields data with it.
fields
id -- name -- program_id -- type -- required -- position -- createdAt -- updatedAt
data
id -- fieldId -- data -- createdAt -- updatedAt
So I guess my question is does this sound like the right way to structure these tables for my needs and if so can I still perform sorting and filtering on it?
Table will be getting new data everyday from source system and i want the duplicates to be deleted automatically as soon as new data gets loaded to table.
Is it possible in bigquery?
I tried to create a view named sites_view in bigquery with below query
SELECT DISTINCT * FROM prd.sites
but duplicates not getting deleted automatically.
Below is for BigQuery:
Duplicates will not be deleted automatically - there is no such functionality in BigQuery
You should have some process to make this happen as frequently as you need or use views
Bigquery is based on append-only kind of a design. So, it accepts all the data.
This is one of the reasons there are no Primary/Unique key constraints on it, so you can't prevent duplicates from entering in the table.
So, you have to have a process like:
1.) Create a new table without duplicates from your original table.
(You can use DISTINCT/ROW_NUMBER() for doing this.)
2.) Drop original table.
3.) Rename new table with original table name.
Let me know if this information helps.
I want to merge or import a column of data from one data table created in VB.NET to another data table's 'same-named' column.
The destination data table has all the required columns created before hand but the source data table contains one column at a time, whose name is the same as that of one of the columns in destination data table.
Whatever coding I have done for the same till now, results in blank rows at starting. The destination data table looks like this:
And I want it to be displayed it as:
Regards
I worked out a way around this.
I store all the details in an 2-D array and then used that array as the data source for the grid. So instead of merging tables, keep storing different values in different columns of array.
There might be better solutions to this problem but this solution worked for me.
i am new to SQL and i have a small question. i am writing a table script and i have question about two fields in that table. Here is the Table structure :
Billing
CustomerName
CustomerPhone
BGFlag (Y/N)
UpdateIndicator (B=Before,A=After)
My question is, do i have to write script for (Y/N) in BGFlag and (B=Before,A=After)
in UpdateIndicator in the create table script. what i am thinking is i just have to create table with these column names and (Y/N), (B=Before,A=After) is the data for that two columns which i will get in sample file. Any suggestions?
Thanks
Sounds like that is just application-specific metadata about those columns. You could put that in extended properties of the table, but nobody except a curious DBA is going to see it.
Keep in mind, even if the data you are importing into your database uses Y/N and B/A, you can always transform that into a bit value (0/1), which seems a better idea from a field design perspective.
Or, if you literally want it to hold those text values (Y/N and B/A), then just use a CHAR(1) field. The risk, though, is that anyone could put any single-character text value in these columns.
When I create a view I can base it on multiple columns from different tables.
When I want to create a lookup table I need information from one table, for example the foreign key of an order table, to get customer details from another table. I can create a view having parameters to make sure it will get all data that I need. I could also - from what I have been reading - make a lookup table. What is the difference in this case and when should I choose for a lookup table?? I hope this ain't a bad question, I'm not very into db's yet ;).
Creating a view gives you a "live" representation of the data as it is at the time of querying. This comes at the cost of higher load on the server, because it has to determine the values for every query.
This can be expensive, depending on table sizes, database implementations and the complexity of the view definition.
A lookup table on the other hand is usually filled "manually", i. e. not every query against it will cause an expensive operation to fetch values from multiple tables. Instead your program has to take care of updating the lookup table should the underlying data change.
Usually lookup tables lend themselves to things that change seldomly, but are read often. Views on the other hand - while more expensive to execute - are more current.
I think your usage of "Lookup Table" is slightly awry. In normal parlance a lookup table is a code or reference data table. It might consist of a CODE and a DESCRIPTION or a code expansion. The purpose of such tables is to provide a lsit of permitted values for restricted columns, things like CUSTOMER_TYPE or PRIORITY_CODE. This category of table is often referred to as "standing data" because it changes very rarely if at all. The value of defining this data in Lookup tables is that they can be used in foreign keys and to populate Dropdowns and Lists Of Values.
What you are describing is a slightly different scenario:
I need information from one table, for
example the foreign key of an order
table, to get customer details from
another table
Both these tables are application data tables. Customer and Order records are dynamic. Now it is obviously valid to retrieve additional data from the Customer table to display along side the Order data, and in that sense Customer is a "lookup table". More pertinently it is the parent table of Order, because it has the primary key referenced by the foreign key on Order.
By all means build a view to capture the joining logic between Order and Customer. Such views can be quite helpful when building an application that uses the same joined tables in several places.
Here's an example of a lookup table. We have a system that tracks Jurors, one of the tables is JurorStatus. This table contains all the valid StatusCodes for Jurors:
Code: Value
WS : Will Serve
PP : Postponed
EM : Excuse Military
IF : Ineligible Felon
This is a lookup table for the valid codes.
A view is like a query.
Read this tutorial and you may find helpful info when a lookup table is needed:
SQL: Creating a Lookup Table
Just learn to write sql queries to get exactly what you need. No need to create a view! Views are not good to use in many instances, especially if you start to base them on other views, when they will kill performance. Do not use views just as a shorthand for query writing.