Background
We have a very old database design. The worst example is the config table. It's one row with a separate column for each value. It should be a key value pair setup (obviously). It's used everywhere in both our old legacy code, and now in our new web based code. Horrifyingly there are now over 300 columns and worse more are added every now and again.
Goal
I'd love to chuck the table in the bin and replace it with a correct key value pair. However coding time is short so I'm trying to come up with a clever way of displaying it as the old one row for the old legacy code retrieving it from a key value pair.
Attempted Solution
Ordinarily I'd use a view - but given that the best code to display it (pinched from here: https://stackoverflow.com/a/15745076/12059261) is dynamic SQL, I've come unstuck as I cannot use dynamic SQL in a view (my searching tells me that views do not allow dynamic SQL - I'm assuming this is correct?)
Table example
CurrentConfig
Key Name Address etc.
1 My Company My Address etc.
WantedConfig
ID Key Value
1 Name MyCompany
2 Address My Address
3 etc. etc.
I want to change the table from its current form to the new form, but still be able to display it as the current form to legacy code.
If the issue is displaying the new config to legacy code, then use conditional aggregation:
create view oldconfig as
select max(case when key = 'Name' then value end) as name,
max(case when key = 'Address' then value end) as address,
. . .
from newconfig;
Don't worry about dynamic SQL. There are a fixed set of columns used in the legacy code and that is what needs to go in the view. If you add new keys, they will not go in the view, but presumably the new keys are not used by the old code anyway.
You can use apply :
select tt.id, tt.Key, tt.Val
from table t cross apply
( values (1, 'Name', Name),
(2, 'Address', Address),
(3, 'etc.', etc)
) tt(id, Key, Val)
Related
Is there are one or some reliable variants to solve easy task?
I've got a number of XML files which will be converting into 6 SQL tables (via SSIS).
Before the end of this process i need to add a new (in fact - common for all tables) column (or field) into each of them.
This column represents ID with assigning range and +1 incrementing step. Like (350000, 1)
Yes, i know how to solve it on SSMS SQL stage. But i need a solution at SSIS's pre-SQL converting lvl.
I'm sure there should be well-known pattern-solutions to deal with it.
I am going to take a stab at this. Just to be clear, I don't have a lot of information in your question to go on.
Most XML files that I have dealt with have a common element (let's call it a customer) with one to many attributes (this can be invoices, addresses, email, contacts, etc).
So your table structure will be somewhat star shaped around the customer.
So your XML will have a core customer information on a 1 to 1 basis that can be loaded into a single main table, and will have array information of invoices and an array of addresses etc. Those arrays would be their own tables referencing the customer as a key.
I think you are asking how to create that key.
Load the customer data first and return the identity column to be used as a foreign key when loading the other tables.
I find it easiest to do so in script component. I'm only going to explain how to get the key back. I personally would handle the whole process in C# (deserializing and all).
Add this to Using Block:
Using System.Data.OleDB;
Add this into your main or row processing depending on where the script task / component is:
string SQL = #"INSERT INTO Customer(CustName,field1, field2,...)
values(?,?,?,...); Select cast(scope_identity() as int);";
OleDBCommanad cmd = new OleDBCommand();
cmd.CommandType = System.Data.CommandType.Text;
cmd.CommandText = SQL;
cmd.Parameters.AddWithValue("#p1",[CustName]);
...
cmd.Connection.Open();
int CustomerKey = (int)cmd.ExecuteScalar(); //ExecuteScalar returns the value in first row / first column which in our case is scope_identity
cmd.Connection.Close();
Now you can use CustomerKey for all of the other tables.
In a recipe database I have two tables. One has the ingredients of every recipe [Recipe_ingr] and the other the available measures and weight for every ingredient [Weight2].
When I input a new ingredient for a recipe, I would like to be able to choose the available units for only that specific food.
I have tried with this expression in the control field but it prompts me to choose first, and then the options remain the same for all the records, not changing dinamically according to the record ingredient code.
SELECT [Weight2].[Msre_Desc], [Weight2].[Gm_Wgt] FROM Weight2 WHERE Weight2.NDB_No Like Recipe_Ingr.NDB_No ORDER BY [Msre_Desc], [Gm_Wgt];
Picture of my tables
Update:
I tried the syntax change suggested by June9 but still the control doesn't update automatically with every record as you can see in this picture: Table
Suggest you name controls different from fields they are bound to, like tbxNDB. The SQL needs to reference a field or control that is on the form. Also, LIKE operator without wildcard accomplishes nothing that an = sign wouldn't. Also recommend not using exactly same name for fields in multiple tables.
If you use that SQL statement in combobox RowSource, try:
SELECT Msre_Desc, Gm_Wgt FROM Weight2 WHERE NDB_No = [tbxNDB] ORDER BY Msre_Desc, Gm_Wgt;
You want to save Msre_Desc as foreign key, not a record id generated by autonumber?
I am sorry for the lngthy question, but it has to be precisely described, if it is to be answered.
I am building a schema-and-data application in SQL Server and .NET winforms.
Table ItemType holds the type of items, table ItemTypeColumn holds the columns for each type and finally, table ItemData holds all the data of the application.
An example of these would be:
<ul>
<li>ItemType: Customer, Customer Category, etc</li>
<li>ItemColumn: Customer Name, Description etc</li>
<li>ItemData: John Doe, International, etc</li>
</ul>
This leads to a very interesting chain of data retrieval. For instance:
<ul>
<li>Customer X is item ID 100</li>
<li>Category Y is item ID 60</li>
<li>To indicate that Customer X is of category Y - (to point towards it) we need to find the line that combines ItemID=100, ItemColumnID=[whatever the id of the item ItemColumn is] and then update the field Data with the ID of Category X (60).</li>
</ul>
I retrieve the data with a Select statement as follows:
SELECT * FROM (
SELECT ItemData.ItemID,
IC.ItemTypeID,
MAX(CASE WHEN ItemData.ItemColumnID = 28
THEN ItemData.Data ELSE NULL END) AS "Name",
MAX(CASE WHEN ItemData.ItemColumnID = 32
THEN ItemData ELSE NULL END)
AS "Code"
FROM ItemData as Data
INNER JOIN (
select *
from ItemColumns
where ItemTypeID=7) as IC
on ItemData.ItemColumnID=IC.ID
GROUP BY ItemData.ItemID, IC.ItemTypeID) as table1
INNER JOIN Item ON Item.ID=table1.ItemID
This works like a charm.
Now, I want to give my user the ability to create calculated columns with ability to select:
<ul>
<li>A column from this type item</li>
<li>A column from a parent type item</li>
<li>A value from a child type item </li>
<li>The specific data found in a specific row id</li>
</ul>
Here 's an example:
<ul>
<li>Item Type "Country" has 1 field: Name</li>
<li>Item Type "City" has 3 fields: Name, Population, Country (pointing to parent country)</li>
</ul>
I ' d like to give the user ability to create new columns that will hold calculated data (as opposed to data entry, like "name" or "decription").
I have managed to create an expression builder and a parser that actually works. Taking the above example into account, you can create a column Urban Population in type Country that brings the sum of column Population of all "children" Cities for each country. This, I accomplished with User Defined Functions, and virtual columns which I call in the select statement.
Here's the problem (finally): if I want to create a calculated column that references another calculated column, say Rural Population - that would show the Country Population (numeric field) minus the sum of Urban Cities (sum of population field for "child cities") it will not work because the newly "created" (in the selecte table) column cannot be referenced by peers.
I sense that I am on the wrong track, in general, concerning the calculated fields. Are there any best practices to follow? Is my approach wrong? Is there a workaround for the calculation-in-calculation error?
Thank you in advance - again, I am sorry for the length of this question.
The approach you are taking in database design is "Name/Key value pair" . Infact you are going one step further also storing the aggregate data in the same name value pair structure. This approach looks very flexible and is very tempting , however the RDBMS by architecture not design for this approach.
This works very well with small data does not with big data. Hence design of this kid mostly fails in production within few months of release.
Please refer following link :
http://geekswithblogs.net/darrengosbell/archive/2006/03/12/KVPsInDatabaseDesign.aspx
it is always suggested to design RDBMS with explicit column name and definition.
I have usually notice number and type of aggregation needs are not very complex and very different and mostly they can be determined in advance . Hence discouraging this approach of yours.
In some scenarios in which the flexibility is absolute needed , one can use external tools like : tableau, R or Python Panda/iPython etc. I understand for these tool to use the users needs to learn them.
It is really commendable Nassosk , that you could achieve all what you have told in your original post, i will interested in seeing your code :-).
Looks like you are designing a database over database :-)
Thank you
From what I 've read here and elsewhere my question is like asking "how should I go about if I want to jump off a skyscraper" - most people will tell you not to jump, instead of giving you their 5 cents :)
In any case, and since I 've put a lot of work to it up to now, I thought I might fail with dignity and go all the way, so here's the answer:
Since my select statement actually returns a virtual table (it transposes the data) it seemed totally plausible to add all the related tables in a dataset, create the relations among them on the fly in the dataset (yes, #user3851404 I am building a database over a database, it's quite rewarding though) and set the Expression property equal to my formula in the datacolumns that I want to display derrived data.
It actually works as expected. I will not comment on performance because I haven't stress-tested it yet, but whatever the outcome regarding performance, it seems that this is the only workaround.
I have two databases, Database A and Database B.
Database A contains some data which needs to be placed in a table in Database B. However, before that can happen, some of that data must be “cleaned up” in the following way:
The table in Database A which contains the data to be placed in Database B has a field called “Desc.” Every now and then the users of the system put city names in with the data they enter into the “Desc” field. For example: a user may type in “Move furniture to new cubicle. New York. Add electric.”
Before that data can be imported into Database B the word “New York” needs to be removed from that data so that it only reads “Move furniture to new cubicle. Add electric.” However—and this is important—the original data in Database A must remain untouched. In other words, Database A’s data will still read “Move furniture to new cubicle. New York. Add electric,” while the data in Database B will read “Move furniture to new cubicle. Add electric.”
Database B contains a table which has a list of the city names which need to be removed from the “Desc” field data from Database A before being placed in Database B.
How do I construct a stored procedure or function which will grab the data from Database A, then iterate through the Cities table in Database B and if it finds a city name in the “Desc” field will remove it while keeping the rest of the information in that field thus creating a recordset which I can then use to populate the appropriate table in Database B?
I have tried several things but still haven’t cracked it. Yet I’m sure this is probably fairly easy. Any help is greatly appreciated!
Thanks.
EDIT:
The latest thing I have tried to solve this problem is this:
DECLARE #cityName VarChar(50)
While (Select COUNT(*) From ABCScanSQL.dbo.tblDiscardCitiesList) > 0
Begin
Select #cityName = ABCScanSQL.dbo.tblDiscardCitiesList.CityName FROM ABCScanSQL.dbo.tblDiscardCitiesList
SELECT JOB_NO, LTRIM(RTRIM(SUBSTRING(JOB_NO, (LEN(job_no) -2), 5))) AS LOCATION
,JOB_DESC, [Date_End] , REPLACE(Job_Desc,#cityName,' ') AS NoCity
FROM fmcs_tables.dbo.Jobt WHERE Job_No like '%loc%'
End
"Job_Desc" is the field which needs to have the city names removed.
This is a data quality issue. You can always make a copy of the [description] in Database A and call it [cleaned_desc].
One simple solution is to write a function that does the following.
1 - Read data from [tbl_remove_these_words]. These are the phrases you want removed.
2 - Compare the input - #var_description, to the rows in the table.
3 - Upon a match, replace with a empty string.
This solution depends upon a cleansing table that you maintain and update.
Run a update query that uses the input from [description] with a call to [fn_remove_these_words] and sets [cleaned_desc] to the output.
Another solution is to look at products like Melisa Data (DQ) product for SSIS or data quality services in the SQL server stack to give you a application frame work to solve the problem.
I have an application that uses Oracle Apex 4.2 . It has a form ( form and report on a table) that needs to display descriptions for columns on the table. For instance, there is a column on the table called fund which has a numeric value ( 1 to 6). There is a separate table that gives a description for each of these 6 values. Under EDIT PAGE ITEM, under SOURCE, I chose SOURCE TYPE -> SQL QUERY
I entered this query below:
SELECT DESCRIPTION FROM
"#OWNER#"."BU19ANT",
"#OWNER#"."FUNDCD"
WHERE ANTFUNDCD = CODE
where BU19ANT is the table that used for this form
FUNDCD is the name of the look up table
ANTFUNDCD and CODE and numeric fields on the respective tables and DESCRIPTION is the value that I want to look up and display on the form.
This gives me the correct answer MOST of the time, but not all the time.
The key to the table ( and the field used to link from the report to the form) is the Soc Security Number. If I run this same query against the Oracle table hard coding the SS Number, I always get the correct answer.
This form has 5 look ups that work this way and they all have the same problem.
I assume that I DONT need to include the Social Security Number as part of the query Apex already knows that.
But I tried to add that and can not figure out how to code it.
I tried
WHERE ANTSOCIALSECURITYNUMBER ( column on table) = P2_SOCIALSECURITYNUMBER ( the item on this page)
but that gave this error
ORA-00904: "P2_SOCIALSECURITYNUMBER ": invalid identifier
Is there some other way to code this? Or to say where SS Number = current record?
Or am I on the wrong track here?
Try :P2_SOCIALSECURITYNUMBER (for items on session) or &P2_SOCIALSECURITYNUMBER. (for items on page)