Calculated Column Editor - sql

Question

I am sorry for the lngthy question, but it has to be precisely described, if it is to be answered.
I am building a schema-and-data application in SQL Server and .NET winforms.
Table ItemType holds the type of items, table ItemTypeColumn holds the columns for each type and finally, table ItemData holds all the data of the application.
An example of these would be:
<ul>
<li>ItemType: Customer, Customer Category, etc</li>
<li>ItemColumn: Customer Name, Description etc</li>
<li>ItemData: John Doe, International, etc</li>
</ul>
This leads to a very interesting chain of data retrieval. For instance:
<ul>
<li>Customer X is item ID 100</li>
<li>Category Y is item ID 60</li>
<li>To indicate that Customer X is of category Y - (to point towards it) we need to find the line that combines ItemID=100, ItemColumnID=[whatever the id of the item ItemColumn is] and then update the field Data with the ID of Category X (60).</li>
</ul>
I retrieve the data with a Select statement as follows:
SELECT * FROM (
SELECT ItemData.ItemID,
IC.ItemTypeID,
MAX(CASE WHEN ItemData.ItemColumnID = 28
THEN ItemData.Data ELSE NULL END) AS "Name",
MAX(CASE WHEN ItemData.ItemColumnID = 32
THEN ItemData ELSE NULL END)
AS "Code"
FROM ItemData as Data
INNER JOIN (
select *
from ItemColumns
where ItemTypeID=7) as IC
on ItemData.ItemColumnID=IC.ID
GROUP BY ItemData.ItemID, IC.ItemTypeID) as table1
INNER JOIN Item ON Item.ID=table1.ItemID
This works like a charm.
Now, I want to give my user the ability to create calculated columns with ability to select:
<ul>
<li>A column from this type item</li>
<li>A column from a parent type item</li>
<li>A value from a child type item </li>
<li>The specific data found in a specific row id</li>
</ul>
Here 's an example:
<ul>
<li>Item Type "Country" has 1 field: Name</li>
<li>Item Type "City" has 3 fields: Name, Population, Country (pointing to parent country)</li>
</ul>
I ' d like to give the user ability to create new columns that will hold calculated data (as opposed to data entry, like "name" or "decription").
I have managed to create an expression builder and a parser that actually works. Taking the above example into account, you can create a column Urban Population in type Country that brings the sum of column Population of all "children" Cities for each country. This, I accomplished with User Defined Functions, and virtual columns which I call in the select statement.
Here's the problem (finally): if I want to create a calculated column that references another calculated column, say Rural Population - that would show the Country Population (numeric field) minus the sum of Urban Cities (sum of population field for "child cities") it will not work because the newly "created" (in the selecte table) column cannot be referenced by peers.
I sense that I am on the wrong track, in general, concerning the calculated fields. Are there any best practices to follow? Is my approach wrong? Is there a workaround for the calculation-in-calculation error?
Thank you in advance - again, I am sorry for the length of this question.

Answer 1

The approach you are taking in database design is "Name/Key value pair" . Infact you are going one step further also storing the aggregate data in the same name value pair structure. This approach looks very flexible and is very tempting , however the RDBMS by architecture not design for this approach.
This works very well with small data does not with big data. Hence design of this kid mostly fails in production within few months of release.
Please refer following link :
http://geekswithblogs.net/darrengosbell/archive/2006/03/12/KVPsInDatabaseDesign.aspx
it is always suggested to design RDBMS with explicit column name and definition.
I have usually notice number and type of aggregation needs are not very complex and very different and mostly they can be determined in advance . Hence discouraging this approach of yours.
In some scenarios in which the flexibility is absolute needed , one can use external tools like : tableau, R or Python Panda/iPython etc. I understand for these tool to use the users needs to learn them.
It is really commendable Nassosk , that you could achieve all what you have told in your original post, i will interested in seeing your code :-).
Looks like you are designing a database over database :-)
Thank you

Answer 2

From what I 've read here and elsewhere my question is like asking "how should I go about if I want to jump off a skyscraper" - most people will tell you not to jump, instead of giving you their 5 cents :)
In any case, and since I 've put a lot of work to it up to now, I thought I might fail with dignity and go all the way, so here's the answer:
Since my select statement actually returns a virtual table (it transposes the data) it seemed totally plausible to add all the related tables in a dataset, create the relations among them on the fly in the dataset (yes, #user3851404 I am building a database over a database, it's quite rewarding though) and set the Expression property equal to my formula in the datacolumns that I want to display derrived data.
It actually works as expected. I will not comment on performance because I haven't stress-tested it yet, but whatever the outcome regarding performance, it seems that this is the only workaround.