Source table contains data for the last 2 years, e.g.:
column name: aug_16, sep_16 ... oct_17 ... jul_18, aug_18
Column name changes every month - one column is added and one is deleted.
For example, next month column started from:
sep_16 to sep_18 which means that aug_16 will be deleted and sep_18 will be added.
So my issue is: I want to copy data to another Interface table and want to do mapping in ODI jobs before loading to Base table. How should I handle dynamic column name?
I'd say that your best option is to abandon such a data model and create a new one. Instead of doing everything in DDL, do it in DML. What I mean to say: alter table and add a DATE datatype column. It will show which month/year that row belongs to. Then, it is a simple matter of a WHERE clause which will handle data.
Instead of: update interface set sep_16 = (select sep_16 from base)
You would : insert into interface select * from base
where to_char(date_column, 'yyyymm') = '201609'
This is just an example, written for simplicity of the general idea; such a WHERE will ruin possible index on DATE_COLUMN, but that can be handled (by using BETWEEN first and last day of the month, for example)
Related
Table info:
I want to add new column and calculated the different of the alarmTime column with this code:
ALTER TABLE [DIALinkDataCenter].[dbo].[DIAL_deviceHistoryAlarm]
ADD dif AS (DATEDIFF(HOUR, LAG((alarmTime)) OVER (ORDER BY (alarmTime)), (alarmTime)));
How to add the calculation on the table? Because always there's error like this:
Windowed functions can only appear in the SELECT or ORDER BY clauses.
You are using the syntax for a generated virtual column that shows a calculated value (ADD columnname AS expression).
This, however, only works on values found in the same row. You cannot have a generated column that looks at other rows.
If you consider now to create a normal column and fill it with calculated values, this is something you shouldn't do. Don't store values redundantly. You can always get the difference in an ad-hoc query. If you store this redundantly instead, you will have to consider this in every insert, update, and delete. And if at some time you find rows where the difference doesn't match the time values, which column holds the correct value then and which the incorrect one? alarmtime or dif? You won't be able to tell.
What you can do instead is create a view for convenience:
create view v_dial_devicehistoryalarm as
select
dha.*,
datediff(hour, lag(alarmtime) over (order by alarmtime), alarmtime) as dif
from dial_devicehistoryalarm dha;
Demo: https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=b7f9b5eef33e72955c7f135952ef55b5
Remember though, that your view will probably read and sort the whole table everytime you access it. If you query only a certain time range, it will be faster hence to calculate the differences in your query instead.
I have a database containing tables filled with information about transactions which took place in specific months and years.
A new table is created at the start of every month and follows the naming convention as follows 'tblposhistory201811'. Each table created at the start of every month has the same structure, the only difference being the name of the table and the data inside.
The first part of the table name is always the same 'tblposhistory' the second part of the table name changes every month and year. For instance '201811' means that the table was created for the transactions which took place in November of 2018.
I have managed to already be able to get the names of all the tables which follow this convention in the database. I have tried to look at a way to loop through the values I have collected but SQL only supports WHILE loops which makes my life tricky as I do not know how the values I collected with my code are stored in the variable table_names. Is it as a string or are they unique values which can be iterated over?
**SELECT** name **AS** table_names
**FROM** sys.tables
**WHERE** sys.tables.name **LIKE** 'tblposhistory%';
What I would like to be able to do is select all of the tables and their data which follow this naming convention, I would like to then be able to select more specific things from this data.
background
I am trying to create an SSRS report which will run select * on a table passed in as a parameter and display all the data from that table. As I understand it, I can't use a table for this. I want to use a pivot table to achieve this.
select * from #table will return something like this (from the adventureworks DB)
I want to display the date in this format:
Question
How do I achieve this? I looked at using PIVOT/UNPIVOT, but all the examples I've seen use static column names and aggregates.
I won't know the column names at design time (or run time), I'm assuming I will need column headers like 'table name', 'value1', 'value2' etc?
Limitations
I won't have access to create stored procedures. Ideally, the report should be able to be run entirely from SSRS without having to create new tables etc.
Performance is not a concern.
Edit
Editing to add some clarity. The column names in the example above are only an example. The #table parameter could be any table, column names won't be know at design time. The column names could be col1, col2, or name, address... etc.
You can do this with a normal tablix, with a column group on BusinessEntityID
Create a normal tablix.
Remove the row group (group only)
Insert a new parent column group, grouped by BusinessEntityID
Delete the top row (row only)
Add each of the row titles in, as per your suggested output
Add each of the values in the second column
What would be the expression to create a calculated column in Table Example 2 called "SZODMAXCALC", that would contain the SZODMAXCALC from Table Example 1 given that the data from Table Example 1 falls between the dates (DTTMSTART and DTTMEND) within Table Example 2?
Maybe this is easier done on the SQL side that loads the data?
there is no way to create a calculated column that references a column in another table.
you will need to do a join either in Spotfire (via Insert...Columns)* or on the SQL-side of things (either via a view on your database or by creating a new information link in Spotfire).
the best method depends on your data structure, implementation, and desired results, so I'm not able to recommed there. take a look at both options and evaluate which one works best.
* NOTE that Spotfire cannot join based on a Calculated Column as a common key. that is, using your example, if [WELLNAME] is a calculated column, you cannot tell Spotfire the equivalent of SELECT wellname, ... FROM table_a LEFT JOIN table_b ON table_a.wellname = table_b.wellname.
the alternative is to Insert...Transformation and choose Insert New Calculated Column, and to join on that instead.
the reason for this is that calculated columns are very mutable; they could change frequently based on a user action. it would be inefficient to re-execute the join each time the column's contents changed. conversely, a "Transformation Calculated Column" is only updated when the data table is loaded.
I have a table that I created with a unique key and each other column representing one day of December 2014 (eg named D20141226 for data from 26/12/2014). So the table consists of 32 columns (key + 31 days). These daily columns are indicating that a customer had a transaction on that specific day or no transaction is indicated by a 0.
Now I want to execute the same query on a daily basis, producing a list of unique keys that had a transaction on that specific day. I used this easy script:
CREATE TABLE C01012015 AS
SELECT DISTINCT CALLING_ISDN AS A_PARTY
FROM CDRICC_012015
WHERE CALL_STA_TIME ::date = '2015-01-01'
Now my question is, how can I add the content of the new daily table to the existing table with the 31 days, making it effectively a table with 32 days of data (and then continue to do so on a daily basis to store up to 360 days of data)?
Please note that new customer are doing transactions every day hence there will unique keys in the daily table that aren't in the big table holding all the previous days.
It would be ideal if those new rows would automatically get a 0 instead of a NULL but I can work around it if it gets a NULL value (not sure how to make sure it gets a 0 instead).
I thought that a FULL OUTER JOIN would be the solution but that would mean that I have to list all variables in the select statement, which becomes quite large as I add one more column each day. Is there a more elegant way to do this?
Or is SQL just not suited to this and a programming language like eg R would be much better at this?
If you have the option to change your schema completely, you should unpivot your table so that your columns are something like CUSTOMER_ID INTEGER, D DATE, DID_TRANSACTION BOOLEAN. There's a post on the Enzee Community website that suggests using a user-defined table function (UDTF) to do this. If you change your schema in this way, a simple insert will work just fine and there will be no need to add columns dynamically.
If you can't change your schema that much but you're still able to add columns, you could add a column for every day of the year up front with a default value of FALSE (assuming it's a boolean column representing whether the customer had a transaction or not on that day). You probably want to script this.
ALTER TABLE table_with_daily_columns MODIFY COLUMN (D20140101 BOOLEAN DEFAULT FALSE);
ALTER TABLE table_with_daily_columns MODIFY COLUMN (D20140102 BOOLEAN DEFAULT FALSE);
-- etc
ALTER TABLE table_with_daily_columns ADD COLUMN (D20150101 BOOLEAN DEFAULT FALSE);
GROOM TABLE table_with_daily_columns;
When you alter a table like this, Netezza creates a new table and an internal view that does a UNION of the new table and the old. You need to GROOM the table to merge the tables back into a single one for improved performance.
If you really must keep one column per day, then you'll have to use the method you described to pivot the data from your daily transaction table. Set the default value for each of your columns to 0 or FALSE as described above, then:
INSERT INTO table_with_daily_columns
SELECT
cust_id,
TRUE as D20150101
FROM C01012015;