I want to pull all the rows from a database for a month and if there are any rows in the beginning or in the end that are missing then I want to create fake rows, so every day there is a schedule that one has to follow. How can I detect missing rows or how can I solve this issue?
From how to fill up the missing dates ??
fill a calendar table use that as your
driving table (i.e. first one in your
select) and do a LEFT JOIN on it.
Create and fill a calendar table
Related
I split a table with 3 billion records into 2 because of slow response. I don't need data often from before 2015. The Archived table ranges from 2011 to 2014-12-31 and the current table carries from 2015-01-01 onward. I want to create a view that connects both tables so I can:
A. Access data from the past when I need to
B. I don't compromise on performance when I only need current data
C. I can optimize or adjust the view all I want without having to change code in all apps accessing these tables.
Problem: While creating the view I create the join based on the current table and then I connect the archive table with matching fields. That seems straightforward enough except that when I put a date in the where clause that's beyond the current table date the view returns nothing. How can I connect the two tables so the view can access the two tables as if they are the same tables? Thanks in advance.
PS.: Please do suggest a better way of what I am trying to achieve if you know so.
Create view XXX....
FROM [Database].[dbo].[DA] AS DA
JOIN [Databse].[dbo].[DA_Archive] AS DA_Arc ON
DA.TradeDate = DA_Arc.TradeDate
AND DA.Node = DA_Arc.Node
AND DA.HE = DA_Arc.HE
Try this:
SELECT * FROM [Database].[dbo].[DA]
UNION ALL
SELECT * FROM [Databse].[dbo].[DA_Archive]
I have 3 tables like the below picture. I need to push data in Conditional table when it reach the ExpireDate, the database must move to Unconditional table (all codes in SQL). How can I do this?
Maybe the easiest way is to make a job that runs daily with:
insert into unconditional_table
select
*
from conditional_table a
where a.expiration_date = trunc(sysdate,'dd')
When the condition is not met - no data will be transferred.
You want to move records from a table to another.
When do you want this to happen ?
You can set up a job. This job will check the dates and move the records to another table at the beginnig of every day.
What would be the expression to create a calculated column in Table Example 2 called "SZODMAXCALC", that would contain the SZODMAXCALC from Table Example 1 given that the data from Table Example 1 falls between the dates (DTTMSTART and DTTMEND) within Table Example 2?
Maybe this is easier done on the SQL side that loads the data?
there is no way to create a calculated column that references a column in another table.
you will need to do a join either in Spotfire (via Insert...Columns)* or on the SQL-side of things (either via a view on your database or by creating a new information link in Spotfire).
the best method depends on your data structure, implementation, and desired results, so I'm not able to recommed there. take a look at both options and evaluate which one works best.
* NOTE that Spotfire cannot join based on a Calculated Column as a common key. that is, using your example, if [WELLNAME] is a calculated column, you cannot tell Spotfire the equivalent of SELECT wellname, ... FROM table_a LEFT JOIN table_b ON table_a.wellname = table_b.wellname.
the alternative is to Insert...Transformation and choose Insert New Calculated Column, and to join on that instead.
the reason for this is that calculated columns are very mutable; they could change frequently based on a user action. it would be inefficient to re-execute the join each time the column's contents changed. conversely, a "Transformation Calculated Column" is only updated when the data table is loaded.
I have a table that I created with a unique key and each other column representing one day of December 2014 (eg named D20141226 for data from 26/12/2014). So the table consists of 32 columns (key + 31 days). These daily columns are indicating that a customer had a transaction on that specific day or no transaction is indicated by a 0.
Now I want to execute the same query on a daily basis, producing a list of unique keys that had a transaction on that specific day. I used this easy script:
CREATE TABLE C01012015 AS
SELECT DISTINCT CALLING_ISDN AS A_PARTY
FROM CDRICC_012015
WHERE CALL_STA_TIME ::date = '2015-01-01'
Now my question is, how can I add the content of the new daily table to the existing table with the 31 days, making it effectively a table with 32 days of data (and then continue to do so on a daily basis to store up to 360 days of data)?
Please note that new customer are doing transactions every day hence there will unique keys in the daily table that aren't in the big table holding all the previous days.
It would be ideal if those new rows would automatically get a 0 instead of a NULL but I can work around it if it gets a NULL value (not sure how to make sure it gets a 0 instead).
I thought that a FULL OUTER JOIN would be the solution but that would mean that I have to list all variables in the select statement, which becomes quite large as I add one more column each day. Is there a more elegant way to do this?
Or is SQL just not suited to this and a programming language like eg R would be much better at this?
If you have the option to change your schema completely, you should unpivot your table so that your columns are something like CUSTOMER_ID INTEGER, D DATE, DID_TRANSACTION BOOLEAN. There's a post on the Enzee Community website that suggests using a user-defined table function (UDTF) to do this. If you change your schema in this way, a simple insert will work just fine and there will be no need to add columns dynamically.
If you can't change your schema that much but you're still able to add columns, you could add a column for every day of the year up front with a default value of FALSE (assuming it's a boolean column representing whether the customer had a transaction or not on that day). You probably want to script this.
ALTER TABLE table_with_daily_columns MODIFY COLUMN (D20140101 BOOLEAN DEFAULT FALSE);
ALTER TABLE table_with_daily_columns MODIFY COLUMN (D20140102 BOOLEAN DEFAULT FALSE);
-- etc
ALTER TABLE table_with_daily_columns ADD COLUMN (D20150101 BOOLEAN DEFAULT FALSE);
GROOM TABLE table_with_daily_columns;
When you alter a table like this, Netezza creates a new table and an internal view that does a UNION of the new table and the old. You need to GROOM the table to merge the tables back into a single one for improved performance.
If you really must keep one column per day, then you'll have to use the method you described to pivot the data from your daily transaction table. Set the default value for each of your columns to 0 or FALSE as described above, then:
INSERT INTO table_with_daily_columns
SELECT
cust_id,
TRUE as D20150101
FROM C01012015;
I'm using one of my MySQL database tables as an actual table, with times of the day as each column, and one column called day. You guessed it, in day it says the day of the week, and in the rest of the cells it says what is happening at that time.
What I want to do is only show the cells that have value in it. In my case, I'm always going to have all the rows and 2 columns full. The 2 columns are 'day' and '19:00', however in the future I might add values for '18:00' etc.
So, how can I only SELECT the columns and rows which have data in them? Some type of 'WHERE: there is data'?
Thanks!
EDIT: Picture
Having time or day as columns means that you have data in your field names. Data belongs inside the table, so you should normalise the database:
table Calendar
--------------
Day
TimeOfDay
Appointment
This way you don't get a lot of empty fields in the table, and you don't have to change the database design to add another time of day.
Now you can easily fetch only the times that exist:
select Day, TimeOfDay, Appointment from Calendar
From what I gathere you are looking something along the lines of
WHERE col1 IS NOT NULL
But it would be helpful if you could elaborate more on your schema, especially if you could draw a sample table.