I'm using Tableau 9.0 to connect to an Amazon Redshift database. The data is such a high volume that only the last 4 months is stored. Each month, a new table is created, named with the current month, and the table from 5 months ago is dropped.
So what we have right now is 4 tables:
events_jun
events_jul
events_aug
events_sep
The custom SQL that I want to define will use the UNION ALL function to merge all 4 tables together into a single query. The issue is that the table names will need to be changed month over month.
Is there a way to define variables based on CURDATE, such that you can take CURDATE() - 90 and extract the first 3 letters, then append it to the string "events_" in order to create a concatenated string that is the table name you want.
Please note, I have tried to resolve the problem by including a parameter in the custom SQL, as described here: http://onlinehelp.tableau.com/current/pro/online/mac/en-us/customsql.html
This does not work, because Tableau will surround the Paramter value with single quotes, which results in events_'jun' which of course throws an error, because there is not such table.
Related
I have several tables that contain sensordata that consist of 2 colums:
timestamp | value
I want a second table that adds a new line when a new value comes into one of the above tables, that takes the last xx values, calculates the median, and adds this line.
How is this best done in BQ / Scheduled query? I know how to calculate a median, just hardest is to do the append 'on schedule'.
For append data into a table in a scheduled query, you require to set the Append to table option in the Destination table write preference when the scheduled query is configured in the by BigQuery UI:
Source table contains data for the last 2 years, e.g.:
column name: aug_16, sep_16 ... oct_17 ... jul_18, aug_18
Column name changes every month - one column is added and one is deleted.
For example, next month column started from:
sep_16 to sep_18 which means that aug_16 will be deleted and sep_18 will be added.
So my issue is: I want to copy data to another Interface table and want to do mapping in ODI jobs before loading to Base table. How should I handle dynamic column name?
I'd say that your best option is to abandon such a data model and create a new one. Instead of doing everything in DDL, do it in DML. What I mean to say: alter table and add a DATE datatype column. It will show which month/year that row belongs to. Then, it is a simple matter of a WHERE clause which will handle data.
Instead of: update interface set sep_16 = (select sep_16 from base)
You would : insert into interface select * from base
where to_char(date_column, 'yyyymm') = '201609'
This is just an example, written for simplicity of the general idea; such a WHERE will ruin possible index on DATE_COLUMN, but that can be handled (by using BETWEEN first and last day of the month, for example)
This question already has answers here:
How does sql server sort your data?
(4 answers)
Closed 5 years ago.
I am facing one issue I cant handle yet.
Here's the deal: I am working on a program which should monitor employees working hours. So far, I created a SQL Server table called TablicaSQL with 4 columns:
Id, Ime (Name), Datum (date), BrojSati (WorkingHours)
It saves data according to the time of saving.
Example: if I enter Kristijan (name) worked on 2017-11-03 4 hours, but tomorrow if I save that Kristijan worked on 2017-11-01 4 hours, it will show which data has been saved first, which in this case is 2017-11-03.
So my question is: How can I sort my data according the column Datum (date), NOT by the date of saving the data.
Also, I am not looking for query which says something like this:
SELECT *
FROM..
ORDER BY...ASC/DESC
I need some kind of "permanetly asc/desc query".
Here is the screenshot of my table
There isn't a permanent order on database table. They are unorder data set. The data isn't order by the data of creation. Is just returned in the order is storage. But that can change if db engine optimizer find a better way to read the data. Multiple Partition, Clusters, etc.
If you want the data return in a specific order YOU MUST include ORDER BY
I have a table that I created with a unique key and each other column representing one day of December 2014 (eg named D20141226 for data from 26/12/2014). So the table consists of 32 columns (key + 31 days). These daily columns are indicating that a customer had a transaction on that specific day or no transaction is indicated by a 0.
Now I want to execute the same query on a daily basis, producing a list of unique keys that had a transaction on that specific day. I used this easy script:
CREATE TABLE C01012015 AS
SELECT DISTINCT CALLING_ISDN AS A_PARTY
FROM CDRICC_012015
WHERE CALL_STA_TIME ::date = '2015-01-01'
Now my question is, how can I add the content of the new daily table to the existing table with the 31 days, making it effectively a table with 32 days of data (and then continue to do so on a daily basis to store up to 360 days of data)?
Please note that new customer are doing transactions every day hence there will unique keys in the daily table that aren't in the big table holding all the previous days.
It would be ideal if those new rows would automatically get a 0 instead of a NULL but I can work around it if it gets a NULL value (not sure how to make sure it gets a 0 instead).
I thought that a FULL OUTER JOIN would be the solution but that would mean that I have to list all variables in the select statement, which becomes quite large as I add one more column each day. Is there a more elegant way to do this?
Or is SQL just not suited to this and a programming language like eg R would be much better at this?
If you have the option to change your schema completely, you should unpivot your table so that your columns are something like CUSTOMER_ID INTEGER, D DATE, DID_TRANSACTION BOOLEAN. There's a post on the Enzee Community website that suggests using a user-defined table function (UDTF) to do this. If you change your schema in this way, a simple insert will work just fine and there will be no need to add columns dynamically.
If you can't change your schema that much but you're still able to add columns, you could add a column for every day of the year up front with a default value of FALSE (assuming it's a boolean column representing whether the customer had a transaction or not on that day). You probably want to script this.
ALTER TABLE table_with_daily_columns MODIFY COLUMN (D20140101 BOOLEAN DEFAULT FALSE);
ALTER TABLE table_with_daily_columns MODIFY COLUMN (D20140102 BOOLEAN DEFAULT FALSE);
-- etc
ALTER TABLE table_with_daily_columns ADD COLUMN (D20150101 BOOLEAN DEFAULT FALSE);
GROOM TABLE table_with_daily_columns;
When you alter a table like this, Netezza creates a new table and an internal view that does a UNION of the new table and the old. You need to GROOM the table to merge the tables back into a single one for improved performance.
If you really must keep one column per day, then you'll have to use the method you described to pivot the data from your daily transaction table. Set the default value for each of your columns to 0 or FALSE as described above, then:
INSERT INTO table_with_daily_columns
SELECT
cust_id,
TRUE as D20150101
FROM C01012015;
Hi I am creating an attendance system where i want that for a particular id given by user, attendance of that id get saved in the MS access sheet.
What i am wanting is that column name should be i date format pre created by me nad as user enters the id the program retrieves date from system and based on that date (taking it to be dynamic column name) writes an p on them.
I am using SQL query to update i.e using update statement
now i am unable to find correct syntax for using dynamic column name..
So can any one help me??
If I understand what you are saying... I would change the schema you are envisioning to have a table with "ID" and Date Columns. This prevents you from having a fixed and huge number number of date columns.
You insert a new row in this table for an ID and date combination.
I am also assuming you want a report that pivots the data where you have ID in Column position 0 and dates as column headers? Since you are using Access you can create a Cross Tab Query
(pivot) to accomplish this.