I am trying to use a custom parameter within DataStudio. The data is hosted in BigQuery.
SELECT
timestamp_trunc(o.created_at, #groupby) AS dateMain,
count(o.id) AS total_orders
FROM `x.default.orders` o
group by 1
When I try this, it returns an error saying that "A valid date part name is required at [2:35]"
I basically need to group the dates using a parameter (e.g. day, week, month).
I have also included a screenshot of how I have created the parameter in Google DataStudio. There is a default value set which is "day".
A workaround that might do the trick here is to use a rollup in the group by with the different levels of aggregation of the date, since I am not sure you can pass a DS parameter to work like that.
See the following example for clarity:
with default_orders as (
select timestamp'2021-01-01' as created_at, 1 as id
union all
select timestamp'2021-01-01', 2
union all
select timestamp'2021-01-02', 3
union all
select timestamp'2021-01-03', 4
union all
select timestamp'2021-01-03', 5
union all
select timestamp'2021-01-04', 6
),
final as (
select
count(id) as count_orders,
timestamp_trunc(created_at, day) as days,
timestamp_trunc(created_at, week) as weeks,
timestamp_trunc(created_at, month) as months
from
default_orders
group by
rollup(days, weeks, months)
)
select * from final
The output, then, would be similar to the following:
count | days | weeks | months
------+------------+----------+----------
6 | null | null | null <- this, represents the overall (counted 6 ids)
2 | 2021-01-01| null | null <- this, the 1st rollup level (day)
2 | 2021-01-01|2020-12-27| null <- this, the 1st and 2nd (day, week)
2 | 2021-01-01|2020-12-27|2021-01-01 <- this, all of them
And so on.
At the moment of visualizing this on data studio, you have two options: setting the metric as Avg instead of Sum, because as you can see there's kind of a duplication at each stage of the day column; or doing another step in the query and get rid of nulls, like this:
select
*
from
final
where
days is not null and
weeks is not null and
months is not null
I'm trying to create a calendar table for 20 years ranging from 2000 - 2020. The aim is to have one row per day along with some other columns that will use logic based on the calendar date generated. An example of this would be having One column as the calendar date (2000-01-01) and the year column reading the year from the values within the calendar date column (2000).
The code for the table is below:
CREATE TABLE TEST.CALENDAR(
CALENDAR_DATE DATE NOT NULL,
CALENDAR_YEAR INTEGER NOT NULL,
CALENDAR_MONTH_NUMBER INTEGER NOT NULL,
CALENDAR_MONTH_NAME VARCHAR(100),
CALENDAR_DAY_OF_MONTH INTEGER NOT NULL,
CALENDAR_DAY_OF_WEEK INTEGER NOT NULL,
CALENDAR_DAY_NAME VARCHAR(100),
CALENDAR_YEAR_MONTH INTEGER NOT NULL);
At the moment, I have a bunch of insert statements that manually insert rows for this table over 20 years. I'm looking to make an insert statement with variables instead and this insert statement would insert data in daily increments until the start date variable is not less than the end date variable.
Currently, I cannot get this to work at all let alone include any logic for any other columns.
Code for the variable insert statement:
declare startdate DATE, enddate DATEset startdate = '2000-01-01'
set enddate = DATEADD(yy,20,startdate)
while startdate < enddate
begin insert into TEST.CALENDAR (CALENDAR_DATE) select startdate
set startdate = DATEADD(dd,1,startdate) end
Would anyone have any ideas of how I can get this to work?
You can do this with a DB2 recursive query and date functions:
Consider:
with cte (
calendar_date,
calendar_year,
calendar_month_number,
calendar_month_name,
calendar_day_of_month,
calendar_day_of_week,
calendar_day_name
) as (
select
calendar_date,
year(calendar_date),
month(calendar_date),
monthname(calendar_date),
dayofmonth(calendar_date),
dayofweek(calendar_date),
dayname(calendar_date)
from (values(date('2000-01-01'))) as t(calendar_date)
union all
select
calendar_date + 1,
year(calendar_date + 1),
month(calendar_date + 1),
monthname(calendar_date + 1),
dayofmonth(calendar_date + 1),
dayofweek(calendar_date + 1),
dayname(calendar_date + 1)
from cte where calendar_date < date('2021-01-01')
)
select * from cte
Note: it is unclear to me what column CALENDAR_YEAR_MONTH means, so I left it apart.
Demo on DB Fiddle for the first 10 days:
CALENDAR_DATE | CALENDAR_YEAR | CALENDAR_MONTH_NUMBER | CALENDAR_MONTH_NAME | CALENDAR_DAY_OF_MONTH | CALENDAR_DAY_OF_WEEK | CALENDAR_DAY_NAME
------------: | ------------: | --------------------: | ------------------: | --------------------: | -------------------: | ----------------:
2000-01-01 | 2000 | 1 | January | 1 | 7 | Saturday
2000-01-02 | 2000 | 1 | January | 2 | 1 | Sunday
2000-01-03 | 2000 | 1 | January | 3 | 2 | Monday
2000-01-04 | 2000 | 1 | January | 4 | 3 | Tuesday
2000-01-05 | 2000 | 1 | January | 5 | 4 | Wednesday
2000-01-06 | 2000 | 1 | January | 6 | 5 | Thursday
2000-01-07 | 2000 | 1 | January | 7 | 6 | Friday
2000-01-08 | 2000 | 1 | January | 8 | 7 | Saturday
2000-01-09 | 2000 | 1 | January | 9 | 1 | Sunday
2000-01-10 | 2000 | 1 | January | 10 | 2 | Monday
Problem • Relational Knowledge
Currently, I cannot get this to work at all let alone include any logic for any other columns.
Well, there is a reason for that:
Since the Relational Model is founded on First Order Predicate Calculus (aka First Order Logic)
there is nothing in the universe that cannot be defined in terms of the RM, and stored in a Relational database (ie. one that complies with the RM), such that it can be retrieved easily and without limitation (including complex queries and DW).
Since SQL is the data sub-language for the RM, and it is Set-oriented
there is nothing, no code requirement, that cannot be implemented in Set-oriented SQL.
DB2 is a Relational database engine, with genuine Set-oriented processing, using SQL.
It appears that you are not aware of that. Instead, you are attempting to:
define low-level data structures
that you think you need for your programming,
rather than ones that are required within the RM and SQL,
that define the data, as data, and nothing but data.
write code that you need to manipulate those data structures, which is:
(a) procedural (one row at a time; WHILE loops; CURSORS; similar abominations), instead of Set-oriented, and
(b) thus the code is consequently complex, if not impossible.
Not to mention, slow as maple syrup in winter
Rather than using the available blindingly fast, Set-oriented code, which will be simple and straight-forward.
The problem may be a bit tricky, but the tables and the code required are not.
Problem • Platform Knowledge
An example of this would be having One column as the calendar date (2000-01-01) and the year column reading the year from the values within the calendar date column (2000)
That breaks two principles, and results in massive duplication within each row:
Use the correct datatype for the datum, as you have with CALENDAR_DATE. Only.
It is a DATE datatype
Using the built-in DATE datatype and DATE_PART(), DATEADD() functions means that DB2 controls the year; month; day; etc values
and all DATE errors such as 29-Feb-2019 and 31-Sep-2019 are prevented
as opposed to your code, which may have one or more errors.
a column that contains any part of a date must be stored as a DATE datatype (any part of a time as TIME; date and time as DATETIME; etc).
It breaks Codd's First Normal Form (as distinct from the ever-changing insanity purveyed as "1NF" by the pretenders)
.
Each domain [column, datum] must be Atomic wrt the platform
.
DB2 handles DATE and DATE_PART() perfectly, so there is no excuse.
All the following columns are redundant, duplicates of CALENDAR_DATE *in the same row`:
CALENDAR_YEAR
CALENDAR_MONTH_NUMBER
CALENDAR_MONTH_NAME
CALENDAR_DAY_OF_MONTH
CALENDAR_DAY_OF_WEEK
CALENDAR_DAY_NAME
CALENDAR_YEAR_MONTH
Kind of like buying a car (CALENDAR_DATE), putting it drive, and then walking beside it (7 duplicated DATE parts). You need to understand the platform; SQL, and trust it a little.
Not only that, but you will have a lot of fun and no hair left, trying to populate those duplicate columns out without making mistakes.
It needs to be said, you need to know all the date Functions in DB2 reasonably well, in order to use it proficiently.
They are duplicate columns because the values can be derived easily via DATE_PART(), something like:
SELECT DATE,
Year = DATE_PART( 'YEAR', DATE ),
MonthNo = DATE_PART( 'MONTH', DATE ),
MonthName = SUBSTR( ‘January February March April May June July August SeptemberOctober November December ‘,
( DATE_PART( 'MONTH', DATE ) - 1 ) * 9 + 1, 9 ),
DayOfMonth = DATE_PART( 'DAY', DATE ),
DayOfWeek = DATE_PART( 'DOW', DATE ),
DayName = SUBSTR( ‘Sunday Monday Tuesday WednesdayThursday Friday Saturday',
( DATE_PART( 'DOW', DATE ) - 1 ) * 9 + 1, 9 ),
YearMonth = DATE_PART( 'YEAR', DATE ) * 100 + DATE_PART( 'MONTH', DATE )
FROM TEST.CALENDAR
In Sybase, I do not have to use SUBSTR() because I have the MonthName and DayName values in tables, the query is simpler still. Or else use CASE.
Do not prefix the columns in each table with the table name. In SQL, to reference a column in a particular table, in order to resolve ambiguity, use:
.
TableName.ColumnName
.
Same as prefixing the table name with an user name TEST.CALENDAR.
The full specification is as follows, with DB2 supplying the relevant defaults based on the context of the command:
.
[SERVER.][Database.][Owner.][Table.]Column
The reason for this rule is this. Columns in one table may well be related to the same column in another table, and should be named the same. That is the nature of Relational. If you break this rule, it will retard your progressive understanding of the Relational Model, and of SQL, its data sub-language.
Problem • Data Knowledge
The aim is to have one row per day along with some other columns that will use logic based on the calendar date generated.
Why on earth would you do that ?
We store Facts about the universe in a Relational database. Only.
We do not need to store non-facts, such as:
Kyle's name isNOT"Fred"
CustomerCode "IBX" doesNOTexist.
A non-fact is simply the absence of a stored Fact.
If Fred does not exist in the Person table, and you
SELECT ... FROM Person WHERE Name = "Fred"
you will obtain an empty result set.
As it should be.
You are storing the grid that you imagine, consisting of
20 years
* 365 days
* whatever Key is relevant [eg. CustomerCode, etc),
in the form of rows.
That will only keep the database chock-full of empty rows, storing non-facts such as [eg.] CustomerCode XYZ has no event on each date for the next 20 years.
What you imagine, is the result set, or the view, which you may construct in the GUI app. It is not the storage.
Store only Facts, [eg.] actual Events per Customer.
Solution
Now for the solution.
Let me assure you that I have implemented this structure, upon which fairly complex logic has been built, in quite a few databases.
- The problem is, educating the developers in order to get them to write correct SQL code.
- Your situation is that of a developer, trying to not only write non-simple code, but to define the structures upon which it depends.
- Two distinct and different sciences.
Data Model
First, a visual data model, in order to understand the data properly.
All my data models are rendered in IDEF1X, the Standard for modelling Relational databases since 1993
My IDEF1X Introduction is essential reading for beginners.
SQL DDL
Only because you appear to work at that level:
CREATE TABLE TEST.Customer (
CustomerCode CHAR(6) NOT NULL,
Name CHAR(30) NOT NULL,
CONSTRAINT PK
PRIMARY KEY ( CustomerCode )
CONSTRAINT AK1
UNIQUE ( Name )
...
);
CREATE TABLE TEST.Event (
CustomerCode CHAR(6) NOT NULL,
Date DATE NOT NULL,
Event CHAR(30) NOT NULL,
CONSTRAINT pk
PRIMARY KEY ( CustomerCode, Date )
CONSTRAINT Customer_Schedules_Event
FOREIGN KEY ( CustomerCode )
REFERENCES Customer ( CustomerCode )
...
);
INSERT a row only when a Customer reserves a Date
Non-facts are not stored.
SELECT ... WHERE CustomerCode = ... AND Date = ...
will obtain a row if the Customer has booked an Event on that Date
will obtain nothing (empty result set) if the Customer has NOT booked an Event on that Date
If you need to store recurring Events, such as a birthday for the next 20 years, use a Projection in SQL to generate the 20 INSERT commands, which is the Set-oriented method.
If you cannot code a Projection, and only then, write a WHILE loop, which is procedural, one row per execution.
Please feel free to ask questions, the more specific the better.
As you can see, this Question is really about How to set up a Calendar for Events, but I won't change the title until I am sure this answer is what you are seeking. And about modelling data for a Relational database. I will add the tag.
SQL Server 2008, rendered in html via aspx webpage.
What I want to achieve, is to get an average per day figure that makes allowance for missing days. To do this I need to count the number of active days in a table.
Example:
Date | Amount
---------------------
2014-08-16 | 234.56
2014-08-16 | 258.30
2014-08-18 | 25.84
2014-08-19 | 259.21
The sum of the lot (777.961) divided by the number of active days (3) would = 259.30
So it needs to go "count number of different dates in the returned range"
Is there a tidy way to do this?
If you just want that one row of output then this should work:
select sum(amount) / count(distinct date) as your_average
from your_table
Fiddle:
http://sqlfiddle.com/#!2/7ffd1/1/0
I don't know this will be help to you, how about using Group By, Avg, count function.
SELECT Date, AVG(Amount) AS 'AmountAverage', COUNT(*) AS 'NumberOfActiveDays'
FROM YourTable WITH(NOLOCK)
GROUP BY Date
About AVG function, see here: Link
I've googled but I cannot get the point
I've a fact table like this one
fact_order
id, id_date, amount id_supplier
1 1 100 4
2 3 200 4
where id_date is the primary key for a dimension that have
id date month
1 01/01/2011 january
2 02/01/2011 january
3
I would like to write a calculated member that give me the last date and the last amount for the same supplier.
Last date and last amount -- it's a maximum values for this supplier?
If "yes", so you can create two metrics with aggregation "max" for fields id_date and amount.
And convert max id_date to appropriate view in the following way:
CREATE MEMBER CURRENTCUBE.[Measures].[Max Date]
AS
IIF([Measures].[Max Date Key] is NULL,NULL,
STRTOMEMBER("[Date].[Calendar].[Date].&["+STR([Measures].[Max Date Key])+"]").name),
VISIBLE = 1 ;
It will works, If maximum dates in your dictionary have maximum IDs. In my opinion You should use date_id not 1,2,3..., but 20110101, 20110102, etc.
If you don't want to obtain max values - please provide more details and small example.