pentaho: Error processing component : how to set parameter for piechart? - pentaho

I want to make a dynamic piechart (that depends on date parameter) on dashboard on CDE.
First, here's an example of my BD:
ID date category
----------------------------
1 2019-01-01 positive
2 2019-01-01 positive
3 2019-01-01 negative
4 2019-01-01 zero
1 2019-02-01 positive
2 2019-02-01 zero
3 2019-02-01 negative
On datasource panel, I added a query below:
select category, count(category) from table where date=${date} group by category;
Then on component panel, I did :
On Datasource panel, I set the parameter in the same way:
But the piechart doesn't show and the error message is : Error processing component
If I replace the parameter ${month} with a value like '2019-01-01', the pie chart does show without problem.
I followed multiple tutorials but can't resolve my problem.
Is it because of date format ?
This is how I set date parameter on datasource panel:I also tried with Date instead of String...

My experience is that treating CDE parameters as strings and then converting them to date in the SQL query works best and gives you the most control.
MySQL example:
where STR_TO_DATE(${DATE_FROM}, '%d-%m-%Y') = MyDateColumn
In my case, I needed to show "dd-mm-yyyy" format in the dashboard and internationalization is rather broken in CDE, so I went with string parameters and the above contains the MySQL date format for it. If you use a different DB, look up the date conversion function in the docs.

Related

CAST and CONVERT both failing when attempting to convert string to date

I'm dealing with a table containing records from questionnaires administered to people after completing an activity. There are several questions on the questionnaire, so each person has multiple records with the same collection date, like so.
PersonID Question Result CollectedDate
-------------------------------------------------------------
1001 First activity? Yes 10/23/2022
1001 Activity date 10/20/2022 10/23/2022
1001 Activity type Painting 10/23/2022
1002 First activity? No 10/24/2022
1002 Activity date 10/23/2022 10/24/2022
1002 Activity type Writing 10/24/2022
Since my end goal is to compare the activity date with the questionnaire collection date and see how much time elapsed between them, I've altered my query a bit so I'm focusing only on each person's question regarding the activity date. It's a super simple query:
SELECT
PersonID,
Question,
Result,
CollectedDate
FROM Questionnaire
WHERE Question LIKE '%date%'
PersonID Question Result CollectedDate
-------------------------------------------------------------
1001 Activity date 10/20/2022 10/23/2022
1002 Activity date 10/23/2022 10/24/2022
My main issue is that the Result field is varchar(50) in order to accommodate text answers, so any dates seen there are actually from free text fields in the front-end interface. I've tried using both CAST() and CONVERT() to turn it into an actual date format so the difference between the dates can be calculated. I've seen both of the following errors depending on which function I'm using or which date/time style I'm attempting to apply:
Conversion failed when converting date and/or time from character string
The conversion of a varchar data type to a datetime data type resulted in an out-of-range value
I've tried:
SELECT
PersonID,
Question,
CAST(Result as date),
CollectedDate
FROM Questionnaire
WHERE Question LIKE '%date%'
and...
SELECT
PersonID,
Question,
CONVERT(DATETIME,Result,101) as Result,
CollectedDate
FROM Questionnaire
WHERE Question LIKE '%date%'
...and have tried several different styles. Does anyone have any further suggestions? Is the date itself likely the problem, or is if the fact that the Result field contains a bunch of other stuff too, even though it's currently omitted from the query results?
UPDATE: There are some kind of wonky date formats in this Result field even when I have the other question types filtered out (I hate free text). For example, there are some formatted like 05/01/2022 and others like 5/1/2022. Some others have something like 5/19/2022 - 5/20/2022, like maybe the person couldn't remember the exact date of their activity. What's the best way to deal with all of this?
You should be able to get past the error by making sure you reject any value that can't be converted to a date. Largely, that is this:
Result = CASE
WHEN ISDATE(Result) = 1 THEN CONVERT(date, Result, 101) END
You'd think it would be enough to say WHERE Question = 'Activity Date' AND ISDATE(Result) = 1, but:
Someone still might have entered bad data on that row.
SQL Server might try to perform the CONVERT() operation before the filter.
You can identify the ones that have bad data using:
WHERE Question = 'Activity Date' AND ISDATE(Result) = 0
But until you've fixed the structure and stored dates in an independent column, fixing that data just means it's a matter of time before it happens again.
You might consider, in the meantime, just displaying what the user entered as a string, instead of trying to force it to be converted to a date. Especially since 101 might be a bad guess - what if the user is from the UK or Canada? They may have entered 05/12 and meant December 5th, not May 12th.

How to filter by last max value/date on DataStudio?

I have a BigQuery dataset updating on irregular times (can be once, twice a week, or less). Data is structured as following.
id
Column1
Column2
data_date(timestamp)
0
Datapoint0
Datapoint00
2022-01-01
1
Datapoint1
Datapoint01
2022-01-01
2
Datapoint2
Datapoint02
2022-01-03
3
Datapoint3
Datapoint03
2022-01-03
4
Datapoint4
Datapoint04
2022-02-01
5
Datapoint5
Datapoint05
2022-02-01
6
Datapoint6
Datapoint06
2022-02-15
7
Datapoint7
Datapoint07
2022-02-15
Timestamp is a string in 'YYYY-MM-DD' format.
I want to make a chart and a pivot table in Google DataStudio that automatically filters by the latest datapoints ('2022-02-15' in the example). All the solutions I tried are either sub-optimal or just don't work:
Creating a support column doesn't work because I need to mix aggregated and non-aggregated fields (data_date and the latest data_date)
Adding a filter to the charts allows me to specify only a specific day - I would need to edit the chart regularly every time the underlyind data is updated
Using a dropdown filter allows me to dynamically filter whatever date I need. However I consider it suboptimal because I can't have it automatically select the latest date. Having a date filter can make it dynamic, but since the update time is not regular it may select a date range with multiple timestamps/or none at all, so it's also a sub-optimal solution
Honestly I'm out of ideas. I stupidly thought it was possible to add a column saying data_date = (select max(data_date) from dataset, but it seems not possible since max needs to work on aggregated data.
One possible solution could be creating a view that can have the latest data point, and referencing the view from the data studio.
CREATE OR REPLACE VIEW `project_id.dataset_id.table_name` AS
SELECT *
FROM `bigquery-public-data.covid19_ecdc_eu.covid_19_geographic_distribution_worldwide`
ORDER BY date DESC # or timestamp DESC
LIMIT 1

Using multiple date parameters in SSRS

I'm trying to build a SSRS report with multiple parameters, and when I add a standard START_DATE and END_DATE parameter I get the expected results from my SQL query line:
WHERE SOME_FIELD BETWEEN (:P_START_DATE) and (:P_END_DATE)
I would like to do something like the following scenario:
Setup:
Date Field 1 & 2: WHERE NEXT_SHIP_DUE_DATE BETWEEN (:P_NEXT_SHIP_START_DATE) AND (:P_NEXT_SHIP_END_DATE)
Date Field 3 & 4: AND PRIMARY_IMAGE_DATE BETWEEN (:P_PID_START_DATE) AND (:P_PID_END_DATE)
All date parameters in SSRS report allow NULL values and are initially set to NULL.
User should be able to:
Get all records when no date field parameters are requested
Get only records based off the date fields that user selects This may be 1 or more.
Scenarios:
Example 1: User selects no date fields - returns no data
Example 2: User selects date fields 1 & 2 only - returns no data
Example 3: User selects all date fields same date range - RETURNS CORRECT DATA
Example 4: User selects all date fields different date ranges - SSRS error: date not valid for month specified
I've tried several different pieces of SQL but no luck so far. Any help would be appreciated!
You need to add some kind of null check, be careful of brackets.
WHERE (NEXT_SHIP_DUE_DATE BETWEEN (:P_NEXT_SHIP_START_DATE) AND (:P_NEXT_SHIP_END_DATE) OR ((:P_NEXT_SHIP_END_DATE) IS NULL AND (:P_NEXT_SHIP_START_DATE) IS NULL))
AND (PRIMARY_IMAGE_DATE BETWEEN (:P_PID_START_DATE) AND (:P_PID_END_DATE) OR ((:P_PID_END_DATE) IS NULL AND (:P_PID_START_DATE) IS NULL))

Using ampscript to get all records in a data extension between 2 dates

Using Marketing Cloud:
I have a data extension with a date field.
On a cloudpage, using ampscript, i need to get all the records from that data extension where the date field is between/including 2 dates.
The LookupRows function seems to be the best available but i can only directly compare values to each other, it can't use a "between" statement.
What would be the best way to achieve this ?
( Other than executing a LookupRows for each date separately )

How to convert a date to string then split it into 4 parts using Pentaho Kettle

I have a table that contains a column named time_created representing a date: 2014-02-19 23:49:59.998557. Now I need to generate a new table that consists of year, month, day and quarter columns.(For analysis purpose).
For example, 2014-02-19 23:49:59.998557 should be converted to:
year| month | day | quarter
2014 | 2 | 19 | Q1
How can I do that using Kettle? Thanks!
Or use the select values step.. this can make copies of fields then you simply specify the format mask for the date to string conversion in the meta tab. Potentially more flexible too..
Use a calculator step to generate the four fields you need (year, month,..). For each field use your timestamp as Field A and choose the appropriate calculation (e.g. Year of date A).
You will need to work around the quarter field to prepend a Q (calculation = set field to constant value A ). You can do this in the same calculator step, using the remove feature to get rid of your Q field.