I'm building a cron-as-a-service, to allow users to input their cron expression, and do some stuff periodically.
Here's a simple version of my table:
create table users (
id serial not null primary key,
-- `text` seems the most straightforward data type, any other suggestions?
cron text not null,
constraint valid_cron_expression CHECK (cron ~* '{some_regex}'),
-- snip --
-- maybe some other fields, like what to do on each cron call
)
My backend runs a SQL query every minute. How can I query all the rows whose cron field match the current timestamp (rounded to the closest minute)?
Edit: users input the cron field as a cron expression, e.g. 5 4 * * *.
Edit 2: corrected the fact that cron time resolution is minute, not second.
First of all you don't need to query every second because the cron has only a one minute resolution.
Next, comparing a cron scheduler expression to a timestamp is not a trivial task.
I'm not aware of any PostgreSQL module that would be able to parse the cron expressions.
There are two options, either you write your own function to do the comparison, or else you use a external library in the programming language you are using to do the comparison outside of the Database.
Here you will find an example implementation of such a function for Oracle that could easily be ported to PostgreSQL: SQL Query to convert cron expression to date/time format
It is incomplete because it doesn't handle complex expressions like */5 or 5,10,15 for individual fields of the cron expression but this is where I would start.
You might need to adjust some functions to mach your SQL dialect, but this seems to work, your only entry here is the value for ETL_CRON and the output is a boolean RUN_ETL.
The example returns TRUE at every 12th minute from 3 through 59 past every hour from 2 through 22 on Monday, Tuesday, Wednesday, Thursday, and Friday.:
select
'3/12 2-22 * * 1,2,3,4,5' as etl_cron
, split_part(etl_cron,' ',1) as etl_minute
, split_part(etl_cron,' ',2) as etl_hour
, split_part(etl_cron,' ',3) as etl_daymonth
, split_part(etl_cron,' ',4) as etl_month
, split_part(etl_cron,' ',5) as etl_dayweek
, convert_timezone('Europe/Amsterdam',current_timestamp) as etl_datetime
, case
when etl_minute = '*' then true
when contains(etl_minute,'-') then minute(etl_datetime) between split_part(etl_minute,'-',1) and split_part(etl_minute,'-',2)
when contains(etl_minute,'/') then mod(minute(etl_datetime),split_part(etl_minute,'/',2)) - split_part(etl_minute,'/',1) = 0
else array_contains(minute(etl_datetime)::varchar::variant, split(etl_minute,','))
end as run_minute
, case
when etl_hour = '*' then true
when contains(etl_hour,'-') then hour(etl_datetime) between split_part(etl_hour,'-',1) and split_part(etl_hour,'-',2)
when contains(etl_hour,'/') then mod(hour(etl_datetime),split_part(etl_hour,'/',2)) - split_part(etl_hour,'/',1) = 0
else array_contains(hour(etl_datetime)::varchar::variant, split(etl_hour,','))
end as run_hour
, case
when etl_daymonth = '*' then true
when contains(etl_daymonth,'-') then day(etl_datetime) between split_part(etl_daymonth,'-',1) and split_part(etl_daymonth,'-',2)
when contains(etl_daymonth,'/') then mod(day(etl_datetime),split_part(etl_daymonth,'/',2)) - split_part(etl_daymonth,'/',1) = 0
else array_contains(day(etl_datetime)::varchar::variant, split(etl_daymonth,','))
end as run_daymonth
, case
when etl_month = '*' then true
when contains(etl_month,'-') then month(etl_datetime) between split_part(etl_month,'-',1) and split_part(etl_month,'-',2)
when contains(etl_month,'/') then mod(month(etl_datetime),split_part(etl_month,'/',2)) - split_part(etl_month,'/',1) = 0
else array_contains(month(etl_datetime)::varchar::variant, split(etl_month,','))
end as run_month
, case
when etl_dayweek = '*' then true
when contains(etl_dayweek,'-') then dayofweek(etl_datetime) between split_part(etl_dayweek,'-',1) and split_part(etl_dayweek,'-',2)
when contains(etl_dayweek,'/') then mod(dayofweek(etl_datetime),split_part(etl_dayweek,'/',2)) - split_part(etl_dayweek,'/',1) = 0
else array_contains(dayofweek(etl_datetime)::varchar::variant, split(etl_dayweek,','))
end as run_dayweek
, run_minute and run_hour and run_daymonth and run_month and run_dayweek as run_etl;
This addresses the original version of the question.
Assuming that cron is a timestamp of some sort, you would use:
where cron >= date_trunc('second', now()) and
cron < date_trun('second', now()) + interval '1 second'
define start_year = 2017;
define end_year = 2020;
with af as (
select
dd.retail_year as year
, dd.retail_quarter_of_year as quarter
, dd.retail_month_of_year as month
, dd.retail_week_of_year as week
, 'BN Asset' as asset_type
, count(bn_sku) as units
, sum(bn_actual_cost) as cost
from o_diamond.diamond_asset_file af
left join o_warehouse.date_dim dd on af.receipt_date = dd.full_date
where 1=1
and dd.retail_year between &&start_year and &&end_year
The error message is compatible with an undefined substitution variable prefix. Thus you should execute
set define &
as the first command in your script / interactively. The prefix may be set to a different character in the glogin.sql of the sqlplus client. Use of substitution variables may have been switched off altogether.
This resource may be of use (though a bit dated, it should get you started).
I'm looking at doing some report development for one of our Training softwares. I finally got some queries working in FB Maestro, as I'm only familiar with SQL and Oracle.
I have the following query that works and returns results, but when trying to set up a parameter for the display name, the query runs (at least it returns no errors) however the dataset does not return any data. Has anyone worked with these before?
Here's the query:
Select CertStatus, DisplayName, Count(CertStatus) From ( With cte as (Select * From (Select COURSEVERSIONSWITHAGGREGATES.CourseTitle, COURSEVERSIONSWITHAGGREGATES.CourseNumber, "MaxTrainedCompletionDate", "Course_ID", PersonnelView.DISPLAYNAME, COURSEVERSIONSWITHAGGREGATES.RecertificationValue, COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONUNIT_ID,
CASE
WHEN COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONUNIT_ID = 3 THEN DATEADD(year, 1*COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONVALUE, MaxTrainingView."MaxTrainedCompletionDate")
WHEN COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONUNIT_ID = 2 THEN DATEADD(month, 1*COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONVALUE, MaxTrainingView."MaxTrainedCompletionDate")
WHEN COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONUNIT_ID = 1 THEN DATEADD(week, 1*COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONVALUE, MaxTrainingView."MaxTrainedCompletionDate")
WHEN COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONUNIT_ID = 0 THEN DATEADD(day, 1*COURSEVERSIONSWITHAGGREGATES.RECERTIFICATIONVALUE, MaxTrainingView."MaxTrainedCompletionDate") END
AS ExpirationDate
From MAXTRAININGVIEW
INNER JOIN PERSONNELVIEW ON (MAXTRAININGVIEW."Personnel_ID" = PERSONNELVIEW.PERSONNELID) INNER JOIN COURSEVERSIONSWITHAGGREGATES ON (MAXTRAININGVIEW."Course_ID" = COURSEVERSIONSWITHAGGREGATES.COURSEID)
WHERE Personnelview.DisplayName = 'Aaron')) Select CourseTitle, CourseNumber, "MaxTrainedCompletionDate", "Course_ID", DisplayName, RecertificationValue, Recertificationunit_ID, ExpirationDate,
CASE WHEN ExpirationDate > current_date Then 'Active' WHEN ExpirationDate < current_date Then 'Expired' END As CertStatus from cte) Group By CertStatus, DisplayName
This returns values with the static value of 'Aaron' in report builder. But trying to use a parameter, it does not throw an error in report builder, however it just does not return any data.
For example this:
WHERE Personnelview.DisplayName = '#DisplayName'))
I've got the parameter based off another data set query, and that seems to work (it gives me the option to select employees)
Here is an example of it passing 'Aaron' (with personal info removed)
Example of passing #FName Parameter:
If you want to pass the parameter in report, other type database might not recognize query like "where [field] in #parameter", so I think you could try to use filter to achieve this goal(create a filter in dataset, and create a parameter, then specify it in filter properties).
I've been digging around the internet, including Stack, for the better part of the morning trying to figure out how to create a parameter and then set its value and pass it into various locations within my where clause in my Oracle SQL. I can't seem to find exactly what I want. it's quite possible that I am not asking the question correctly, so I apologize if this is a redundant question (it seems likely that it is).
Here is the SQL I am running in Oracle SQL Developer:
SELECT t.mta_tenure_sub_type_code , t.mta_tenure_type_code , COUNT(t.tenure_number_id), SUM(t.area_in_hectares)
FROM MTA_TENURE t
WHERE
t.mta_tenure_type_code in ('M','P') AND
to_char(t.issue_date, 'YYYY') <= '&year' AND
(
to_char(t.termination_date, 'YYYY') > '&year' OR
t.termination_date IS NULL OR
t.mta_termination_type_code = 'PROT'
)
GROUP BY t.mta_tenure_sub_type_code, t.mta_tenure_type_code;
As you can see, I have '&year' located twice within my query which prompts the user to enter a value each time that variable occurs. The value is always the same year at each variable (i.e. every time the prompt pops up, the user will always enter the same year twice) Being a bit of a python programmer, I had thought that I could make this query more efficient by setting a parameter to '&year' so the user only has to enter it once and the parameter will pass the value into the where clause. Is this even possible? I really thought this would have been an easy google search, but maybe my feeble brain overlooked something. Thanks.
You can replace the &year references with &&year as #Glenn suggested; see Avoiding Unnecessary Prompts for Values in the SQL*Plus docs (most of which apply to SQL Developer too).
You could also explicitly define the substitution variable, which allows you to tailor the prompt:
accept year number prompt 'Enter year'
SELECT t.mta_tenure_sub_type_code , t.mta_tenure_type_code ,
COUNT(t.tenure_number_id), SUM(t.area_in_hectares)
FROM MTA_TENURE t
WHERE
t.mta_tenure_type_code in ('M','P') AND
to_char(t.issue_date, 'YYYY') <= '&year' AND
(
to_char(t.termination_date, 'YYYY') > '&year' OR
t.termination_date IS NULL OR
t.mta_termination_type_code = 'PROT'
)
GROUP BY t.mta_tenure_sub_type_code, t.mta_tenure_type_code;
or you could define the variable with a specific value so it doesn't prompt at all:
define year=2018
If you know the value you could also use a bind variable:
var year number
exec :year := 2018;
SELECT t.mta_tenure_sub_type_code , t.mta_tenure_type_code ,
COUNT(t.tenure_number_id), SUM(t.area_in_hectares)
FROM MTA_TENURE t
WHERE
t.mta_tenure_type_code in ('M','P') AND
to_char(t.issue_date, 'YYYY') <= :year AND
(
to_char(t.termination_date, 'YYYY') > :year OR
t.termination_date IS NULL OR
t.mta_termination_type_code = 'PROT'
)
GROUP BY t.mta_tenure_sub_type_code, t.mta_tenure_type_code;
Notice the prefix for the variable has changed from & to :.
You could combine both if you wanted to:
var year number
exec :year := &year;
... and then use the bind instead of substitution variable in the query.
In either case I'd probably either convert the to_char(...) value to a number, or use extract instead:
...
extract(year from t.issue_date) <= :year AND
(
extract(year from t.termination_date) > :year OR
...
I have been trying to get an append query to work but I keep getting an error stating that 0 rows are being appended whenever I use more than 1 parameter in the query. This is for a
The table in question has 1 PK which is a GUID [which is generating values with newid()] and one required field (Historical) which I am explictly defining in the query.
INSERT INTO dbo_sales_quotas ( salesrep_id
, [year]
, territory_id
, sales_quota
, profit_quota
, product_super_group_uid
, product_super_group_desc
, class_9
, Historical
, sales_quotas_UID )
SELECT dbo_sales_quotas.salesrep_id
, dbo_sales_quotas.Year
, dbo_sales_quotas.territory_id
, dbo_sales_quotas.sales_quota
, dbo_sales_quotas.profit_quota
, dbo_sales_quotas.product_super_group_uid
, dbo_sales_quotas.product_super_group_desc
, dbo_sales_quotas.class_9
, dbo_sales_quotas.Historical
, dbo_sales_quotas.sales_quotas_UID
FROM dbo_sales_quotas
WHERE (((dbo_sales_quotas.salesrep_id)=[cboSalesRepID])
AND ((dbo_sales_quotas.Year)=[txtYear])
AND ((dbo_sales_quotas.territory_id)=[txtTerritoryID])
AND ((dbo_sales_quotas.sales_quota)=[txtSalesQuota])
AND ((dbo_sales_quotas.profit_quota)=[txtProfitQuota])
AND ((dbo_sales_quotas.product_super_group_uid)=[cboProdSuperGroup])
AND ((dbo_sales_quotas.product_super_group_desc)=[txtProductSuperGroupDesc])
AND ((dbo_sales_quotas.class_9)=[cboClass9])
AND ((dbo_sales_quotas.Historical)='No')
AND ((dbo_sales_quotas.sales_quotas_UID)='newid()'));
Even if I assign specific values, I still get a 0 rows error except when I reduce the number of parameters to 1 (which it then works perfectly regardless of which parameter) I have verified that the parameters have the correct formats.
Can anyone tell me what I'm doing wrong?
Break out the SELECT part of your query and examine it separately. I'll suggest a simplified version which may be easier to study ...
SELECT
dsq.salesrep_id,
dsq.Year,
dsq.territory_id,
dsq.sales_quota,
dsq.profit_quota,
dsq.product_super_group_uid,
dsq.product_super_group_desc,
dsq.class_9,
dsq.Historical,
dsq.sales_quotas_UID
FROM dbo_sales_quotas AS dsq
WHERE
dsq.salesrep_id=[cboSalesRepID]
AND dsq.Year=[txtYear]
AND dsq.territory_id=[txtTerritoryID]
AND dsq.sales_quota=[txtSalesQuota]
AND dsq.profit_quota=[txtProfitQuota]
AND dsq.product_super_group_uid=[cboProdSuperGroup]
AND dsq.product_super_group_desc=[txtProductSuperGroupDesc]
AND dsq.class_9=[cboClass9]
AND dsq.Historical='No'
AND dsq.sales_quotas_UID='newid()';
I wonder about the last 2 conditions in the WHERE clause. Is the Historical field type bit instead of text? Does the string 'newid()' match sales_quotas_UID in any rows in the table?