Azure AlertRule queries sometimes changes query - kql
I have an Azure AlertRule what validates a query, where to tables are joining on a timestamp.
Is seems like Azure is changing the query, replacing the statement bin(..) with bin_at(..).
The Original query:
The query, opened after the alert triggered.
In my case this change is enough to alter the result of the query, becuase the extra added parameter (yellow arrow).
Is there any way around the issue?
EDIT:
as David pointed out in the comments -
This behaviour can be reproduced by running this kql as the query of an alart rule:
print bin(now(), 1h)
Original:
For completness, i've added the entier query here. I was not able to shorten it more that this. (sorry)
let frame_size = 1h;
let messages = datatable (timestamp: datetime )
[
datetime(2022-11-09T23:01:00Z),datetime(2022-11-09T23:02:00Z),datetime(2022-11-09T23:03:00Z),datetime(2022-11-09T23:04:00Z),
datetime(2022-11-09T22:01:00Z),datetime(2022-11-09T22:02:00Z),datetime(2022-11-09T22:03:00Z),datetime(2022-11-09T22:04:00Z),
datetime(2022-11-09T21:01:00Z),datetime(2022-11-09T21:02:00Z),datetime(2022-11-09T21:03:00Z),datetime(2022-11-09T21:04:00Z),
datetime(2022-11-09T20:01:00Z),datetime(2022-11-09T20:02:00Z),datetime(2022-11-09T20:03:00Z),datetime(2022-11-09T20:04:00Z),
datetime(2022-11-09T19:01:00Z),datetime(2022-11-09T19:02:00Z),datetime(2022-11-09T19:03:00Z),datetime(2022-11-09T19:04:00Z),
datetime(2022-11-09T18:01:00Z),datetime(2022-11-09T18:02:00Z),datetime(2022-11-09T18:03:00Z),datetime(2022-11-09T18:04:00Z),
datetime(2022-11-09T17:01:00Z),datetime(2022-11-09T17:02:00Z),datetime(2022-11-09T17:03:00Z),datetime(2022-11-09T17:04:00Z),
datetime(2022-11-09T16:01:00Z),datetime(2022-11-09T16:02:00Z),datetime(2022-11-09T16:03:00Z),datetime(2022-11-09T16:04:00Z),
datetime(2022-11-09T15:01:00Z),datetime(2022-11-09T15:02:00Z),datetime(2022-11-09T15:03:00Z),datetime(2022-11-09T15:04:00Z),
datetime(2022-11-09T14:01:00Z),datetime(2022-11-09T14:02:00Z),
datetime(2022-11-09T13:01:00Z),datetime(2022-11-09T13:02:00Z),
datetime(2022-11-09T12:01:00Z),datetime(2022-11-09T12:02:00Z),
datetime(2022-11-09T11:01:00Z),datetime(2022-11-09T11:02:00Z),
datetime(2022-11-09T10:01:00Z),datetime(2022-11-09T10:02:00Z),
datetime(2022-11-09T09:01:00Z),datetime(2022-11-09T09:02:00Z),
datetime(2022-11-09T08:01:00Z),datetime(2022-11-09T08:02:00Z),
datetime(2022-11-09T07:01:00Z),datetime(2022-11-09T07:02:00Z),
datetime(2022-11-09T06:01:00Z),datetime(2022-11-09T06:02:00Z),
datetime(2022-11-09T06:01:00Z),datetime(2022-11-09T06:02:00Z),
datetime(2022-11-09T05:01:00Z),datetime(2022-11-09T05:02:00Z),
datetime(2022-11-09T04:01:00Z),datetime(2022-11-09T04:02:00Z),
datetime(2022-11-09T03:01:00Z),datetime(2022-11-09T03:02:00Z),
datetime(2022-11-09T02:01:00Z),datetime(2022-11-09T02:02:00Z),
datetime(2022-11-09T01:01:00Z),datetime(2022-11-09T01:02:00Z),
datetime(2022-11-09T00:01:00Z),datetime(2022-11-09T00:02:00Z),
];
let create_time_intervals = (start_datetime: datetime, end_datetime: datetime, frame_size: timespan)
{
let hourly_distribution_as_dk_time = datatable (hour: int, expected_documents_count: int)
[0,2, 1,2, 2,2, 3,2, 4,2, 5,2, 6,2, 7,2, 8,2, 9,2, 10,2, 11,2, 12,2, 13,2, 14,2, 15,4, 16,4, 17,4, 18,4, 19,4, 20,4, 21,4, 22,4, 23,4 ]
;
let start_datetime_dk = datetime_utc_to_local(start_datetime, "Europe/Copenhagen");
let end_datetime_dk = datetime_utc_to_local(end_datetime, "Europe/Copenhagen");
// Generate all time frames for given interval excluding the first incomplete frame.
range frame_start from bin(start_datetime + frame_size, frame_size) to end_datetime step frame_size
| project bin(frame_start, frame_size)
| extend date_dk = datetime_utc_to_local(frame_start, "Europe/Copenhagen")
| extend hour = toint(datetime_part("Hour", date_dk))
| join kind=leftouter hourly_distribution_as_dk_time on hour
| project frame_start, expected_documents_count
| order by frame_start asc
};
let msg = messages
| where timestamp >= ago(1h)
|summarize cnt = count()
by frame_start = bin(timestamp,frame_size); // *** <==== this bin(..) changes when running ***
let frame_results = create_time_intervals(ago(1h), now(), frame_size)
| join kind=leftouter msg on frame_start
| extend ok = cnt;
frame_results
| summarize
sum_expected = sum(expected_documents_count),
sum_ok = sum(ok)
| extend ok = sum_expected == sum_ok
| extend ok_int = toint(ok) //for alerting purposes
Well, an obvious work-around would be to replace bin with bin_at, e.g. -
let timestamp = datetime(2000-03-04 11:22:33);
let frame_size = 1h;
print bin_at(timestamp, frame_size, datetime(2000))
print_0
2000-03-04T11:00:00Z
Fiddle
Related
Passing a variable into SQL string
I'm modifying someone else's showershell script that's used within our database to try and have it do some extra steps behind the scenes: reject code, location code, and start date.
It starts by asking for 3 values to pass as variables, but I want to add 2 additional values using a dictionary based on the location code. I don't want to share the entire batch - I know it works without this one specific part, but I'll include enough to illustrate my point.
The issue is with the variables $AREA1 and $AREA2 - these are what I'm trying to add.
The "parameters" are being injected using one way where it asks for the user to supply them but i don't want the user to supply these, they're supposed to be a lookup.
For instance, if the LOC was 5300 then AREA1 should be 101 and AREA 2 should be 111.
param(
[ImdsParameter(FullName = "REJ_CODE", Description = "Enter a 4 digit reject code to filter the data. Will not run without one.", Length = 4)]
[string] $REJ,
[ImdsParameter(FullName = "LOC", Description = "The Location code you want to run this against. Example 5300.", Length = 4)]
[string] $LOC,
[ImdsParameter(FullName = "STDTE", Description = "Start date. This is the start date from how back to run the report. Example 2021-01-01", Length = 10)]
[string] $STDTE
)
$areaStart = #{
5300 = 101 ;
5304 = 112 ;
5305 = 123 ;
5306 = 134 ;
//there’s well over 100 of these
}
$areaEnd = #{
5300 = 111 ;
5304 = 122 ;
5305 = 133 ;
5306 = 144 ;
//same
}
$AREA1 = $areaStart.Get_Item($LOC)
$AREA2 = $areaEnd.Get_Item($LOC)
$jobSql = #"
SELECT TB1.dataitem1,
TB1. Dataitem2,
TB1. Dataitem3,
TB2. Dataitem1
FROM firstTable AS TB1 INNER JOIN secondTable AS TB2
ON TB1. dataitem = TB2. dataitem
WHERE TB1.LOCATION = #LOC AND TB2.AREA BETWEEN '$($AREA1)' AND '$($AREA2)'
"#
$edbms = Get-Rdms -SQL $jobSql -Parameters #{LOC = $LOC; REJ = $REJ; STDTE = $STDTE; }
$edbms | ConvertTo-Csv -NoTypeInformation
EDIT: I need to add that the script will run, but returns no data. If I just plug the SQL in with all the values manually, it will run just fine.
Thank you for your time.
The values in the SQL were being treated as strings while the values in the dictionaries were being treated as numbers. I changed this: 5300 = 101 ; 5304 = 112 ; 5305 = 123 ; 5306 = 134 ; To this: '5300' = '101' ; '5304' = '112' ; '5305' = '123' ; '5306' = '134' ; And it provided the data expected.
gocql - using Cassandra 3 timeuuid functions
In cqlsh I can execute something like:
SELECT id FROM names WHERE name = 'dave' AND id > maxTimeuuid('2015-04-05 00:05') AND id < minTimeuuid('2019-01-01 00:05');
... this returns a list as expected.
id
--------------------------------------
632f4960-375b-11e8-90b1-02420a000203
91f03523-3761-11e8-a777-02420a000a05
32dbffc4-3762-11e8-a778-02420a000a05
9ffb1ab8-3762-11e8-a779-02420a000a05
c28a0372-3764-11e8-a77b-02420a000a05
However I am struggling to get this to work in gocql:
var found bool = false
m := map[string]interface{}{}
qString := "SELECT id FROM names WHERE name=? AND id > maxTimeuuid(?) AND id < minTimeuuid(?);"
query := Cassandra.Sessionds.Query(qString, name, toFrom.From, toFrom.To).Consistency(gocql.One)
fmt.Println("sending IsRunning query: " + query.String())
iterable := query.Iter()
for iterable.MapScan(m) {
found = true
lookupTable = HostLookup {
Id: m["id"].(gocql.UUID),
}
}
Even though the debug line looks ok,
sending IsRunning query: [query statement="SELECT id FROM names WHERE name=? AND id > maxTimeuuid(?) AND id < minTimeuuid(?);" values=[dave 2015-04-05 00:05 2019-01-01 00:05] consistency=ONE]
... I always end up with an empty iterable. I am using very similar code elsewhere successfully, the difference here are the timeuuid functions. I've tried various arrangements with literal quotes to no avail, any advice is appreciated.
When I looked at the iterable in more detail I found : "can not marshal string into timestamp" ... this led me to the answer - GoCQL : Marshal string into timestamp
Finding index using switch case statement in javascript
I'm using Pentaho(ETL) tool to achieve the output using a javascript component which accepts javascript code to achieve the desired transformation.The following table is imported into pentaho from a .csv file(source file).
For example this is my table structure
+--------+--------+--------+
| RLD | MD | INC |
+--------+--------+--------+
| 0 | 3868 | 302024 |
| 53454 | 7699 | 203719 |
| 154508 | 932 | 47694 |
| 107547 | 36168 | 83592 |
I want to use a script which would give me the max_value and its index number, such that my output would look like
Output Table
+--------+--------+--------+-----------+-----------+
| RQD | MT | IZC | max_value | max_index |
+--------+--------+--------+-----------+-----------+
| 0 | 3868 | 302024 | 302024 | 3 |
| 53454 | 7699 | 203719 | 203719 | 3 |
| 154508 | 932 | 47694 | 154508 | 1 |
| 456 | 107547| 83592 | 107547 | 2 |
To get the max value from rows I have used
var max_value = Math.max(RQD,MT,IZC);
println(max_value);
I tried to get their index using the following script
var max_index = switch (Math.max(RQD,MT,IZC))
{
case "RQD":document.write("1")
case "MT":document.write("2")
case "MT":document.write("3")
default:document.write("0")
}
How can I get the desired result in the form of javascript data structure? Any help would be much appreciated.Thanks
var list = [
{RLD:0,
MD:3868,
INC:302024
},
{RLD:53454,
MD:7699,
INC:203719
},
{RLD:154508,
MD:932,
INC:47694
},
{RLD:107547,
MD:36168,
INC:83592
},
];
list = list.map(function(item){
var keys = Object.keys(item);
item.max_value = item[keys[0]];
item.max_index = '';
for(var i = 1, l = keys.length; i < l; i++) {
var key = keys[i];
var keyValue = item[key];
if (item.max_value < keyValue){
item.max_value = keyValue;
item.max_index = key;
}
}
return item;
})
There are several issues with your code, lets solve them! Use breaks: you must use breaks in order to avoid the switch moving to cases below its match. switch cases do not return a value like functions, you cannot use a switch return to define a variable, you need to define the variable inside the switch case. Math.max does not return the name of its maximum variable, instead it returns the maximum number from its given parameters. to solve this issue, i would not use a switch case with math.max to be honest, however to answer your question: var tableArray = [RQD,MT,IZC]; var maxIndex = tableArray.indexOf(Math.max.apply(null, arr)); if(maxIndex > -1) document.write(maxIndex+1); i used +1 because you have your index in the example table starting from 1 instead of 0. the way the array is sorted should match the way the table is sorted per raw.
First of all you could not solve this problem with a switch statement.
In a javascript switch you should provide a value that is one of the followed case(s), otherwise the switch will go to the default if defined.
Your problem seems to be to find out the higher value of 3 columns and print out, the colums row by row adding a column with the max value and the index of the column where you found it.
So for example on the row:
1, RLD : 0
2, MD : 3868
3, INC : 302024
In this case the higher value is INC with the column index 3.
If you have just the variables with the number values, you could do nothing more than something like this:
function getMaxValueRow (RLD, MD, INC) {
var max_value = RLD;
var max_index = 1;
if (MD > max_value) {
max_value = MD;
max_index = 2;
}
if (INC > max_value) {
max_value = INC;
max_index = 3;
}
return [RLD, MD, INC, max_value, max_index];
}
You could return an object too like this:
retrun {
'RQD': RLD,
'MT': MD,
'IZC': INC,
'max_value': max_value,
'max_index': max_index
}
coldfusion 9, params not found when using cfscript query
I am getting this error:
Parameter 'user_id AND league_id' not found in the list of parameters specified
I am passing them in, and I can see them in a dump placed inside the function.
here's whats going in...
ac = {
league_id = 1
,user_id = 3
,score1 = 14
,score2 = 10
};
this is the dump that is throwing the error....
writedump( Game.saveGame( argumentcollection = ac ) );
Here is the Game.saveGame function
public any function saveGame( required numeric league_id, required numeric user_id, required numeric score1, required numeric score2 ) {
// writeDump( var=arguments ); // this dump shows league_id & user_id...
var sql = '';
var tmp = '';
var r = '';
var q = new query();
q.setDatasource( this.dsn );
q.addParam(
name = 'league_id'
,value = '#val( arguments.league_id )#'
,cfsqltype = 'CF_SQL_BIGINT'
);
q.addParam(
name = 'user_id'
,value = '#val( arguments.user_id )#'
,cfsqltype = 'CF_SQL_BIGINT'
);
q.addParam(
name = 'score1'
,value = '#val( arguments.score1 )#'
,cfsqltype = 'CF_SQL_SMALLINT'
);
q.addParam(
name = 'score2'
,value = '#val( arguments.score2 )#'
,cfsqltype = 'CF_SQL_SMALLINT'
);
sql = 'INSERT INTO
games
(
user_id
,league_id
,score1
,score2
)
VALUES
(
:user_id
,:league_id
,:score1
,:score2
)
';
tmp = q.execute( sql = sql );
r = tmp.getPrefix( q );
q.clearParams();
return r;
}
Here is some history of the issue -
I am writing this locally and my system is running CF 11.2 - everything works fine... However, I had to host it on a CF 9.02 server, and that is when this error showed up - I cannot recreate it on my system although, I do recall seeing this error before, but for the life of me I cant find how I solved it then....
Any help or insight is appreciated.
Other params
Windows server, MySQL 5.5, Apache 2.2
Adam Wrote: I can't spot what's wrong with your code, but you could perhaps clean things up a bit. There's no need to set each "property" on the query separately: blog.adamcameron.me/2014/01/ Following the ordinal param, instead of the named param, AND passing in the array of params in the condensed format, has solved my issue. I may play with the name attribute and see if it is that, precisely.. or I may just deal with this as a solution. Now, to change all my script queries!!!!! (I wish his article came up on google, when I was looking into the query.cfc syntax. :( Thanks a bunch Adam!
SQL date comparison in WHERE clause, TypoScript
I want to compare dates in a TypoScript select.
Here's what I have (note that I commented the were clauses) :
lib.my_val = CONTENT
lib.my_val {
select {
pidInList = 100000
max = 1
#where = effective_date < CURDATE()
#where = TIMESTAMP(effective_date) < NOW()
orderBy = effective_date DESC
}
table = tx_my_table
renderObj = COA
renderObj {
5 = TEXT
5{
field = my_field
wrap = <span>|</span>
}
[...]
}
}
Which returns lines.
I tried to add a where statement any way I could with static dates or variables... without success. My understanding of the where clause is that everything after the = is dumped as is in the SQL query. But it seems I missed something.
Basically I want the TypoScript to generate a SQL Query smilar to this :
SELECT * FROM tx_my_table WHERE effective_date < NOW() ORDER BY effective_date DESC LIMIT 1;
This should be simple. Has anyone done this in the past?
Thanks!
Your TypoScript seems to be OK.
What happens if you enter the SQL Query directly into MySQL?
Note that with your code, only one record with pid=100000 is
selected.
Have you tried this:
--
lib.my_val {
select {
pidInList = 100000
max = 1
where = UNIX_TIMESTAMP(effective_date) < UNIX_TIMESTAMP()
orderBy = UNIX_TIMESTAMP(effective_date) DESC
}
table = tx_my_table
}
TYPO3 Wiki on select