I am trying to use has_any in sentinel to pass a list (comma delimited) of IPs to a query in a workbook.
The IP values will be passed into the query from a workbook parameter that the user enters.
With the below test code, if I use BadIPList variable for the has_any expression, I get the error "has_any(): failed to cast argument 2 to scalar constant"
If I use BadIPList2 it works fine, although they should be the same once I convert BadIPList to a Dynamic type.
let StartTime = "2022-08-07";
let TimeOffset = 4d;
let BadIPList = '10.1.1.100,10.1.1.102,10.1.1.110,10.1.1.120';
let BadIPlist2 = dynamic(['10.1.1.100','10.1.1.102','10.1.1.110','10.1.1.120']);
DeviceNetworkEvents
| extend BadIPList=todynamic(split(BadIPList,","))
| where TimeGenerated between (startofday(todatetime(StartTime)) .. endofday(todatetime(StartTime) + TimeOffset))
//next line errors
//| where RemoteIP has_any(BadIPList)
//next line works
| where RemoteIP has_any(BadIPlist2)
| project RemoteIP, BadIPList, BadIPlist2
| take 10
//verify variable types
| extend ipType = gettype(BadIPList), ipType2 = gettype(BadIPlist2)
| getschema
output of BadIPList2
I have checked the types of the two variables (Using gettype and getschema), and they seem to be the same any ideas about what I have done wrong?
DataTypes for variables
As the error message notes, the parameter should be a "scalar constant" (and not a row-level expression)
// Data sample generation. Not part of the solution.
let DeviceNetworkEvents = datatable(RemoteIP:string)["yada yada 10.1.1.102 yada"];
// Solution starts here
let BadIPList = '10.1.1.100,10.1.1.102,10.1.1.110,10.1.1.120';
let BadIPList_split = split(BadIPList, ",");
DeviceNetworkEvents
| where RemoteIP has_any(BadIPList_split)
RemoteIP
yada yada 10.1.1.102 yada
Fiddle
Related
For a Query in Microsoft Defender Advanced Hunting I want to use Data from an external Table (here the KQL_Test_Data.csv) but when I try to run it I get the Error message:
'where' operator: Failed to resolve table or column or scalar expression named 'IOC'
and when i highlight the whole Query as told in 'where' operator: failed to resolve scalar expression named 'timeOffsetMin' i get this error message:
No tabular expression statement found
This is the code i used:
let IOC = externaldata(column:string)
[
h#"https://raw.githubusercontent.com/Kornuptiko/TEMP/main/KQL_Test_Data.csv"
]
with(format="csv");
DeviceNetworkEvents
| where Timestamp > ago(30d)
| where RemoteIP in (IOC);
Assuming microsoft365-defender supports externaldata:
Your file is not a valid CSV, and KQL is strict about this.
As a work-around we can read the file as txt and then parse it.
let IOC = externaldata(column:string)
[
h#"https://raw.githubusercontent.com/Kornuptiko/TEMP/main/KQL_Test_Data.csv"
]
with(format="txt")
| parse column with * '"' ip '"' *
| project ip;
DeviceNetworkEvents
| where Timestamp > ago(30d)
| where RemoteIP in (IOC);
Is it possible to write a function that will get Tabular expression as input, and return string?
E.g. if I want a function that will return the name of a datetime column in a schema:
let timeCol = (T1:(*)){T1 | getschema
| ColumnType == "datetime"
tostring(any(ColumnName));
};
Sure. Here you go (note the toscalar function, that transforms a tabular result into a scalar):
let timeCol = (T1:(*)){
toscalar(T1
| getschema
| where ColumnType == "datetime"
| project ColumnName
| take 1);
};
print (MyTable | invoke timeCol())
According to the documentation, ascli should be able to take arguments when applying a record udf against a record (docs on ascli command 'udf-record-apply').
However, I have failed to get this to work. Here is a minimal example:
-- Minimal failing example
function testfunc(rec, param1, param2, param3)
local ret = map()
-- Just print what was passed into UDF
ret['debug_param1'] = param1
ret['debug_param2'] = param2
ret['debug_param3'] = param3
return ret
end
It was registered with ascli udf-put minimal-example.lua. When used with aql, the record udf works fine:
aql> execute minimal-example.testfunc(12345) on test.test where PK = '1'
+----------------------------------------------------------------+
| testfunc |
+----------------------------------------------------------------+
| {"debug_param1":12345, "debug_param2":NIL, "debug_param3":NIL} |
+----------------------------------------------------------------+
1 row in set (0.000 secs)
However, when used from cli with param1 = 12345, it ignores any parameters passed to the udf:
$ ascli udf-record-apply test test 1 minimal-example testfunc 12345
{ "debug_param1": null, "debug_param2": null, "debug_param3": null }
Are my calls to ascli incorrect or is this a bug?
You can pass in integer lists (or empty lists) into UDFs via ascli. For example:
ascli udf-record-apply test test 1 udfModule funcName [123] [234,235] [1,[2]] []
A couple of notes:
- Parameter passing appears to work only for integer lists (no strings, maps, etc.)
- There's no white space after the comma in the list
Instead of ascli, it is better to use the aql tool which is more feature rich and well maintained.
(extracted from ocaml: exposing a printf function in an object's method, so it can be answered independently)
I have the following (simplified) ocaml code, for a logger:
type log_level =
| Error
| Warn
| Info
let ord lvl =
match lvl with
| Error -> 50
| Warn -> 40
| Info -> 30
let current_level = ref (ord Warn)
let logf name lvl =
let do_log str =
if (ord lvl) >= !current_level then
print_endline str
in
Printf.ksprintf do_log
The logf function can be used with a printf format, as in:
logf "func" Warn "testing with string: %s and int: %d" "str" 42;
Is there any way to achieve the typical logging behaviour of only formatting arguments when they're actually needed? i.e something like:
let logf name lvl <args> =
if (ord lvl) >= !current_level then
Printf.printf <args>
I'm thinking it's because only the compiler knows how many arguments there will be in a format expression, and I guess there's no such thing as varargs in ocaml? So you can't ever define the full body of a printf function, you can only use currying and let the compiler magic figure it out. Is there any way to achieve what I want? Perhaps with mkprintf?
The continuation functions such as Printf.kfprintf are provided specifically to allow format wrappers such as this. It looks something like this:
open Printf
type log_level = Error | Warn | Info
let ord = function Error -> 50 | Warn -> 40 | Info -> 30
let string_of_lvl = function
| Error -> "error"
| Warn -> "warn"
| Info -> "info"
let current_level = ref (ord Warn)
let printf_with_info name lvl =
kfprintf fprintf stdout "[<%s>] <%s>: " name (string_of_lvl lvl)
let logf name lvl =
if ord lvl >= !current_level then match lvl with
| Error | Warn -> printf
| Info -> printf_with_info name lvl
else
ifprintf stdout
This seems to get me some of the way:
let logf name lvl =
if (ord lvl) >= !current_level then
Printf.fprintf stdout
else
Printf.ifprintf stdout
I'm surprised, as I thought this might mean !current_level would be evaluated too early (at module load time or something). But changing the current_level at runtime does seem to work correctly. I guess currying isn't pure like in haskell (which makes sense, given the impurity above ;)).
But it restricts me to using the fprintf family. I'd prefer to use ksprintf, because I also want a formatter, which (depending on log level) may add information like:
`[<name>] <level>: <message>`
Perhaps I could achieve this by concatenating formats (using ^^), but I don't see how.
Whatever you do, calling any Printf.*printf funciton will parse the format string at runtime. This may change in 4.02. Or you can use some syntax extension for conditional printing.
I'm using mksqlite to create and access an SQL database from matlab, and I want to get the number of rows in a table. I've tried this:
num = mksqlite('SELECT COUNT(*) FROM myTable');
, but the returned value isn't very helpful. If I put a breakpoint in my script and examine the variable, I find that it's a struct with a single field, called 'COUNT(_)', which seems to actually be an invalid name for a field, so I can't access it:
K>> class(num)
ans =
struct
K>> num
num =
COUNT(_): 0
K>> num.COUNT(_)
??? num.COUNT(_)
|
Error: The input character is not valid in MATLAB statements or expressions.
K>> num.COUNT()
??? Reference to non-existent field 'COUNT'.
K>> num.COUNT
??? Reference to non-existent field 'COUNT'.
Even the MATLAB IDE can't access it. If I try to double click the field in the variable editor, this gets spat out:
??? openvar('num.COUNT(_)', num.COUNT(_));
|
Error: The input character is not valid in MATLAB statements or expressions.
So how can I access this field?
You are correct that the problem is that mksqlite somehow manages to create an invalid field name that can't be read. The simplest solution is to add an AS clause to your SQL so that the field has a sensible name:
>> num = mksqlite('SELECT COUNT(*) AS cnt FROM myTable')
num =
cnt: 0
Then to remove the extra layer of indirection you can do:
>> num = num.cnt;
>> num
num =
0