Anyone here knows how can I use built-in functions(case) in a Splunk Query? All examples I found were to handle the query results (I can not put it after eval or | )
I need something like.
index=case(indexVar == "qa", "qa-all", indexVar == "prod", "prod-all") sourcetype="kube:container:rail-service"
OBS I can not just concat the indexVar + "-all"
The case function may be built-in, but that doesn't mean you can use it anywhere. It's only valid with the eval, fieldformat, and where commands.
A workaround would be to put the eval in a subsearch.
sourcetype="kube:container:rail-service" [
| makeresults
| eval index=case(indexVar == "qa", "qa-all", indexVar == "prod", "prod-all")
| fields index ]
Related
I have simple elastic SQL query like this:
GET /_sql?format=txt
{
"query" :"""
DESCRIBE "index_name"
"""
}
and it works, and the output is like this:
column | type | mapping
-----------------------------------------------------------
column_name1 | STRUCT | object
column_name1.Id | VARCHAR | text
column_name1.Id.keyword | VARCHAR | keyword
Is there a possibility to the prepare above query using filter or where, for example something like this:
GET /_sql?format=txt
{
"query":"""
DESCRIBE "index_name"
""",
"filter": {"terms": {"type.keyword": ["STRUCT"]}}
}
or
GET /_sql?format=txt
{
"query":"""
DESCRIBE "index_name"
WHERE "type" = 'STRUCT'
"""
}
That is not possible, no.
While the DESCRIBE sql command seems to return tabular data, it is not a query and it does not support WHERE clauses or can be used within a SELECT statement. That is actually not specific to Elasticsearch, but the same in RDBMs.
The same apparently is true for the Elasticsearch filter clause. This again will work with SELECT SQL statements, but with DESCRIBE or SHOW COLUMNS - while not producing an error - it simply will have no effect on the results.
In "real" SQL, you could work around this by querying information_schema.COLUMNS, but that is not an option in Elasticsearch.
I have a Splunk Dashboard. This dashboard has a Text input where the user can enter a path. After entering the input, I would like to apply some conditional logic to the path input by the user before the search is executed. Is this possible in Splunk? Is there a way for me to take the Text input (i.e. path) and do something like:
var parameter1 = "value-a";
if (path == "/endpoint-1")
parameter1 = "value-b";
else if (path == "/endpoint-2")
parameter1 = "/endpoint-3";
// Execute search with parameter1
Thank you.
Subsearches!
Eg:
index=data [
| makeresults 1
| eval path="$inputToken$"
| eval parameter1=case(
path="/endpoint-1","value-b,
path="/endpoint-2","/endpoint-3")
| fields parameter1
| format]
the subsearches are run before the main search, and alter that main search.
the main search here after the subsearch would be something like.
index=data parameter1="value-b"
Related reading to help on your sub search journey
https://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/Useasubsearch
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Format
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Return
I am trying to access a variable (in this example; sampleFromDate and sampleToDate) from a sub-query. I have defined the variables with syntax eval variableName = value and would like to access with syntax filterName=$variableName$. See the example below where I am trying to access values using earliest=$sampleFromDate$ latest=$sampleToDate$
index=*
earliest=-8d latest=-1d
| eval sampleToDate=now()
| eval sampleFromDate=relative_time(now(), "-1d")
| appendcols [
search (index=*)
earliest=$sampleFromDate$ latest=$sampleToDate$
]
This produces the error:
Invalid value "$sampleFromDate$" for time term 'earliest'
The value of sampleFromDate is in the format seconds since epoch time, e.g.
1612251236.000000
I know I can do earliest=-d latest=now() - but I don't want to do this because I want to reference the variables in several locations and output them at the end.
Why are you trying to eval those time values?
Just do:
index=* earliest=-8d latest=-1d
| <rest of search>
| appendcols [
search (index=*) earliest=-1d
| <rest of appended search>
]
There's no need to explicitly set latest unless you want something other than now()
I am a new developer in Spark Scala. I am not familiar with Regex but I want to write a regex that can extract an ID like this :
abcd_mss5884_mww020_025_b => mss5884
abv_c_e_mss478_mww171_172 => mss478
abv_c_e_mww171_172 => otherwise, return THE SAME input string
So, in our input string, I should return the first characters starting from "mss...." and stop when I find the first "_" after the "mss" of course (i should ignore the other underscores).
How can I do this please ?
Should I use a regex ? A regex in SQL or in Scala ?
Or should I just use a simple substring method ??
Simply use regexp_extract function. Something like this:
val df = Seq(("abcd_mss5884_mww020_025_b"), ("abv_c_e_mss478_mww171_172"), ("abv_c_e_mww171_172")).toDF("input")
df.withColumn("ID", regexp_extract($"input", "^(.*)(mss[^_]+)_(.*)$", 2))
.withColumn("ID", when($"ID" =!= "", $"ID").otherwise($"input"))
.show()
+-------------------------+------------------+
|input |ID |
+-------------------------+------------------+
|abcd_mss5884_mww020_025_b|mss5884 |
|abv_c_e_mss478_mww171_172|mss478 |
|abv_c_e_mww171_172 |abv_c_e_mww171_172|
+-------------------------+------------------+
I have a JSON object that's written in a weird way.
> {"custom": [ { "name": "addressIdNum", "valueNum": 12345678}, {
> "name": "cancelledDateAt", "valueAt": "2017-02-30T01:43:04.000Z" }] }
Not sure how to parse something like this. The keys are addressIdNum and cancelledDateAt and the values are 12345678 and 2017-02-30T01:43:04.000Z respectively.
How do I parse this using Snowflake SQL?
Thanks for all your help!
Best,
Preet Rajdeo
If your input is ALWAYS in this form (two elements in an array, with the same fields in the same element), you can combine PARSE_JSON function and the path access.
Just try this:
with input as (
select parse_json(
'{"custom": [ { "name": "addressIdNum", "valueNum": 12345678}, {"name": "cancelledDateAt", "valueAt": "2017-02-30T01:43:04.000Z" }] }')
as json)
select json:custom[0].valueNum::integer, json:custom[1].valueAt::timestamp from input;
----------------------------------+-----------------------------------+
JSON:CUSTOM[0].VALUENUM::INTEGER | JSON:CUSTOM[1].VALUEAT::TIMESTAMP |
----------------------------------+-----------------------------------+
12345678 | 2017-03-01 17:43:04 |
----------------------------------+-----------------------------------+
However, if the structure of your data might be different (e.g. elements in the array might be in a different order), it's probably best to write a JavaScript UDF in Snowflake to convert such messy data into something easier.