How can we write the Splunk Query to find subField2 is present or not and if present get the counts of all subFiled2 - splunk

{
index:"myIndex",
field1: "myfield1",
field2: {"subField1":"mySubField1","subField2":145,"subField3":500},
...
..
.
}
SPL : index:"myIndex" eval result = if(field.subField2) .....
is the dot operator works in SPL ?


I am assuming your data is in JSON format. If so, you can use spath to extract fields from your structured data. Then just check if the field is present or not with isnotnull
index="myIndex" | spath | where isnotnull(field2.subField2)


Presuming your data is in JSON format, this should do it:
index=myIndex sourcetype=srctp field2{}.subField2=*
If those are multivalue fields, you'll need to do an mvexpand first

Related

In Hive, how to compare array of string with hivevar list?

In Hive, I have a column date that looks like below, array of string. I have another hivevar that look like this
set hivevar:sunny = ('2022-12-17', '2022-12-21', '2023-01-15');
|date|
|----|
|[["2022-11-14"],["2022-12-14"]]|
|[["2022-11-14","2022-11-17"],["2022-12-14","2022-12-17"]]|
|[["2022-11-21"],["2022-12-21"]]|
|[["2023-01-08"]]|
|[["2022-11-15"],["2022-12-15"],["2023-01-15"]]|
I want to check - for each row, if any of the value is part of the sunny list. So i want to get something like. I thought of using any, array && but they don't work in Hive. Can anyone help?
|result|
|----|
|false|
|true|
|true|
|false|
|true|

SELECT date, (array_contains(sunny, explode(date)) as result
FROM mytable

Elasticsearch, Elasticsearch SQL, SHOW COLUMNS or DESCRIBE - is there a posibility to filter the output

I have simple elastic SQL query like this:
GET /_sql?format=txt
{
"query" :"""
DESCRIBE "index_name"
"""
}
and it works, and the output is like this:
column | type | mapping
-----------------------------------------------------------
column_name1 | STRUCT | object
column_name1.Id | VARCHAR | text
column_name1.Id.keyword | VARCHAR | keyword
Is there a possibility to the prepare above query using filter or where, for example something like this:
GET /_sql?format=txt
{
"query":"""
DESCRIBE "index_name"
""",
"filter": {"terms": {"type.keyword": ["STRUCT"]}}
}
or
GET /_sql?format=txt
{
"query":"""
DESCRIBE "index_name"
WHERE "type" = 'STRUCT'
"""
}

That is not possible, no.
While the DESCRIBE sql command seems to return tabular data, it is not a query and it does not support WHERE clauses or can be used within a SELECT statement. That is actually not specific to Elasticsearch, but the same in RDBMs.
The same apparently is true for the Elasticsearch filter clause. This again will work with SELECT SQL statements, but with DESCRIBE or SHOW COLUMNS - while not producing an error - it simply will have no effect on the results.
In "real" SQL, you could work around this by querying information_schema.COLUMNS, but that is not an option in Elasticsearch.

How to extract the data present in {} in Splunk Search

If the data present in json format {[]} get extracted, however when data present in {} as shown below doesn't behave same. How fields and values can be extracted from data in {}
_raw data:
{"AlertEntityId": "abc#domai.com", "AlertId": "21-3-1-2-4--12", "AlertType": "System", "Comments": "New alert", "CreationTime": "2022-06-08T16:52:51", "Data": "{\"etype\":\"User\",\"eid\":\"abc#domai.com\",\"op\":\"UserSubmission\",\"tdc\":\"1\",\"suid\":\"abc#domai.com\",\"ut\":\"Regular\",\"ssic\":\"0\",\"tsd\":\"Jeff Nichols <jeff#Nichols.com>\",\"sip\":\"1.2.3.4\",\"srt\":\"1\",\"trc\":\"abc#domai.com\",\"ms\":\"Grok - AI/ML summary, case study, datasheet\",\"lon\":\"UserSubmission\"}"}
When I perform query "| table Data", I get the below result, But how to get values of "eid", "tsd".
{"etype":"User","eid":"abc#domai.com","op":"UserSubmission","tdc":"1","suid":"abc#domai.com","ut":"Regular","ssic":"0","tsd":"Jeff Nichols <jeff#Nichols.com>","sip":"1.2.3.4","srt":"1","trc":"abc#domai.com","ms":"Grok - AI/ML summary, case study, datasheet","lon":"UserSubmission"}

| spath
by default this will parse the _raw field if the data is in the field "Data"
| spath input=Data
After which eid and tsd will be in fields of the same name.
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath

Splunk extract a value from string which begins with a particular value

Could you help me extract file name in table format.
Here the below field just before file name is always constant. "Put File /test/abc/test/test/test to /test/test/test/test/test/test/test/test/test/test destFolderPath: /test/test/test/test/test/test/test/abc/def/hij"
This is an event from splunk
2021-04-08T01:03:40.155069+00:00 somedata||someotherdata||..|||Put File /test/abc/test/test/test to /test/test/test/test/test/test/test/test/test/test destFolderPath: /test/test/test/test/test/test/test/abc/def/hij/CHARGEBACK_20210407_060334_customer.csv
Result should be in table format: (font / format doesnt matter)
File Name
CHARGEBACK_20210407_060334_customer.csv

Assuming the original event/field ends with the file name, you should use this regular expression:
(?<file_name>[^\/]+)$
This will extract the text between the last "/" and the end of the event/field ("$").
You can test it here: https://regex101.com/r/J6bU3m/1
Now you can use Splunk's rex command to extract fields at search-time:
| makeresults
| eval _raw="2021-04-08T01:03:40.155069+00:00 somedata||someotherdata||..|||Put File /test/abc/test/test/test to /test/test/test/test/test/test/test/test/test/test destFolderPath: /test/test/test/test/test/test/test/abc/def/hij/CHARGEBACK_20210407_060334_customer.csv"
| fields - _time
| rex field=_raw "(?<file_name>[^\/]+)$"
Alternatively, you could also use this regular expression since you mentioned that the file path is always the same:
| rex field=_raw "abc\/def\/hij\/(?<file_name>.+)"

Splunk query to get field from JSON cell

The splunk query outputs a table where one of the column has these kind of json
the part of the query that gives this output is details.ANALYSIS
{"stepSuccess":false,"SR":false,"propertyMap":{"Url":"https://example.com","ErrCode":"401","transactionId":"7caf34342524-3d232133da","status":"API failing with error code 401"}}
I want to edit my splunk query so that instead of this json, I get only Url in this same column.
Here is my splunk query I was using
|dbxquery connection="AT" query="select service.req_id, service.out,details.ANALYSIS from servicerequest service,SERVICEREQUEST_D details where service.out like 'XYZ is%' and service.row_created > sysdate-1 and service.SERVICEREQUEST_ID = details.SERVICEREQUEST_ID and details.ANALYSIS_CLASS_NAME = 'GetProduction' " shortnames=0 maxrows=100000001
I tried using details.ANALYSIS.propertyMap.Url but it throws error.

You can probably use spathto extract the fields from details.ANALYSIS
Try the following to extract all fields
| spath field="details.ANALYSIS"
Or this just for the url field you are after
| spath field="details.ANALYSIS" path="propertyMap.Url"