Reading a field from a JSON log in Splunk using SPATH - splunk

I am trying to read an attribute from a Splunk log that looks like this
context=context{version="1.1.2", id="123", userId=""}
I need to get count by version
My Splunk query :
index="$index" "$filterString" | spath input=context output=versionId path=version | stats count by versionId
The value of version is not being read correctly. Is the spath correct here?


spath is the right command, but it only works with valid JSON strings. The given string is considered invalid by jsonlint.com.
Here is a workaround that uses rex to extract the version ID.
index="$index" "$filterString"
| rex field=context "version=\\\"(?<versionId>[^\\\"]+)"
| stats count by versionId

Related

Extract a field from nested json in a splunk query

This is the data I have:
{ "a":"1",
"b":2,
"c": { "x":"3", "y":"4",}
}
let's suppose I have tons of events in that format. What I want to do is to write a query that will only extract "x"s from all events. I don't want anything else to be returned, just the "x"s.
I've tried multiple examples and I went through pages of documentation and yet I still did not succeed with this, there must be something I'm missing. Please advise.

It would help to know what you've tried so far and how those attempts failed to meet expectations.
Have you tried the rex command?
| rex max_match=0 "\\\"x\\\":\\\"(?<x>[^\\\"]+)"
The forest of backslashes is needed to escape the embedded quotation marks through multiple parsers.

This query works for me. Note the missing comma after the "y" value. Splunk will produce unexpected results if the JSON is not valid.
| makeresults
| eval _raw="{ \"a\":\"1\",
\"b\":2,
\"c\": { \"x\":\"3\", \"y\":\"4\"}
}"
| spath output=foo path=c.x
| table foo

Querying about field with JSON type value

I've the follow log:
INFO [http-nio-80-exec-30] class:ControllerV3,
M=method, UA=ua, URI=/v3/transactions,
QS=limit=21&offset=0&sort=-createDate, V=v3, P=3, RT=50,
ET=25, ELAPSE-TIME=50,
REQ={"userId":98745569,"initialCreationDate":"2020-03-13T00:00:00","finalCreationDate":"2020-03-16T15:41:36","source":"SOURCE","statusIds":[2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,79],"accountingEntryType":"ENTRY_TYPE","considerPartialTransaction":true},
GW=false
So I don't know how to get metrics and data about the REQ JSON field. I want know which values are passed on statusIds, accountingEntryType, considerPartialTransaction and the range of date of initialCreationDate and finalCreationDate. To get metric with normal field I use something like | stats count by UA. I'm newbie with Splunk and I don't know some functions to get the results.

Your best bet is to extract the REQ field and then use spath on it to extract the details from the JSON.
To extract the REQ field, you can use the following command. Note that this will not handle nested JSON, but if your events contain that, you can use a different regular expression.
| rex field=raw "REQ=(?<REQ>[^}]+})"
Once you have the REQ field, you can use spath to extract all the fields and values from the JSON, with the following command
| spath input=REQ
The following is an example showing that the extraction and spath work appropriately.
| makeresults | eval raw="
INFO [http-nio-80-exec-30] class:ControllerV3, M=method, UA=ua, URI=/v3/transactions, QS=limit=21&offset=0&sort=-createDate, V=v3, P=3, RT=50, ET=25, ELAPSE-TIME=50,
REQ={\"userId\":98745569,\"initialCreationDate\":\"2020-03-13T00:00:00\",\"finalCreationDate\":\"2020-03-16T15:41:36\",\"source\":\"SOURCE\",\"statusIds\":[2,3,4,5,6,7,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,79],\"accountingEntryType\":\"ENTRY_TYPE\",\"considerPartialTransaction\":true},
GW=false
"
| rex field=raw "REQ=(?<REQ>[^}]+})"
| spath input=REQ

Splunk Rex: Extracting fields of a string to a value

I'm a newbie to SPlunk trying to do some dashboards and need help in extracting fields of a particular variable
Here in my case i want to extract only KB_List":"KB000119050,KB000119026,KB000119036" values to a column
Expected output:
KB_List
KB000119050,KB000119026,KB000119036
i have tried:
| rex field=_raw "\*"KB_List":(?<KB_List>\d+)\*"
highlighted the part below in the log
svc_log_ERROR","Impact":4.0,"CategoryId":"94296c474f356a0009019ffd0210c738","hasKBList":"true","lastNumOfAlerts":1,"splunkURL":false,"impactedInstances":"","highestSeverity":"Minor","Source":"hsym-plyfss01","reqEmail":"true","AlertGroup":"TIBCOP","reqPage":"","KB_List":"KB000119050,KB000119026,KB000119036","reqTicket":"true","autoTicket":true,"SupportGroup":"TESTPP","Environment":"UAT","Urgency":4.0,"AssetId":"AST000000000159689","LiveSupportGroup":"TESTPP","sentPageTo":"TESTPP"},"Notification":{"":{"requestId":"532938335"}},"":

rex field=_raw "KB_List\":\"(?<KB_List>[^\"])\""
This regular expression will look for anything that begins with KB_List":", the capture everything except a ".
In your example, you are only capturing digits (\d+), whereas the contents in the KB_List field also contain characters ("KB" and ",")

Alas:
I figured out by looking into so many articles:
| rex "KB_List\":\"(?<KB_Listed>[^\"]+)" | table KB_Listed

How to extract data from the String in splunk?

I was given a log from splunk and I want to get a particular data in the middle of the string and use it for the dashboard. For example:
msg="somestring1 somestring2 500 somestring3 ..."
How do I get the value 500?
Sorry, I am not expert in splunk. Thanks in advance

I think what you're looking for is the rex command.
Example usage:
... | rex field=message "\S+ \S+ (?<extracted_field>\d+) \S+" | stats count by extracted_field

The stats command isn't returning any results?

I have the following query:
search (...) AND ERROR
| rex field=error "^.*(?<vcbn>Value cannot be null.)$"
| stats count(vcbn) by error
but for whatever reason the stats count(vcbn) by error isn't generating any results.
Additionally, the rex field=error "^.*(?<vcbn>Value cannot be null.)$" isn't building a new field in the list on the left of the event search results.
The search itself returns 170 events.
Splunk Version: 4.3.3

looks like rex command is not able to extract at search time.
Can you provide sample _raw log event or 'error' field from the log event?
Also refer,
http://docs.splunk.com/Documentation/Splunk/6.0.1/SearchReference/Rex

So after a good bit of research, I found a solution. The first problem was I misunderstood the field parameter for the rex command. It's meant to tell the parser was field to search through. The next thing I had to do was make sure to use the line characters ^ and $. Finally, I had to add the trailing .* to the mix so that it would look through the entire _raw field.
rex "^.*(?<vcbn>Value cannot be null).*$"
| stats count(vcbn)
NOTE: the _raw field is built in.