The following is my ideal final query to be used in a dashboard.
index=cdn_app httpMessage.host=taxes*
| eval _env=$env_host$
| eval _hostName=case(_env=="http:abc-h123-apps-prod","taxes.sf.com", _env=="http:abc-h123-apps-qa", "taxes-qa.sf.com")
| search httpMessage.host=_hostName
| spath output=status path=httpMessage.status
| eval status=case(like(status, "2%"),"2xx",like(status, "4%"),"4xx",like(status, "5%"),"5xx") | stats count by status
It seems _hostName is not resolving even when I hardcode and values like so.
index=cdn_app httpMessage.host=taxes*
| eval _env="taxes.sf.com"
| eval _hostName=case(_env=="http:abc-h123-apps-prod","taxes.sf.com", _env=="http:abc-h123-apps-qa", "taxes-qa.sf.com")
| search httpMessage.host=_hostName
| stats count by httpMessage.host
I'm sure its with my eval case because this works just fine.
index=cdn_app httpMessage.host=taxes*
| search httpMessage.host="taxes.sf.com"
| stats count by httpMessage.host
Open to any suggestions. Thanks!
Avoid leading underscores in field names as they can be problematic.
The case function is missing a default clause so any value of env not listed will set hostName to null. The stats command will ignore all null values of hostName.
index=cdn_app httpMessage.host=taxes*
| eval env="taxes.sf.com"
| eval hostName=case(env=="http:abc-h123-apps-prod","taxes.sf.com",
env=="http:abc-h123-apps-qa", "taxes-qa.sf.com",
1==1, "???")
| search httpMessage.host=hostName
| stats count by httpMessage.host
Related
I have multiple log messages each containing a list of JobIds -
IE -
1. `{"JobIds":["661ce07c-b5f3-4b37-8b4c-a0b76d890039","db7a18ae-ea59-4987-87d5-c80adefa4475"]}`
2. `{"JobIds":["661ce07c-b5f3-4b37-8b4c-a0b76d890040","db7a18ae-ea59-4987-87d5-c80adefa4489"]}`
3. `{"JobIds":["661ce07c-b5f3-4b37-8b4c-a0b76d890070"]}`
I have a rex to get those jobIds. Next I want to count the number of jobIds
My query looks like this -
| rex field=message "\"(?<job_ids>(?:\w+-\w+-\w+-\w+-\w+)+),?\""
| stats count(job_ids)
But this will only give me a count of 3 when I am looking for 5. How can I get a count of all jobIds? I am not sure if this is a splunk limitation or I am missing something in my regex.
Here is my regex - https://regex101.com/r/vqlq5j/1
Also with max-match=0 but with mvcount() instead of mvexpand():
| makeresults count=3 | streamstats count
| eval message=case(count=1, "{\"JobIds\":[\"a1a2a2-b23-b34-d4d4d4\", \"x1a2a2-y23-y34-z4z4z4\"]}", count=2, "{\"JobIds\":[\"a1a9a9-b93-b04-d4d4d4\", \"x1a9a9-y93-y34-z4z4z4\"]}", count=3, "{\"JobIds\":[\"a1a9a9-b93-b04-d14d14d14\"]}")
``` above is test data setup ```
``` below is the actual query ```
| rex field=message max_match=0 "\"(?<id>[\w\d]+\-[\w\d]+\-[\w\d]+\-[\w\d]+\")"
| eval cnt=mvcount(id)
| stats sum(cnt)
In Splunk, to capture multiple matches from a single event, you need to add max_match=0 to your rex, per docs.Splunk
But to get them then separated into a singlevalue field from the [potential] multivalue field job_ids that you made, you need to mvxepand or similar
So this should get you closer:
| rex field=message max_match=0 "\"(?<job_id>(?:\w+-\w+-\w+-\w+-\w+)+),?\""
| mvexpand job_id
| stats dc(job_id)
I also changed from count to dc, as it seems you're looking for a unique count of job IDs, and not just a count of how many in total you've seen
Note: if this is JSON data (and not JSON-inside-JSON) coming into Splunk, and the sourcetype is configured correctly, you shouldn't have to manually extract the multivalue field, as Splunk will do it automatically
Do you have a full set of sample data (a few entire events) you can share?
I am new to Splunk, trying to fetch the values from json request body. I am able to fetch values one by one by using
"json_extract(json,path)"
but I have more than 10 fields so I am trying to use
"json_extract(json,path1,path2..pathN)"
which is returning the json array.
But I am not getting how to read the values from json array and check if it is null or not.
eval keyValues ="json_extract(json,"firstname","lastname","dob")"
| table keyValues
output: ["testfirstname","testlastname","1/1/1999"]
["testfirstname","testlastname",null]
[null,"testlastname",null]
Can someone please help how to loop above json array and check the value, if it is null or not(eval isnotnull(firstname))
if your events are in JSON and it is easily accessible through field names then I suggest using the Spath command like below. Just replace your condition with YOUR_REQUIRED_CONDITION.
| makeresults
| eval json="{ \"firstname\": \"firstname123\", \"lastname\": \"lastname123\", \"dob\": \"1/1/1999\"}"
| spath input=json
| where YOUR_REQUIRED_CONDITION
You can also go with your approach but here you have to extract the same 10 fields and need to compare them as per your requirements, see the below code.
| makeresults
| eval json="{ \"firstname\": \"firstname123\", \"lastname\": \"lastname123\", \"dob\": \"1/1/1999\"}"
| eval keyValues =json_extract(json,"firstname","lastname","dob")
| table keyValues
| eval keyValues = json_array_to_mv(keyValues)
| eval firstname=mvindex(keyValues,0),lastname=mvindex(keyValues,1),dob=mvindex(keyValues,2)
| where YOUR_REQUIRED_CONDITION
If you share your SAMPLE events and the exact where conditions with us then we can provide the optimum solution.
Thanks
KV
You don't need to loop through the values, you need to treat them as multivalue fields, and expand/filter appropriately
For example:
index=ndx sourcetype=srctp fieldname.subname{}.value=*
| rename fieldname.subname{}.value as subname
| mvexpand subname
| stats count by subname
| fields - count
I have a log statement like 2017-06-21 12:53:48,426 INFO transaction.TransactionManager.Info:181 -{"message":{"TransactionStatus":true,"TransactioName":"removeLockedUser-1498029828160"}} .
How can i extract TransactionName and TranscationStatus and print in table form TransactionName and its count.
I tried below query but didn't get any success. It is always giving me 0.
sourcetype=10.240.204.69 "TransactionStatus" | rex field=_raw ".TransactionStatus (?.)" |stats count((status=true)) as success_count
Solved it with this :
| makeresults
| eval _raw="2017-06-21 12:53:48,426 INFO transaction.TransactionManager.Info:181 -{\"message\":{\"TransactionStatus\":true,\"TransactioName\":\"removeLockedUser-1498029828160\"}}"
| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"
| rex "{\"TransactionStatus\":(?[^,]),\"TransactioName\":\"(?[^\"])\""
| chart count OVER TransactioName BY TransactionStatus
I apologize if this is asked already but I search to no avail.
When writing a Splunk query that will eventually be used for summary indexing using sitimechart, I have this query:
index=app sourcetype=<removed> host=<removed> earliest=-10d
| eval Success_Count=if(scs=="True",1,0)
| eval Failure_Count=if(scs=="False",1,0)
| timechart span=1d sum(Success_Count) as SuccessCount sum(Failure_Count) as FailureCount count as TotalCount by host
Results are as expected. However, some data was accidentally indexed twice, so I need to remove duplicates. If I'm doing a regular search, I just use | dedup _raw to remove the identical events. However, if I run the following query, I get zero results returned (no matter where I put | dedup _raw):
index=app sourcetype=<removed> host=<removed> earliest=-10d
| dedup _raw
| eval Success_Count=if(scs=="True",1,0)
| eval Failure_Count=if(scs=="False",1,0)
| timechart span=1d sum(Success_Count) as SuccessCount sum(Failure_Count) as FailureCount count as TotalCount by host
What am I doing wrong? I'm using Splunk 4.3.2.
I've also posted this identical question to the Splunk>Answers page. I will remove if that is against terms.
I need to use an if statement to set the dates in startDateFrom and startDateTo if not specified in the selectedStartDateFrom and selectedStartDateTo variables.
I then want to use startDateFrom and startDateTo to filter for entries with Experiment_Instance_Start_Date between startDateFrom and startDateTo.
The date comparison works fine, it's only when I add the if statements that it stops working. Any ideas what I'm doing wrong here?
| eval compare=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
| where compare>=strptime(startDateFrom,"%m/%d/%Y")
| eval compare=strptime(startDateTo,"%m/%d/%Y")
| where compare>=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
| eval startDateFrom=if("$selectedStartDateFrom$"="", "01/01/1970", "$selectedStartDateFrom$")
| eval startDateTo=if("$selectedStartDateTo$"="", "01/01/2100", "$selectedStartDateTo$")
Turns out "$selectedStartDateFrom$"="" is wrong, so I changed that to len("$selectedStartDateFrom$")>0. I also needed to move the if statements to before the date comparison for the startDateFrom and startDateTo to be available. Also start date of 01/01/1970 returns no results, so I changed it to a later date.
| eval startDateFrom=if(len("$selectedStartDateFrom$")>0, "$selectedStartDateFrom$", "01/01/2000")
| eval startDateTo=if(len("$selectedStartDateTo$")>0, "$selectedStartDateTo$", "01/01/2099")
| eval compare=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
| where compare>=strptime(startDateFrom,"%m/%d/%Y")
| eval compare=strptime(startDateTo,"%m/%d/%Y")
| where compare>=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")