I apologize if this is asked already but I search to no avail.
When writing a Splunk query that will eventually be used for summary indexing using sitimechart, I have this query:
index=app sourcetype=<removed> host=<removed> earliest=-10d
| eval Success_Count=if(scs=="True",1,0)
| eval Failure_Count=if(scs=="False",1,0)
| timechart span=1d sum(Success_Count) as SuccessCount sum(Failure_Count) as FailureCount count as TotalCount by host
Results are as expected. However, some data was accidentally indexed twice, so I need to remove duplicates. If I'm doing a regular search, I just use | dedup _raw to remove the identical events. However, if I run the following query, I get zero results returned (no matter where I put | dedup _raw):
index=app sourcetype=<removed> host=<removed> earliest=-10d
| dedup _raw
| eval Success_Count=if(scs=="True",1,0)
| eval Failure_Count=if(scs=="False",1,0)
| timechart span=1d sum(Success_Count) as SuccessCount sum(Failure_Count) as FailureCount count as TotalCount by host
What am I doing wrong? I'm using Splunk 4.3.2.
I've also posted this identical question to the Splunk>Answers page. I will remove if that is against terms.
Related
I have multiple log messages each containing a list of JobIds -
IE -
1. `{"JobIds":["661ce07c-b5f3-4b37-8b4c-a0b76d890039","db7a18ae-ea59-4987-87d5-c80adefa4475"]}`
2. `{"JobIds":["661ce07c-b5f3-4b37-8b4c-a0b76d890040","db7a18ae-ea59-4987-87d5-c80adefa4489"]}`
3. `{"JobIds":["661ce07c-b5f3-4b37-8b4c-a0b76d890070"]}`
I have a rex to get those jobIds. Next I want to count the number of jobIds
My query looks like this -
| rex field=message "\"(?<job_ids>(?:\w+-\w+-\w+-\w+-\w+)+),?\""
| stats count(job_ids)
But this will only give me a count of 3 when I am looking for 5. How can I get a count of all jobIds? I am not sure if this is a splunk limitation or I am missing something in my regex.
Here is my regex - https://regex101.com/r/vqlq5j/1
Also with max-match=0 but with mvcount() instead of mvexpand():
| makeresults count=3 | streamstats count
| eval message=case(count=1, "{\"JobIds\":[\"a1a2a2-b23-b34-d4d4d4\", \"x1a2a2-y23-y34-z4z4z4\"]}", count=2, "{\"JobIds\":[\"a1a9a9-b93-b04-d4d4d4\", \"x1a9a9-y93-y34-z4z4z4\"]}", count=3, "{\"JobIds\":[\"a1a9a9-b93-b04-d14d14d14\"]}")
``` above is test data setup ```
``` below is the actual query ```
| rex field=message max_match=0 "\"(?<id>[\w\d]+\-[\w\d]+\-[\w\d]+\-[\w\d]+\")"
| eval cnt=mvcount(id)
| stats sum(cnt)
In Splunk, to capture multiple matches from a single event, you need to add max_match=0 to your rex, per docs.Splunk
But to get them then separated into a singlevalue field from the [potential] multivalue field job_ids that you made, you need to mvxepand or similar
So this should get you closer:
| rex field=message max_match=0 "\"(?<job_id>(?:\w+-\w+-\w+-\w+-\w+)+),?\""
| mvexpand job_id
| stats dc(job_id)
I also changed from count to dc, as it seems you're looking for a unique count of job IDs, and not just a count of how many in total you've seen
Note: if this is JSON data (and not JSON-inside-JSON) coming into Splunk, and the sourcetype is configured correctly, you shouldn't have to manually extract the multivalue field, as Splunk will do it automatically
Do you have a full set of sample data (a few entire events) you can share?
The following is my ideal final query to be used in a dashboard.
index=cdn_app httpMessage.host=taxes*
| eval _env=$env_host$
| eval _hostName=case(_env=="http:abc-h123-apps-prod","taxes.sf.com", _env=="http:abc-h123-apps-qa", "taxes-qa.sf.com")
| search httpMessage.host=_hostName
| spath output=status path=httpMessage.status
| eval status=case(like(status, "2%"),"2xx",like(status, "4%"),"4xx",like(status, "5%"),"5xx") | stats count by status
It seems _hostName is not resolving even when I hardcode and values like so.
index=cdn_app httpMessage.host=taxes*
| eval _env="taxes.sf.com"
| eval _hostName=case(_env=="http:abc-h123-apps-prod","taxes.sf.com", _env=="http:abc-h123-apps-qa", "taxes-qa.sf.com")
| search httpMessage.host=_hostName
| stats count by httpMessage.host
I'm sure its with my eval case because this works just fine.
index=cdn_app httpMessage.host=taxes*
| search httpMessage.host="taxes.sf.com"
| stats count by httpMessage.host
Open to any suggestions. Thanks!
Avoid leading underscores in field names as they can be problematic.
The case function is missing a default clause so any value of env not listed will set hostName to null. The stats command will ignore all null values of hostName.
index=cdn_app httpMessage.host=taxes*
| eval env="taxes.sf.com"
| eval hostName=case(env=="http:abc-h123-apps-prod","taxes.sf.com",
env=="http:abc-h123-apps-qa", "taxes-qa.sf.com",
1==1, "???")
| search httpMessage.host=hostName
| stats count by httpMessage.host
I have a Splunk query that works fine currently.
The Splunk query is:
index=my-index sourcetype=my-type
| timechart span=2h perc95(time) as m95
| eval test1=200
| eval test2=400
| timechart span=2h avg(m95) as mm95, avg(test1) as "TEST_1, min(test2) as "TEST_2"
If I only want to use timechart span=2h one time in the query, but the search results should be the same. How can I modify my query?
I have a log statement like 2017-06-21 12:53:48,426 INFO transaction.TransactionManager.Info:181 -{"message":{"TransactionStatus":true,"TransactioName":"removeLockedUser-1498029828160"}} .
How can i extract TransactionName and TranscationStatus and print in table form TransactionName and its count.
I tried below query but didn't get any success. It is always giving me 0.
sourcetype=10.240.204.69 "TransactionStatus" | rex field=_raw ".TransactionStatus (?.)" |stats count((status=true)) as success_count
Solved it with this :
| makeresults
| eval _raw="2017-06-21 12:53:48,426 INFO transaction.TransactionManager.Info:181 -{\"message\":{\"TransactionStatus\":true,\"TransactioName\":\"removeLockedUser-1498029828160\"}}"
| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"
| rex "{\"TransactionStatus\":(?[^,]),\"TransactioName\":\"(?[^\"])\""
| chart count OVER TransactioName BY TransactionStatus
I'm running a splunk query and trying to generate my first timechart in a few years on a different splunk instance.
I made a query. And I can see that there is data. 2741 events
If I add to the query
| stats count
Then I see the 2741
But when I try to do a count
| timechart per_hour(_cd) as "count" span=1h
I don't get anything back.
Can someone please help?
My query sucks
| timechart span=1h count as requests_per_hour