My Aim :
This below query gives me count of success, failure by b_key. I want to now get the sum of all success and failures as shown in the image below. Also I want to count the number of b_key for which the failure occured. In the example below it will be 2.
Query :
| stats dc(test_events) as events by a_key,b_key
| eval status = if(events=2,"Success","Failure")
| chart count over b_key by status
Use the addcoltotals command to create the "Sum" field.
| stats dc(test_events) as events by a_key,b_key
| eval status = if(events=2,"Success","Failure")
| chart count over b_key by status
| addcoltotals labelfield=b_key label="Sum"
Related
From splunk logs,how can I get a count of all those methods whose Time taken is > 10ms?
Splunk logs which look some thing like this :
c.s.m.c.advice.ExecutionTimeAdvice : <>
relationId = aa12 | Method Name = methodA() Time taken is = 0ms
c.s.m.c.advice.ExecutionTimeAdvice : <>
relationId = ab12 | Method Name = methodA(). Time taken is = 15ms
c.s.m.c.advice.ExecutionTimeAdvice : <>
relationId = ab12 | Method Name = methodB(). Time taken is = 1ms
This would be the general idea:
| rex field=_raw "Name = (?<methodName>\w+)\("
| rex field=_raw "s = (?<duration>\d+)\D"
| where duration > 10
| stats count by methodName
Within your search, you will need to
Create a rex field to grab the method name
Create a rex field to grab the duration in milliseconds
Use the where command to filter the results to where your new "duration" field > 10ms
Use the stats command with count by to count the current results, binning by your new "methodName" field
If this is not exactly correct for your logs, it should at least get you very close.
My use case is analysing ticket in order to attribute a state regarding all the status of a specific ticket.
Raw data look like this :
Id
Version
Status
Event Time
0001
1
New
2021-01-07T09:14:00Z
0001
1
Completed - Action Performed
2021-01-07T09:38:00Z
Data looks like this after transaction command:
Id
Version
Status
Event Time
state
0001, 0001
1, 1
New, Completed - Action Performed
2021-01-07T09:14:00Z, 2021-01-07T09:38:00Z
Acknowlegdement, Work
I'm using transcation command in order to calculate the duration of acknnowlegdement and resolution of the ticket.
I have predefine rule to choose the correct state. This rules compare the n-1 status (New), and the current status (Completed - Action Performed) to choose the state.
Issue
Each ticket has a different number of status. We can not know in advance the max status number. I can not write a static search comparing each value of the Status field.
Expected Solution
I have a field that inform me the number of index on the status (number of status of a ticket) field.
I want to use a loop (Why not a loop for), to iterate on each index of the field Status and compare the value i-1 and i.
I can not find how to do this. Is this possible ?
Thank you
Update to reflect more details
Here's a method with streamstats that should get you towards an answer:
index=ndx sourcetype=srctp Id=* Version=* Status=* EventTime=* state=*
| eval phash=sha256(Version.Status)
| sort 0 _time
| streamstats current=f last(phash) as chash by Id state
| fillnull value="noprev"
| eval changed=if(chash!=phash OR chash="noprev","true","false")
| search NOT changed="false"
| table *
original answer
Something like the following should work to get the most-recent status:
index=ndx sourcetype=srctp Id=* Version=* Status=* EventTime=* state=*
| stats latest(Status) as Status latest(Version) as Version latest(state) state latest(EventTime) as "Event Time" by Id
edit in light of mentioning g the transaction command
Don't use transaction unless you really really really need to.
99% of the time, stats will accomplish what transaction does faster and more efficiently.
For example:
index=ndx sourcetype=srctp Id=* Version=* Status=* EventTime=* state=*
| stats earliest(Status) as eStatus latest(Status) as lStatus earliest(Version) as eVersion latest(Version) as lVersion earliest(status) as estate latest(state) lstate earliest(EventTime) as Opened latest(EventTime) as MostRecent by Id
Will yield a table you can then manipulate further with eval and such. Eg (presuming the time format is subtractable (ie still in Unix epoch format)):
| eval ticketAge=MostRecent-Opened
| eval Versions=eVersion+" - "+lVersion
| eval Statuses=eStatus+" - "+lStatus
| eval State=estate+", ",lstate
| eval Opened=strftime(Opened,"%c"), MostRecent=strftime(MostRecent,"%c")
| eval D=if(ticketAge>86400,round(ticketAge/86400),0)
| eval ticketAge=if(D>0,round(ticketAge-(D*86400)),ticketAge)
| eval H=if(ticketAge>3600,round(ticketAge/3600),0)
| eval ticketAge=if(H>0,round(ticketAge-(H*3600)),ticketAge)
| eval M=if(ticketAge>60,round(ticketAge/60),0)
| eval ticketAge=if(M>0,round(ticketAge-(M*60)),ticketAge)
| rename ticketAge as S
| eval Age=D+" days "+H+" hours"+M+" minutes"+S+" seconds"
| table Id Versions Statuses Opened MostRecent State Age
| rename MostRecent as "Most Recent"
Note: I may have gotten the conversion from raw seconds into days, hours, minutes, seconds off - but it should be close
As part of my requirements, I have to calculate the duration between two different logs using Splunk query.
For example:
Log 2:
2020-04-22 13:12 ADD request received ID : 123
Log 1 :
2020-04-22 12:12 REMOVE request received ID : 122
The common String between two logs is " request received ID :" and unique strings between two logs are "ADD", "REMOVE". And the expected output duration is 1 hour.
Any help would be appreciated. Thanks
You can use the transaction command, https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction
Assuming you have the field ID extracted, you can do
index=* | transaction ID
This will automatically produce a field called duration, which is the time between the first and last event with the same ID
While transaction will work, it's very inefficient
This stats should show you what you're looking for (presuming the fields are already extracted):
(index=ndxA OR index=ndxB) ID=* ("ADD" OR "REMOVE")
| stats min(_time) as when_added max(_time) as when_removed by ID
| eval when_added=strftime(when_added,"%c"), when_removed(when_removed,"%c")
If you don't already have fields extracted, you'll need to modify thusly (remove the "\D^" in the regex if the ID value isn't at the end of the line):
(index=ndxA OR index=ndxB) ("ADD" OR "REMOVE")
| rex field=_raw "ID \s+:\s+(?<ID>\d+)\D^"
| stats min(_time) as when_added max(_time) as when_removed by ID
| eval when_added=strftime(when_added,"%c"), when_removed(when_removed,"%c")
I have results like below:
1. DateTime=2019-07-02T16:17:20,913 Thread=[], Message=[Message(userId=124, timestamp=2019-07-02T16:17:10.859Z, notificationType=CREATE, userAccount=UserAccount(firstName=S, lastName=K, emailAddress=abc#xyz.com, status=ACTIVE), originalValues=OriginalValue(emailAddress=null)) Toggle : true]
2. DateTime=2019-07-02T16:18:20,913 Thread=[], Message=[Message(userId=124, timestamp=2019-07-02T16:17:10.859Z, notificationType=CREATE, userAccount=UserAccount(firstName=S, lastName=K, emailAddress=abc#xyz.com, status=ACTIVE), originalValues=OriginalValue(emailAddress=new#xyz.com)) Toggle : true]
3. DateTime=2019-07-02T16:19:20,913 Thread=[], Message=[Message(userId=124, timestamp=2019-07-02T16:17:10.859Z, notificationType=CREATE, userAccount=UserAccount(firstName=S, lastName=K, emailAddress=abc#xyz.com, status=ACTIVE), originalValues=OriginalValue(emailAddress=new#xyz.com)) Toggle : true]
And I am trying to group results where the contents of the entire "Message" field is same and "emailAddress=null" is not contained in the Message.
So in the results above 2 and 3 should be the output.
The following query works fine for me but I need to optimize it further according to the following conditions:
Working Query: index=app sourcetype=appname host=appname* splunk_server_group=us-east-2 | fields Message | search Message= "[Message*" | regex _raw!="emailAddress=null" | stats count(Message) as count by Message | where count > 1
Conditions to optimize
Cannot rex against raw
Message key/value pair needs to be in the main search, not a sub-search
You don't have any subsearches in your current query. A subsearch is a query surrounded by square brackets.
What's wrong with rex against _raw?
Try this:
index=app sourcetype=appname host=appname* splunk_server_group=us-east-2 Message="[Message*"
| fields Message
| regex Message!="emailAddress=null"
| stats count(Message) as count by Message | where count > 1
I have a query as below. The result is always predefined as -
If the query result has 3 events and if the 3rd event has event="delivered" as value then the whole transaction needs to be returned as "COMPLETE".
If the 3rd event is present and event!="delivered" then the status becomes "PENDING"
If the 3rd event is not present at all, then the transaction is marked as ERROR
My Query -
index=myindex OR index=myindex2 uuid=98as786-ffe6-4de1-929y-080e99bc2e6r (status="202") OR (TransactionStatus="PUBLISHED") | append [search index=myindex2 (logMessage="Producer created new event") event="delivered" OR event="processed" serviceName="abc" [search index=myindex uuid=98as786-ffe6-4de1-929y-080e99bc2e6r AND status="SUCCESS" AND serviceName="abc" | top limit=1 headerId | fields + headerId | rename headerId as message_id]]
Result events -
Event1 - 202 Accepted
Event 2 - Adapter Success
Event 3 - delivered or error or processed
My high level dashboard should look like below -
Complete - 6378638
Pending - 2173
Error - 6356
The unique ID will be the UUID on which the count to be performed.
What can be the possible way we can do this - eval ? Lookup ? not sure as I am new to splunk.
Please let me know if more information is needed if I am missing something.
See if this helps. The terminology in your question is a little inconsistent so you may need to adjust the field names in this query.
index=myindex OR index=myindex2 uuid=98as786-ffe6-4de1-929y-080e99bc2e6r ((status="202") OR (TransactionStatus="PUBLISHED")) OR (index=myindex2 (logMessage="Producer created new event") event="delivered" OR event="processed" serviceName="abc") (index=myindex uuid=98as786-ffe6-4de1-929y-080e99bc2e6r AND status="SUCCESS" AND serviceName="abc" )
| stats count, latest(event) as event by headerId
| eval result=case(count=3 AND event="delivered", "COMPLETE", count=3 AND event!="delivered", "PENDING", count!=3, "ERROR", 1=1, "UNKNOWN")
| stats count by result
| table result count