I need to use an if statement to set the dates in startDateFrom and startDateTo if not specified in the selectedStartDateFrom and selectedStartDateTo variables.
I then want to use startDateFrom and startDateTo to filter for entries with Experiment_Instance_Start_Date between startDateFrom and startDateTo.
The date comparison works fine, it's only when I add the if statements that it stops working. Any ideas what I'm doing wrong here?
| eval compare=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
| where compare>=strptime(startDateFrom,"%m/%d/%Y")
| eval compare=strptime(startDateTo,"%m/%d/%Y")
| where compare>=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
| eval startDateFrom=if("$selectedStartDateFrom$"="", "01/01/1970", "$selectedStartDateFrom$")
| eval startDateTo=if("$selectedStartDateTo$"="", "01/01/2100", "$selectedStartDateTo$")
Turns out "$selectedStartDateFrom$"="" is wrong, so I changed that to len("$selectedStartDateFrom$")>0. I also needed to move the if statements to before the date comparison for the startDateFrom and startDateTo to be available. Also start date of 01/01/1970 returns no results, so I changed it to a later date.
| eval startDateFrom=if(len("$selectedStartDateFrom$")>0, "$selectedStartDateFrom$", "01/01/2000")
| eval startDateTo=if(len("$selectedStartDateTo$")>0, "$selectedStartDateTo$", "01/01/2099")
| eval compare=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
| where compare>=strptime(startDateFrom,"%m/%d/%Y")
| eval compare=strptime(startDateTo,"%m/%d/%Y")
| where compare>=strptime(Experiment_Instance_Start_Date,"%m/%d/%Y")
Related
I am new to Splunk, trying to fetch the values from json request body. I am able to fetch values one by one by using
"json_extract(json,path)"
but I have more than 10 fields so I am trying to use
"json_extract(json,path1,path2..pathN)"
which is returning the json array.
But I am not getting how to read the values from json array and check if it is null or not.
eval keyValues ="json_extract(json,"firstname","lastname","dob")"
| table keyValues
output: ["testfirstname","testlastname","1/1/1999"]
["testfirstname","testlastname",null]
[null,"testlastname",null]
Can someone please help how to loop above json array and check the value, if it is null or not(eval isnotnull(firstname))
if your events are in JSON and it is easily accessible through field names then I suggest using the Spath command like below. Just replace your condition with YOUR_REQUIRED_CONDITION.
| makeresults
| eval json="{ \"firstname\": \"firstname123\", \"lastname\": \"lastname123\", \"dob\": \"1/1/1999\"}"
| spath input=json
| where YOUR_REQUIRED_CONDITION
You can also go with your approach but here you have to extract the same 10 fields and need to compare them as per your requirements, see the below code.
| makeresults
| eval json="{ \"firstname\": \"firstname123\", \"lastname\": \"lastname123\", \"dob\": \"1/1/1999\"}"
| eval keyValues =json_extract(json,"firstname","lastname","dob")
| table keyValues
| eval keyValues = json_array_to_mv(keyValues)
| eval firstname=mvindex(keyValues,0),lastname=mvindex(keyValues,1),dob=mvindex(keyValues,2)
| where YOUR_REQUIRED_CONDITION
If you share your SAMPLE events and the exact where conditions with us then we can provide the optimum solution.
Thanks
KV
You don't need to loop through the values, you need to treat them as multivalue fields, and expand/filter appropriately
For example:
index=ndx sourcetype=srctp fieldname.subname{}.value=*
| rename fieldname.subname{}.value as subname
| mvexpand subname
| stats count by subname
| fields - count
The following is my ideal final query to be used in a dashboard.
index=cdn_app httpMessage.host=taxes*
| eval _env=$env_host$
| eval _hostName=case(_env=="http:abc-h123-apps-prod","taxes.sf.com", _env=="http:abc-h123-apps-qa", "taxes-qa.sf.com")
| search httpMessage.host=_hostName
| spath output=status path=httpMessage.status
| eval status=case(like(status, "2%"),"2xx",like(status, "4%"),"4xx",like(status, "5%"),"5xx") | stats count by status
It seems _hostName is not resolving even when I hardcode and values like so.
index=cdn_app httpMessage.host=taxes*
| eval _env="taxes.sf.com"
| eval _hostName=case(_env=="http:abc-h123-apps-prod","taxes.sf.com", _env=="http:abc-h123-apps-qa", "taxes-qa.sf.com")
| search httpMessage.host=_hostName
| stats count by httpMessage.host
I'm sure its with my eval case because this works just fine.
index=cdn_app httpMessage.host=taxes*
| search httpMessage.host="taxes.sf.com"
| stats count by httpMessage.host
Open to any suggestions. Thanks!
Avoid leading underscores in field names as they can be problematic.
The case function is missing a default clause so any value of env not listed will set hostName to null. The stats command will ignore all null values of hostName.
index=cdn_app httpMessage.host=taxes*
| eval env="taxes.sf.com"
| eval hostName=case(env=="http:abc-h123-apps-prod","taxes.sf.com",
env=="http:abc-h123-apps-qa", "taxes-qa.sf.com",
1==1, "???")
| search httpMessage.host=hostName
| stats count by httpMessage.host
So i have a impala table that contains data like
+----------+-----------------------------------------------------+
| id | properties |
+----------+-----------------------------------------------------+
| 1 | {"prop1.enabled": "true", "prop2.enabled": "false"} |
| 2 | {"prop1.enabled": "true", "prop2.enabled": "true"} |
+----------+-----------------------------------------------------+
Im trying to use get_json_object() in impala with a query like
select get_json_object(properties, '$.prop1.enabled') from mytable
However i get null results because in my understanding "." operator is used to parse json nodes. Is there a workaround that can be used? I tried escaping the "." using get_json_object(properties, '$.prop1\\.enabled') however this doesnt seem to work as well. Any help greatly appreciated. Thanks
My use case is analysing ticket in order to attribute a state regarding all the status of a specific ticket.
Raw data look like this :
Id
Version
Status
Event Time
0001
1
New
2021-01-07T09:14:00Z
0001
1
Completed - Action Performed
2021-01-07T09:38:00Z
Data looks like this after transaction command:
Id
Version
Status
Event Time
state
0001, 0001
1, 1
New, Completed - Action Performed
2021-01-07T09:14:00Z, 2021-01-07T09:38:00Z
Acknowlegdement, Work
I'm using transcation command in order to calculate the duration of acknnowlegdement and resolution of the ticket.
I have predefine rule to choose the correct state. This rules compare the n-1 status (New), and the current status (Completed - Action Performed) to choose the state.
Issue
Each ticket has a different number of status. We can not know in advance the max status number. I can not write a static search comparing each value of the Status field.
Expected Solution
I have a field that inform me the number of index on the status (number of status of a ticket) field.
I want to use a loop (Why not a loop for), to iterate on each index of the field Status and compare the value i-1 and i.
I can not find how to do this. Is this possible ?
Thank you
Update to reflect more details
Here's a method with streamstats that should get you towards an answer:
index=ndx sourcetype=srctp Id=* Version=* Status=* EventTime=* state=*
| eval phash=sha256(Version.Status)
| sort 0 _time
| streamstats current=f last(phash) as chash by Id state
| fillnull value="noprev"
| eval changed=if(chash!=phash OR chash="noprev","true","false")
| search NOT changed="false"
| table *
original answer
Something like the following should work to get the most-recent status:
index=ndx sourcetype=srctp Id=* Version=* Status=* EventTime=* state=*
| stats latest(Status) as Status latest(Version) as Version latest(state) state latest(EventTime) as "Event Time" by Id
edit in light of mentioning g the transaction command
Don't use transaction unless you really really really need to.
99% of the time, stats will accomplish what transaction does faster and more efficiently.
For example:
index=ndx sourcetype=srctp Id=* Version=* Status=* EventTime=* state=*
| stats earliest(Status) as eStatus latest(Status) as lStatus earliest(Version) as eVersion latest(Version) as lVersion earliest(status) as estate latest(state) lstate earliest(EventTime) as Opened latest(EventTime) as MostRecent by Id
Will yield a table you can then manipulate further with eval and such. Eg (presuming the time format is subtractable (ie still in Unix epoch format)):
| eval ticketAge=MostRecent-Opened
| eval Versions=eVersion+" - "+lVersion
| eval Statuses=eStatus+" - "+lStatus
| eval State=estate+", ",lstate
| eval Opened=strftime(Opened,"%c"), MostRecent=strftime(MostRecent,"%c")
| eval D=if(ticketAge>86400,round(ticketAge/86400),0)
| eval ticketAge=if(D>0,round(ticketAge-(D*86400)),ticketAge)
| eval H=if(ticketAge>3600,round(ticketAge/3600),0)
| eval ticketAge=if(H>0,round(ticketAge-(H*3600)),ticketAge)
| eval M=if(ticketAge>60,round(ticketAge/60),0)
| eval ticketAge=if(M>0,round(ticketAge-(M*60)),ticketAge)
| rename ticketAge as S
| eval Age=D+" days "+H+" hours"+M+" minutes"+S+" seconds"
| table Id Versions Statuses Opened MostRecent State Age
| rename MostRecent as "Most Recent"
Note: I may have gotten the conversion from raw seconds into days, hours, minutes, seconds off - but it should be close
I apologize if this is asked already but I search to no avail.
When writing a Splunk query that will eventually be used for summary indexing using sitimechart, I have this query:
index=app sourcetype=<removed> host=<removed> earliest=-10d
| eval Success_Count=if(scs=="True",1,0)
| eval Failure_Count=if(scs=="False",1,0)
| timechart span=1d sum(Success_Count) as SuccessCount sum(Failure_Count) as FailureCount count as TotalCount by host
Results are as expected. However, some data was accidentally indexed twice, so I need to remove duplicates. If I'm doing a regular search, I just use | dedup _raw to remove the identical events. However, if I run the following query, I get zero results returned (no matter where I put | dedup _raw):
index=app sourcetype=<removed> host=<removed> earliest=-10d
| dedup _raw
| eval Success_Count=if(scs=="True",1,0)
| eval Failure_Count=if(scs=="False",1,0)
| timechart span=1d sum(Success_Count) as SuccessCount sum(Failure_Count) as FailureCount count as TotalCount by host
What am I doing wrong? I'm using Splunk 4.3.2.
I've also posted this identical question to the Splunk>Answers page. I will remove if that is against terms.