The objective of the query im trying to write is to take a count of raw data from the previous month and add that to a count from a lookup table (.csv)
What I have attempted to do is…
index=*** source=***
| stats count(_raw) as monthCount
| join
[ | inputlookup Log_Count_YTD.csv]
| eval countYTD = toNumber(monthCount) + toNumber(TOTAL_COUNT_YTD)
| table countYTD
This query doesn’t return any value on a table. The TOTAL_COUNT_YTD is the only field from the inputlookup file. Let me know if there is any other information you need to help me out with this one. Thanks!
The stats command transforms the data so it has only 1 field: monthCount. The inputlookup returns only the TOTAL_COUNT_YTD field. The join command works by comparing values of common fields between the main search and the subsearch. Since there are no common fields no events are joined.
There is no need for join in this case. The appendcols command will do, assuming the CSV contains a single field in a single row.
index=*** source=***
| stats count() as monthCount
| appendcols
[ | inputlookup Log_Count_YTD.csv]
| eval countYTD = toNumber(monthCount) + toNumber(TOTAL_COUNT_YTD)
| table countYTD
FWIW, the tonumber function is unnecessary, but doesn't hurt.
Related
I am new to Splunk, trying to fetch the values from json request body. I am able to fetch values one by one by using
"json_extract(json,path)"
but I have more than 10 fields so I am trying to use
"json_extract(json,path1,path2..pathN)"
which is returning the json array.
But I am not getting how to read the values from json array and check if it is null or not.
eval keyValues ="json_extract(json,"firstname","lastname","dob")"
| table keyValues
output: ["testfirstname","testlastname","1/1/1999"]
["testfirstname","testlastname",null]
[null,"testlastname",null]
Can someone please help how to loop above json array and check the value, if it is null or not(eval isnotnull(firstname))
if your events are in JSON and it is easily accessible through field names then I suggest using the Spath command like below. Just replace your condition with YOUR_REQUIRED_CONDITION.
| makeresults
| eval json="{ \"firstname\": \"firstname123\", \"lastname\": \"lastname123\", \"dob\": \"1/1/1999\"}"
| spath input=json
| where YOUR_REQUIRED_CONDITION
You can also go with your approach but here you have to extract the same 10 fields and need to compare them as per your requirements, see the below code.
| makeresults
| eval json="{ \"firstname\": \"firstname123\", \"lastname\": \"lastname123\", \"dob\": \"1/1/1999\"}"
| eval keyValues =json_extract(json,"firstname","lastname","dob")
| table keyValues
| eval keyValues = json_array_to_mv(keyValues)
| eval firstname=mvindex(keyValues,0),lastname=mvindex(keyValues,1),dob=mvindex(keyValues,2)
| where YOUR_REQUIRED_CONDITION
If you share your SAMPLE events and the exact where conditions with us then we can provide the optimum solution.
Thanks
KV
You don't need to loop through the values, you need to treat them as multivalue fields, and expand/filter appropriately
For example:
index=ndx sourcetype=srctp fieldname.subname{}.value=*
| rename fieldname.subname{}.value as subname
| mvexpand subname
| stats count by subname
| fields - count
I am attempting to use a search lookup table csv to filter out holidays for some Splunk queries.
To do this, I created a holidays.csv in the following style:
dateof,dateafter,description
01/17/2022,01/18/2022,MLK Day 22
02/21/2022,02/22/2022,Presidents Day 22
05/30/2022,05/31/2022,Memorial Day 22
[...]
Some of the queries run the day after the holiday, which is why I created a dateof and datefater field. I am trying to pipe the condition onto the end of the existing queries.
environment=staging "message=\"This line would contain the original search query"
| eval date=strftime(_time,"%m/%d/%Y")
| search NOT [ inputlookup holidays.csv | fields dateof ]
Note that an Event from the original query will look something like this:
time=2022-08-31T12:01:39,495Z [...] message="This line would contain the original search query"
Despite giving read access to the csv, the above condition does not filter out anything, regardless of whether it is a holiday listed in the csv file or not.
I suspect something is missing. However, I have a limited knowledge of the Spunk querying language. Would anyone be able to give guidance on this? Thanks in advance!
Subsearches can be tricky. If the result of the subsearch isn't just right, the query will fail. It helps to run the subsearch by itself to confirm the string produced makes sense as part of a query. In this case, check that
| inputlookup holidays.csv | fields dateof | rename dateof as date | format
produces something that works with search NOT.
An alternative to try is to explicitly look for the date field in the lookup.
environment=staging "message=\"This line would contain the original search query"
| eval date=strftime(_time,"%m/%d/%Y")
| where NOT [ | inputlookup holidays.csv | fields dateof | rename dateof as date | format ]
Here is another way to do it without a subsearch. A null description field tells us the date was not found in the lookup file and so is not a holiday.
environment=staging "message=\"This line would contain the original search query"
| eval date=strftime(_time,"%m/%d/%Y")
| lookup holidays.csv dateof as date output description
| where isnull(description)
I want to perform a search where I need to use a static search string + input from a csv file with usernames:
Search query-
index=someindex host=host*p* "STATIC_SEARCH_STRING"
Value from users.csv where the list is like this- Please note that User/UserList is NOT a field in my Splunk:
**UserList**
User1
User2
User3
.
.
UserN
I have tried using multiple one of them being-
| inputlookup users.csv | join [search index=someindex host=host*p* "STATIC_SEARCH_STRING"] | lookup users.csv UserList OUTPUT UserList as User| stats count by User
The above one just outputs the list of users with count as '1' - which I assume it is getting from the table itself.
When I try searching events for a single user like-
index=someindex host=host*p* "User1" "STATIC_SEARCH_STRING". I get 100's of events for that user.
Can someone please help me with this?
Sorry if this is a noob question, I have been trying to learn splunk in order to reduce my workload and am stuck here.
Thanks in advance!
index=someindex host=host*p* "STATIC_SEARCH_STRING" [ | inputlookup users.csv | fields UserList | rename UserList as query]
What is happening here is that there is a sub-search, which does an inputlookup on the users.csv file. We then use fields to ensure there is only a single field (UserList) in the data. We then rename that field to query. This is a special field in sub-searches; when the sub-search returns the field query, it is expanded out into the expression (field_value_1) OR (field_value_2) OR ....
This expression is then appended to the original search string, so the final search that Splunk executes is index=someindex host=host*p* "STATIC_SEARCH_STRING" ("alice") OR ("bob") OR ("charlie")
This approach is outlined at https://docs.splunk.com/Documentation/Splunk/8.0.3/Search/Changetheformatofsubsearchresults
You can also look at the Splunk format command, https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Format if you need to alter the sub-search's expression format, for example, adding * around each returned expression.
I think you're doing the search inside out
What I think you may want is the following:
index=ndx sourcetype=srctp host=host*p* User=*
| search
[| inputlookup users.csv ]
| stats count by User
If I understand your question correctly, you want to use the values in your lookup as a filter on the data (ie, only where User is in that list)
If that is the case, the above will do just that
If you need to make the fieldnames match because the lookup table has a different name, change the subsearch to the following:
[| inputlookup users.csv
| rename lookup_field_name as User ]
I apologise if this has been asked before. I'm still not certain how to phrase my question for the title, so wasn't sure what to search for.
I have a hundred or so databases in the same instance, one for each of my customers, named for the customer, and they all have the same structure. I want to select a single result set that includes the database name along with the most recent date entry in one of the tables. I can pull the database names from sys.databases, but then for each database I want to select the most recent date from Events.Date_Logged so that my result set looks something like this:
_______________________________
| | |
|Cust_Name |Latest_Event |
|_______________|_______________|
| | |
|Customer1 |01/02/2020 |
|_______________|_______________|
| | |
|Customer2 |02/02/2020 |
|_______________|_______________|
| | |
|Customer3 |03/02/2020 |
|_______________|_______________|
I'm really struggling with the syntax though. I either get just a single row returned or every single event for each customer. I think my joins are as rusty as hell.
Any help would be appreciated.
What I suggest you do:
Declare a result variable (of type table)
Use a cursor to go over every database
Inside the cursor: do a select top 1 ... order by date desc to get the most recent record. Save this result in the result variable.
After the cursor print the result variable.
That should do the trick.
I want to make time chart table like this:
Currently I using two queries
1.Get transaction column :
sourcetype="mysource" host="myhost" | timechart count span=1h
2.Get transaction_success column :
sourcetype="mysource" host="myhost" status="2" | timechart count span=1h
Then combine them manually with Excel.
How to search that data with only one query?
currently i found how to join the queries:
sourcetype="mysource" host="myhost" | timechart count as transaction count(eval(status="2")) as transaction_success span=1h
I hope append should work in your case. Query 1 append Query 2
Please go thru the following posts in splunk
https://answers.splunk.com/answers/28621/combine-2-splunk-queries.html
https://answers.splunk.com/answers/182453/how-to-combine-my-two-search-queries-using-join-or.html
https://answers.splunk.com/answers/30909/combine-two-queries-into-a-single-value.html
https://answers.splunk.com/answers/123204/how-to-combine-two-queries-into-one-without-using-eventtypes.html