I'm trying to monitor performance/metrics of my application as an external system is going through a heavy data ingest. Currently, I can easily watch one endpoint using the following
index=my_index environment=prod service=myservice api/myApi1 USER=user1 earliest=07/19/2021:12:00:00 | stats count by RESPONSECODE
How can I adjust this query to include the additional endpoints I'd like to monitor? Ultimately I'd like a pie chart showing the total numbers of successes and failures across this API for the user.
Thanks all!
Edit: In the above query, api/myApi1 is the field I'm referring to. How can I include additional api/myApi# endpoints properly?
Include additional endpoints by adding them to the base query or by making the base query less specific.
index=my_index environment=prod service=myservice api/myApi1 USER IN (user1 user2 user3) earliest=07/19/2021:12:00:00
| stats count by USER, RESPONSECODE
OR
index=my_index environment=prod service=myservice api/myApi1 USER=* earliest=07/19/2021:12:00:00
| stats count by USER, RESPONSECODE
Related
We are currently using ADO (pipeline artifacts) to Splunk for Build step reviews. I am working on a project to migrate Splunk Dashboards to powerbi. I need a query to list down the Dashboards and reports from Splunk on the below criteria so we would identify which ones to be created in powerbi
List of Dashboards / reports (possibly with Author details)
frequency of usage - like how many times the Dashboard / reports was viewed in the last 30 days
I tried few queries from mysplunk but it did not give the result. Thanks for any inputs / suggestions. Thanks.
It would help to know what you've tried so far.
To get a list of dashboards, try this query:
| rest /servicesNS/-/-/data/ui/views
To see which dashboards have been viewed, search the Splunk UI access log.
index=_internal sourcetype=splunkd_ui_access
| rex "\\/data\\/ui\\/views\\/(?<dashboard>[^\?]+)"
| stats count by dashboard
The below queries worked for me ..
To list the Dashboards
| rest /servicesNS/-/-/data/ui/views | table title,app,owner,eai:data,description,updated,version
To list the reports
|rest /servicesNS/-/-/saved/searches |table title,app,owner
Please note this will list down the default dashboards from Splunk that you may want to filter out.
I am still working on the query for the usage frequency . Thanks.
I have the following | stats count by HOST, USER, COMMAND | table HOST USER COMMAND count and it gives me a list of what I expect, but I can't seem to figure out how to consolidate HOST and USER and just count how many commands there were so it's just one row.
I'm pretty sure I'm supposed to use list in some way but my results still don't seem to consolidate correctly. Any clues?
I'm trying like this:
stats list(HOST) as HOST list(USER) as USER count(COMMAND) list(count) as count by COMMAND
Try this:
| stats values(COMMAND) as COMMAND by HOST USER
I have 5 different searches I am doing in Splunk where I am getting the count of how many results from that search query.
I've had a look at this thread here:
https://answers.splunk.com/answers/757081/pie-chart-with-count-from-different-search-criteri.html
but its not quite working for me, I'm not 100% sure if its what I want.
My search queries all look something like this:
index=A variable="foo" message="Created*" | stats count
index=A variable="foo" message="Deleted*" | stats count
I ideally want to assign each query to a keyword - such as created, deleted, etc, then do a pie chart based on the counts.
The following should be sufficient.
index=A variable="foo" message="Created*" OR message="Deleted*" OR message="<repeat this for any other message types you want>" | stats count by message
If you can provide some more examples of the events you are trying to chart, there may be alternate approaches that can work for you.
This version will extract the key part of the message (Created, Deleted. etc...) into a field called mtype and you can then perform stats on that field.
index=A variable="foo" message="Created*" OR message="Deleted*" OR message="<repeat this for any other message types you want>" | rex field=message "(?<mtype>Created|Deleteted|...)" | stats count by mtype
From Splunk, I am trying to get the user, saved search name and last time a query ran ?
A single Splunk query will be nice.
I am very new to Splunk and I have tried these queries :-
index=_audit action=search info=granted search=*
| search IsNotNull(savedsearch_name) user!="splunk-system-user"
| table user savedserach_name user search _time
The above query , is always empty for savesearch_name.
Splunk's audit log leaves a bit to be desired. For better results, search the internal index.
index=_internal savedsearch_name=* NOT user="splunk-system-user"
| table user savedsearch_name _time
You won't see the search query, however. For that, use REST.
| rest /services/saved/searches | fields title search
Combine them something like this (there may be other ways)
index=_internal savedsearch_name=* NOT user="splunk-system-user"
| fields user savedsearch_name _time
| join savedsearch_name [| rest /services/saved/searches
| fields title search | rename title as savedsearch_name]
| table user savedsearch_name search _time
Note that you have a typo in your query. "savedserach_name" should be "savedsearch_name".
But I also recommend a free app that has a dedicated search tool for this purpose.
https://splunkbase.splunk.com/app/6449/
Specifically the "user activity" view within that app.
Why it's a complex problem - part of the puzzle is in the audit log's info="granted" event, another part is in the audit log's info="completed" event, even more of it is over in the introspection index. You need those three stitched together, and the auditlog is plagued with parsing problems and autokv compounds the problem by extracting all of fields from the SPL itself.
That User Activity view will do all of this for you, sidestep pretty thorny autokv problems in the audit data, and not just give you all of this per search, but also present stats and rollups by user, app, dashboard, even by sourcetypes-that-were-actually-searched
it also has a macro called "calculate pain" that will score a "pain" number for each search, and then sum up all the "pain" in the by-user, by-app, by-sourcetype rollups etc. So that admins can try and pick off the worst offenders first.
it's up on SB here and approved for both Cloud and onprem - https://splunkbase.splunk.com/app/6449/
(and there's a #sideview_ui channel for it in the community slack.)
I am looking to see how many servers are reporting into splunk over time. This is a query similar to the one I have tried:
sourcetype=defined | dedup host | timechart count by pop
What is happening is the host gets deduped before the time chart (obviously) so I'm not exactly getting the results I'm looking for.
How can I deduplicate the server list per time slice in the timechart?
Please let me know if further clarification is necessary.
Looks like the option I was looking for was the distinct_count function for charts. This is the final query that is returning results that I am looking for:
sourcetype=defined | timechart dc(host) by pop