Small question for a Splunk query please.
May I ask if there is a way to search for “a first log that got printed, but the second was not printed” statement please? Background, I have a very simple piece of Java logic as follow:
LOGGER.info("START/END compute something that might result in a bad exception for id START " + id);
invote_method_which_can_fail(id);
LOGGER.info("START/END compute something that might result in a bad exception for id END " + id);
Which results in something like (snippet from a million):
START/END compute something that might result in a bad exception for id START 12345
START/END compute something that might result in a bad exception for id END 12345
START/END compute something that might result in a bad exception for id START 88888
START/END compute something that might result in a bad exception for id START 98765
START/END compute something that might result in a bad exception for id END 98765
As you can see, the id 88888 in my example got the start statement printed, but not the end statement, because something bad happened in the java code. (the question is not about how to make the java code reliable)
May I ask if there is a Splunk query which can find me those id please?
What I tried: So far, I am downloading the search result containing all the starts. Then, downloading the search results with all the ends. Once having both, I am running another offline script in order to find all the id from the first search result that are not there from the second...
I do not think this is "the smart thing to do" and was wondering if there is a smarter query which can give me the expected result directly in Splunk.
Thank you
You can try something along these lines (with rex and stats):
index=... "START/END compute something that might result in a bad exception for id"
| rex "(?<operation>(START|END))\s+(?<id>\d+)"
| stats count(eval(operation="START")) as start count(eval(operation="END")) as end by id
| where NOT start=end
I have not tested this SPL code
Related
I don't think this is possible, but wanted to ask anyway -- is there a way in Cloudwatch Insights where I can find the count of how often a log with a dynamic value is emitted with the same value from distinct logs? The use case I have is we want to compare log statements from two different code paths, so we attach the same requestID to both log statements. So to illustrate what might happen, two logs may get emitted
Log 1:
{
message: "SPECIAL_LOG_EMITTED Operation1_Emitted"
requestID: "123456"
}
Log2:
{
message: "SPECIAL_LOG_EMITTED Operation2_Emitted"
requestID: "123456"
}
So ideally I could do something like
fields #timestamp, #message, requestID
| filter #message like "SPECIAL_LOG_EMITTED"
| parse #message '*_Emitted' as operation
| stats count(*) as all, sum (operation LIKE 'Operation1') as Op1, sum (operation LIKE 'Operation2') as Operation2 by bin(5m)
And then from this find out where the requestID is matching. The requestID is dynamic, though, so I can't just hard-code it -- I want to find how often logs are emitted with matching requestIDs.
I've considered looking into count_distinct but that seems like the wrong approach (correct me if I'm wrong)
I'm trying to write a query that returns the vulnerabilities found by "Built-in Qualys vulnerability assessment" in log analytics.
It was all going smoothly I was getting the values from the properties Json and turning then into separated strings but I found out that some of the terms posses more than one value, and I need to get all of them in a single cell.
My query is like this right now
securityresources | where type =~ "microsoft.security/assessments/subassessments"
| extend assessmentKey=extract(#"(?i)providers/Microsoft.Security/assessments/([^/]*)", 1, id), IdAzure=tostring(properties.id)
| extend IdRecurso = tostring(properties.resourceDetails.id)
| extend NomeVulnerabilidade=tostring(properties.displayName),
Correcao=tostring(properties.remediation),
Categoria=tostring(properties.category),
Impacto=tostring(properties.impact),
Ameaca=tostring(properties.additionalData.threat),
severidade=tostring(properties.status.severity),
status=tostring(properties.status.code),
Referencia=tostring(properties.additionalData.vendorReferences[0].link),
CVE=tostring(properties.additionalData.cve[0].link)
| where assessmentKey == "1195afff-c881-495e-9bc5-1486211ae03f"
| where status == "Unhealthy"
| project IdRecurso, IdAzure, NomeVulnerabilidade, severidade, Categoria, CVE, Referencia, status, Impacto, Ameaca, Correcao
Ignore the awkward names of the columns, for they are in Portuguese.
As you can see in the "Referencia" and "CVE" columns, I'm able to extract the values from a specific index of the array, but I want all links of the whole array
Without sample input and expected output it's hard to understand what you need, so trying to guess here...
I think that summarize make_list(...) by ... will help you (see this to learn how to use make_list)
If this is not what you're looking for, please delete the question, and post a new one with minimal sample input (using datatable operator), and expected output, and we'll gladly help.
I want to search the log event
"Closure request counts: startAssets: "
and find occurrences where the startAssets are larger than 50.
How would I do that?
Something like:
Closure request counts: startAssets: 51
would maybe give a search similar to
"Closure request counts: startAssets: {num} AND num >=50"
perhaps?
What does that look like in SPL?
That's pretty simple, but you'll need to extract the number to do it. I like to use the rex command to do that, but there may be other ways.
index=foo "Closure request counts: startAssets: *"
| rex "startAssets: (?<startAssets>\d+)"
| where startAssets > 50
I have a search query like below.
index = abc_dev sourcetype = data RequestorSystem = * Description="Request Receieved from Consumer Service"
OR Description="Total Time taken in sending response"
| dedup TId
| eval InBoundCount=if(Description="Request Receieved from Consumer Service",1,0)
| eval OutBoundCount=if(Description="Total Time taken in sending response",1,0)
| stats sum(InBoundCount) as "Inbound Count",sum(OutBoundCount) as "Outbound Count"
I am not sure why inbound count is always showing as 0, outbound count works perfectly
There is a typo in your eval InBoundCount=... Received is spelled wrong, and if your events have it spelled correctly it won't match!
If that's not the case:
try running the query for both counts separately and make sure you are getting events. Also, posting some example input events will make our answer be more precise.
Splunk queries are joined by an implicit AND which means that your OR needs to either be included in parenthesis or (if you are using Splunk 6.6 or newer) use the IN keyword like so:
index = abc_dev sourcetype = data RequestorSystem = *
Description IN ("Request Receieved from Consumer Service", "Total Time taken in sending response")
Using IN is more portable in case you want add other strings later on. With some tweaks, you could even use a variation of stats count by Description with this.
I am trying to query using Rest API on splunk with the following:
curl -u "<user>":"<pass>" -k https://splunkserver.com:8089/services/search/jobs/export -d'search=search index%3d"<index_name" sourcetype%3d"access_combined_wcookie" starttime%3d06/02/2013:0:0:0 endtime%3d06/10/2013:0:0:0 uri_path%3d"<uri1>" OR uri_path%3d"<uri2>" user!%3d"-" referer!%3d"-" | eval Time %3d request_time_length%2f1000000 | stats stdev%28Time%29 as stdev, mean%28Time%29 as mean, count%28uri_path%29 as count by uri_path'
However I do not get the computed mean and stdev, I only see count. How can I add the mean and stdev?
The query looks about right. I tried a similar query on my end it seemed to give me all 3 aggregates. Only thing I can think of is to make sure you have events that match the search criteria. It could be your time boundaries. Try expanding those or maybe removing one/both of them to see if you get any data for mean and stdev.