Splunk search no subsearch - splunk

I have events something like:
{
taskId:5a6d
category:created
when:1517131461
...
}
{
taskId:5a6d
category:started
when:1517131609
...
}
{
taskId:5a6d
category:ended
when:1517134657
...
}
For each task (task id is same), we have events when it is created / started / ended.
I'd like to search if there is any task never be processed (task is created but not started). Here is my search statement:
index=XXX sourcetype=XXX category=created | search NOT [search index=XXX sourcetype=XXX category=started | fields taskId]
This statement works correctly if the time range is less than 48 hours.
If the time range is set to, for example, latest 7 days, the above search statement works incorrectly. It returns a lot of tasks (category=created) which means these tasks are never processed. Actually, they are processed, I can search the events (category=started) by taskId.
I have no idea what's wrong with it. it seems subsearch doesn't return correct results in the range of main search.

This will be hard to debug without seeing your exact data.
To make it simpler, you can try something like this to do everything with one search:
index=XXX sourcetype=XXX category=created
| eventstats values(category) as categories by taskId
| search categories = created NOT categories = started

Related

Count how often logs with dynamic field value that is the same gets emitted?

I don't think this is possible, but wanted to ask anyway -- is there a way in Cloudwatch Insights where I can find the count of how often a log with a dynamic value is emitted with the same value from distinct logs? The use case I have is we want to compare log statements from two different code paths, so we attach the same requestID to both log statements. So to illustrate what might happen, two logs may get emitted
Log 1:
{
message: "SPECIAL_LOG_EMITTED Operation1_Emitted"
requestID: "123456"
}
Log2:
{
message: "SPECIAL_LOG_EMITTED Operation2_Emitted"
requestID: "123456"
}
So ideally I could do something like
fields #timestamp, #message, requestID
| filter #message like "SPECIAL_LOG_EMITTED"
| parse #message '*_Emitted' as operation
| stats count(*) as all, sum (operation LIKE 'Operation1') as Op1, sum (operation LIKE 'Operation2') as Operation2 by bin(5m)
And then from this find out where the requestID is matching. The requestID is dynamic, though, so I can't just hard-code it -- I want to find how often logs are emitted with matching requestIDs.
I've considered looking into count_distinct but that seems like the wrong approach (correct me if I'm wrong)

splunk date time difference

I am new to Splunk. My goal is to optimize the API call, since that particular API method is taking more than 5 minutes to execute.
In Splunk I searched using context ID, I got all the functions and sub functions call by main API call function for that particular execution. Now I want to figure what which sub function took the maximum time. In Splunk in left side, in the list of fields, I see field name CallStartUtcTime (e.g. "2021-02-12T20:17:42.3308285Z") and CallEndUtcTime (e.g. "2021-02-12T20:18:02.3702937Z"). In search how can I write a function which will give me difference between these two times. I google and found we can use eval() function but for me its returning null value.
Additional Info:
search:
clicked on "create table view" and checked start, end and diff fields in the left side fields list. but all three are coming as null
not sure what wrong I am doing. I want to find out the time taken by each function.
Splunk cannot compare timestamps in string form. They must be converted to epoch (integer) form, first. Use the strptime() function for that.
...
| eval start = strptime(CallStartUtcTime, "%Y-%m-%dT%H:%M:%S.%7N%Z")
| eval end = strptime(CallEndUtcTime, "%Y-%m-%dT%H:%M:%S.%7N%Z")
| eval diff = end - start
...

Search using Lookup from a single field CSV file

I have a list of usernames that I have to monitor and the list is growing every day. I read Splunk documentation and it seems like lookup is the best way to handle this situation.
The goal is for my query to leverage the lookup function and prints out all the download events from all these users in the list.
Sample logs
index=proxy123 activity="download"
{
"machine":"1.1.1.1",
"username":"ABC#xyz.com",
"activity":"download"
}
{
"machine":"2.2.2.2",
"username":"ASDF#xyz.com",
"activity":"download"
}
{
"machine":"3.3.3.3",
"username":"GGG#xyz.com",
"activity":"download"
}
Sample Lookup (username.csv)
users
ABC#xyz.com
ASDF#xyz.com
BBB#xyz.com
Current query:
index=proxy123 activity="download" | lookup username.csv users OUTPUT users | where not isnull(users)
Result: 0 (which is not correct)
I probably don't understand lookup correctly. Can someone correct me and teach me the correct way?
In the lookup file, the name of the field is users, whereas in the event, it is username. Fortunately, the lookup command has a mechanism for renaming the fields during the lookup. Try the following
index=proxy123 activity="download" | lookup username.csv users AS username OUTPUT users | where isnotnull(users)
Now, depending on the volume of data you have in your index and how much data is being discarded when not matching a username in the CSV, there may be alternate approaches you can try, for example, this one using a subsearch.
index=proxy123 activity="download" [ | inputlookup username.csv | rename users AS username | return username ]
What happens here in the subsearch (the bit in the []) is that the subsearch will be expanded first, in this case, to (username="ABC#xyz.com" OR username="ASDF#xyz.com" OR username="BBB#xyz.com"). So your main search will turn into
index=proxy123 activity="download" (username="ABC#xyz.com" OR username="ASDF#xyz.com" OR username="BBB#xyz.com")
which may be more efficient than returning all the data in the index, then discarding anything that doesn't match the list of users.
This approach assumes that you have the username field extracted in the first place. If you don't, you can try the following.
index=proxy123 activity="download" [ | inputlookup username.csv | rename users AS search | format ]
This expanded search will be
index=proxy123 activity="download" "ABC#xyz.com" OR "ASDF#xyz.com" OR "BBB#xyz.com")
which may be more suitable to your data.

Splunk search issue

I have a search query like below.
index = abc_dev sourcetype = data RequestorSystem = * Description="Request Receieved from Consumer Service"
OR Description="Total Time taken in sending response"
| dedup TId
| eval InBoundCount=if(Description="Request Receieved from Consumer Service",1,0)
| eval OutBoundCount=if(Description="Total Time taken in sending response",1,0)
| stats sum(InBoundCount) as "Inbound Count",sum(OutBoundCount) as "Outbound Count"
I am not sure why inbound count is always showing as 0, outbound count works perfectly
There is a typo in your eval InBoundCount=... Received is spelled wrong, and if your events have it spelled correctly it won't match!
If that's not the case:
try running the query for both counts separately and make sure you are getting events. Also, posting some example input events will make our answer be more precise.
Splunk queries are joined by an implicit AND which means that your OR needs to either be included in parenthesis or (if you are using Splunk 6.6 or newer) use the IN keyword like so:
index = abc_dev sourcetype = data RequestorSystem = *
Description IN ("Request Receieved from Consumer Service", "Total Time taken in sending response")
Using IN is more portable in case you want add other strings later on. With some tweaks, you could even use a variation of stats count by Description with this.

Is there any KQL queries to extract page views, download counts from the W3C IISlogs on Azure-Log analytics?

We're trying to extract page views, file download count, users list from w3c IIS logs. we want to define what's page view, i.e. any user stayed on same page more than 10 sec to be a one page view. anything less is not a page view. w3c logs doesn't seem to be having enough data to extract this. can this be possible with what's already available?
This is the data available to extract the above info from,
Datatable operator
datatable (TimeGenerated:datetime, csUriStem:string, scStatus:string, csUserName:string, sSiteName :string)
[datetime(2019-04-12T11:55:13Z),"/Account/","302","-","WebsiteName",
datetime(2019-04-12T11:55:16Z),"/","302","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Account/","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Content/site.css","200","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Scripts/modernizr-2.8.3.js","200","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Scripts/bootstrap.js","200","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Content/bootstrap.css","200","-","WebsiteName",
datetime(2019-04-12T11:55:18Z),"/Scripts/jquery-3.3.1.js","200","-","WebsiteName",
datetime(2019-04-12T11:55:23Z),"/","302","-","WebsiteName",
datetime(2019-04-12T11:56:39Z),"/","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:57:13Z),"/Home/About","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:58:16Z),"/Home/Contact","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:59:03Z),"/","200","myemail#mycom.com","WebsiteName"]
I am not sure I got all your requirements right, but here is something to get started and provide you initial direction.
datatable (TimeGenerated:datetime, csUriStem:string, scStatus:string, csUserName:string, sSiteName :string)
[datetime(2019-04-12T11:55:13Z),"/Account/","302","-","WebsiteName",
datetime(2019-04-12T11:55:16Z),"/","302","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Account/","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Content/site.css","200","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Scripts/modernizr-2.8.3.js","200","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Scripts/bootstrap.js","200","-","WebsiteName",
datetime(2019-04-12T11:55:17Z),"/Content/bootstrap.css","200","-","WebsiteName",
datetime(2019-04-12T11:55:18Z),"/Scripts/jquery-3.3.1.js","200","-","WebsiteName",
datetime(2019-04-12T11:55:23Z),"/","302","-","WebsiteName",
datetime(2019-04-12T11:56:39Z),"/","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:57:13Z),"/Home/About","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:58:16Z),"/Home/Contact","200","myemail#mycom.com","WebsiteName",
datetime(2019-04-12T11:59:03Z),"/","200","myemail#mycom.com","WebsiteName"]
| where scStatus !in ('302') // exclude status 302
| where csUriStem !startswith '/Scripts' and csUriStem !endswith ".css" // exclude pages coming from '/Script' and .css files
| order by TimeGenerated asc
| summarize t=make_list(TimeGenerated) by csUriStem, csUserName // create time-series of visit events
| mv-apply t to typeof(datetime) on // run subquery on each of the series
(
project isVisit = (t - prev(t)) > 1min // compare with previous timestamp, and see if >1min passed
| summarize Visits=sum(isVisit)
)
| project csUriStem, csUserName, Visits
Here are links to make_list() (aggregation function), prev() (window function), summarize operator, and mv-apply operator