Splunk Concurrency Calculation - splunk

Question

I have some data from logs in Splunk where I need to determine what other requests were running concurrently at the time of any single event.
Using the following query, I was able to have it return a column for the number of requests that ran at the same time within my start time and duration.
index="sfdc" source="sfdc_event_log://EventLog_SFDC_Production_eventlog_hourly" EVENT_TYPE IN (API, RestAPI) RUN_TIME>20000
| eval endTime=_time
| eval permitTimeInSecs=(RUN_TIME-20000)/1000
| eval permitAcquiredTime=endTime-permitTimeInSecs
| eval dbTotalTime=DB_TOTAL_TIME/1000000
| concurrency start=permitAcquiredTime duration=permitTimeInSecs
| table _time API_TYPE EVENT_TYPE ENTITY_NAME apimethod concurrency permitAcquiredTime permitTimeInSecs RUN_TIME CPU_TIME dbtotalTime REQUEST_ID USER_ID
| fieldformat dbTotalTime=round(dbTotalTime,0)
| rename permitAcquiredTime as "Start Time", permitTimeInSecs as "Concurrency Duration", concurrency as "Concurrent Running Events", API_TYPE as "API Type", EVENT_TYPE as "Event Type", ENTITY_NAME as "Entity Name", apimethod as "API Method", RUN_TIME as "Run Time", CPU_TIME as "CPU Time", dbtotalTime as "DB Total Time", REQUEST_ID as "Request ID", USER_ID as "User ID"
| sort "Concurrent Running Events" desc
I am now trying to investigate a single event in these results. For example, the top event says that at the time it ran, there were 108 concurrent requests running in the 20 second window of time.
How can I identify those 108 events using this data?
I imagine it would be querying the events that had a specific time frame range, but I am not sure if I need to check something like _time + - 10 seconds to see what was running within the 20 second window?
Just need to understand the data behind this 108 events a little more for this top example. My end goal here is to be able to add a drill-down to the dashboard so that when I click on the 108, I can see those events that were running.

Answer 1

Essentially, you are on right lines. What you want to do is create a search (presumably on the original data) using 'earliest=<beginning of 20 second window> latest=<end of 20 second window> using your calculated values.
You have start time and can calculate end time. Then pipe these as variables into a new search.
| search earliest=start_time latest=end_time index="sfdc" etc..
I cant check this here right now. But its probably something along those lines. Quite likely more elegant ways to do the same. Hope I'm not wildly off mark and this at least helps a little.