Fetch values from multiple lines and combine - splunk

We forwarded app logs to splunk from different host and different sources.
Thread1 requestId=aUniqueID1 table=Table1
Thread1 size=2gb
Thread2 requestId=aUniqueID5 some other log
Thread1 requestId=aUniqueID1 some other log
Thread2 size=5gb
Thread1 requestId=aUniqueID2 table=Table2
Thread1 size=4gb
Thread1- requestId=aUniqueID3 table=Table3
Here I want to a table of request Id and size
aUniqueId1 2gb
aUniqueId2 4gb
aUniqueID5 5gb
Basically, thread name stays the same per host, but the same thread will be reused once a request is completed.
Can someone help write a Splunk query?
When a request comes, a log can be identified with thread name (Thread1,Thread2,etc)
Basically for a transaction, most log statements have requestId and I know the start and end times, thus I know the size within that time range (by host and sourcetype and thread name)

Here is a run-anywhere example of how to do that.
| makeresults | eval data="Thread1 requestId=aUniqueID1 table=Table1
Thread1 size=2gb
Thread1 requestId=aUniqueID1 some otherlog
Thread1 requestId=aUniqueID2 table=Table2
Thread1 size=4gb
Thread1 requestId=aUniqueID3 table=Table3" | eval data=split(data,"
") | mvexpand data | eval _raw=data
```Everything above sets up test data. Omit when using live data```
```Combine requestId and size lines```
| transaction startswith="requestId" endswith="size"
```Parse the results```
| extract kvdelim=" ", pairdelim="="
```Display request ID and corresponding size```
| table requestId size

Related

Redis streams is returning an empty array

I created a new Redis steam using the following command.
XGROUP CREATE A mygroup $ MKSTREAM
I added the below mentioned data
xadd A * X 1
xadd A * X 2
xadd A * X 3
xadd A * X 4
I am reading the data using the following command.
XREADGROUP GROUP mygroup Alice COUNT 1 STREAMS A 0
Its returning an empty array
1) 1) "A"
2) (empty array)
I am using Redis version 6.2.1. Kindly help me to debug the error.
When you use XREADGROUP command to read message, you should specify > as ID, instead of 0.
Reference from the doc:
The special > ID, which means that the consumer want to receive only messages that were never delivered to any other consumer. It just means, give me new messages.
Any other ID, that is, 0 or any other valid ID or incomplete ID (just the millisecond time part), will have the effect of returning entries that are pending for the consumer sending the command with IDs greater than the one provided. So basically if the ID is not >, then the command will just let the client access its pending entries: messages delivered to it, but not yet acknowledged. Note that in this case, both BLOCK and NOACK are ignored.
If ID is not >, you can only read pending messages, however, in your case, there's no pending message, since you have not consume anything.

how to check if splunk has received the logs from 100 different hosts

I am new to splunk. Wanted to create a splunk alert to check if logs has been received from all the host or not and if not need to set a alert trigger.
| tstats latest(_time) as latest where index=* earliest=-24h by host
| eval recent = if(latest > relative_time(now(),"-5m"),1,0), realLatest = strftime(latest,"%c")
| where recent=0
is the above splunk Query correct?
The query looks good, but the best way to know is to try it. Does it produce the desired results?

Kusto memory status for an operation id

I executed the following control command
.set-or-append async XXXX<|fillXXXX()
This returned me an operation id
Now I want to check how much CPU/MEMORY usage (Query stats) happened for this operation id.
How can we do that?
When you run the command, you also get the ClientRequestId, and that's what you should use to get the resources used to run the command, :
.show commands
| where StartedOn > ago(1d)
| where ClientActivityId == "KE.RunCommand;9763ec24-910c-4e86-a823-339db440216e"
| where CommandType == "TableSetOrAppend"
| project ResourcesUtilization

Is it possible to create log source health alerts in Azure Sentinel?

I am attempting to create an alert that lets me know if a data source stops providing logs to Sentinel. While I know it displays anomalies in log data on the dash board, I am hoping to receive alerts if a source stops providing logs for an extended period of time.
Something like creating a rule with the following query (CEF in this case):
CommonSecurityLog
| where TimeGenerated > ago(24h)
| summarize count() by DeviceVendor, DeviceProduct, DeviceName, DeviceExternalID
| where count_ == 0

getting the thread id of threads in the threadpool from outside threadprocessing method

I am implementing multi-threading concepts using thread-pooling to pick up messages from queues. There is a necessity that the id/number of the thread which picks up a particular message has to be accessed from outside the thread method. Supposes if there are three threads and they have to pick up 5 messages from the queue. the first thread picks up the first message processes it and it is released. meanwhile the second thread would have picked up some other message and would've started to process it(multi-threading).
Here from some other method, i want to know which thread actually picks up message1, and the consistency has to maintained not only inside the threading function but throughout the application(from outside the thread function also).
i have implemented multi-threading as follows
For m_intThread1 = 0 To m_stuTPConfig.intNumThreads - 1
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf **MultiProcessMQ**), m_intThread1)
System.Threading.Thread.Sleep(1000)
Next m_intThread1 'm_intThread
Public Shared Sub MultiProcessMQ(ByVal state As Object)
Dim objParentProcess As New ParentProcess
objParentProcess.ProcessThread(CType(state, String))
If Interlocked.Decrement(CInt(m_stuTPConfig.intNumThreads)) = 0 Then
m_asyncOpsAreDone.Set()
End If 'Interlocked.Decrement(CInt(m_stuTPConfig.intNumThreads)) = 0
End Sub 'MultiProcessMQ()
Public Sub ProcessThread(ByVal strThread As String)
intThrd = Convert.ToInt32(strThread)
Console.WriteLine("Parent Thread started " & strThread)
End Sub
How do i access the variable strThread From another method and maintain its consistency.This is required for logging purposes.
Thanks for formatting. We are using WCFMQAdapter services which is a for configuring the app in accordance with the input and output queues. once I host this service from ProcessThread the control goes to some other file, say a service.vb file wherein all the operations pertaining to the thread are performed and never comes back. The actual usage of threads here are for logging the processing details onto a file in the service.vb file. Now the PROCESSTHREAD method is called. till now the multithreading properties are maintained. once the service is hosted how to pass on these thread info to the service.vb. Or should there be a parent threading/child threading concept? And, if i obtain a threadid using managedThreadId, dynamic thread ids are generated. Are there any chances to obtain them and convert it into user managed ids. say, instead of thread 21,thread 44,thread66,thread89........ can we convert them like thread1, thread 2, thread 3, thread 4? Thanks in Advance :-)
You can use System.Threading.Thread.CurrentThread.ManagedThreadId to get a unique id for each thread. I'd use that to identify your threads.