I am implementing multi-threading concepts using thread-pooling to pick up messages from queues. There is a necessity that the id/number of the thread which picks up a particular message has to be accessed from outside the thread method. Supposes if there are three threads and they have to pick up 5 messages from the queue. the first thread picks up the first message processes it and it is released. meanwhile the second thread would have picked up some other message and would've started to process it(multi-threading).
Here from some other method, i want to know which thread actually picks up message1, and the consistency has to maintained not only inside the threading function but throughout the application(from outside the thread function also).
i have implemented multi-threading as follows
For m_intThread1 = 0 To m_stuTPConfig.intNumThreads - 1
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf **MultiProcessMQ**), m_intThread1)
System.Threading.Thread.Sleep(1000)
Next m_intThread1 'm_intThread
Public Shared Sub MultiProcessMQ(ByVal state As Object)
Dim objParentProcess As New ParentProcess
objParentProcess.ProcessThread(CType(state, String))
If Interlocked.Decrement(CInt(m_stuTPConfig.intNumThreads)) = 0 Then
m_asyncOpsAreDone.Set()
End If 'Interlocked.Decrement(CInt(m_stuTPConfig.intNumThreads)) = 0
End Sub 'MultiProcessMQ()
Public Sub ProcessThread(ByVal strThread As String)
intThrd = Convert.ToInt32(strThread)
Console.WriteLine("Parent Thread started " & strThread)
End Sub
How do i access the variable strThread From another method and maintain its consistency.This is required for logging purposes.
Thanks for formatting. We are using WCFMQAdapter services which is a for configuring the app in accordance with the input and output queues. once I host this service from ProcessThread the control goes to some other file, say a service.vb file wherein all the operations pertaining to the thread are performed and never comes back. The actual usage of threads here are for logging the processing details onto a file in the service.vb file. Now the PROCESSTHREAD method is called. till now the multithreading properties are maintained. once the service is hosted how to pass on these thread info to the service.vb. Or should there be a parent threading/child threading concept? And, if i obtain a threadid using managedThreadId, dynamic thread ids are generated. Are there any chances to obtain them and convert it into user managed ids. say, instead of thread 21,thread 44,thread66,thread89........ can we convert them like thread1, thread 2, thread 3, thread 4? Thanks in Advance :-)
You can use System.Threading.Thread.CurrentThread.ManagedThreadId to get a unique id for each thread. I'd use that to identify your threads.
Related
I'm struggling to model asynchronous servlet request processing with custom JFR events.
The challenge I'm facing is that in asynchronous processing a request may be #dispatch()ed several times. This means the whole request processing chain may be executed multiple times, some time apart in different threads. How do I model this with custom JFR events?
What would help me is either the concept of a "parent" event (possibly in a different thread) or the suspension and resumption of an event.
Edit
To illustrate the issue a bit. An async request may take 100 seconds wall clock time to process. However the actual processing may happen in only 4 seconds user time in a Servlet#service() method:
second 0-1 in thread A, Servlet#service() method returns, AsyncContext started
second 10-11 in thread B, Servlet#service() method returns, AsyncContext started
second 80-81 in thread A, Servlet#service() method returns, AsyncContext started
second 99-100 in thread C, Servlet#service() method returns
I'm only interested in generating events for these four durations in these three threads and then correlating them with a single request.
You can add a thread field to the event
public class MyEvent extends Event [
#Label("Start Thread")
#TransitionFrom
private final Thread startThread;
MyEvent(Thread thread) {
this.startThread = thread;
}
]
When you commit the event the end thread will be stored.
If you want to track an event over several threads, you would need to create an event for every thread and have an id so you can understand the flow.
class MyEvent extends Event {
#Label("Transition id");
long id;
}
If you like you can create a relational id to describe the relation and JMC should be able to hint (in context menus etc.) there is a relation among events.
#Label("Transition Id")
#Relational
#Target({ ElementType.FIELD })
#Retention(RetentionPolicy.RUNTIME)
#interface TransitionId {
}
If you don't want to repeat yourself, you can write the above functionality in a method in a base class, which you can call for every new thread the event visits.
abstract AbstractTransition extends Event {
#TransitionId
#Label("Transition Id")
private long id;
public void setTransitionId(long id) {
this.id = id;
}
}
There is no other way to do this.
It's not possible for the JVM to know what thread an event object is in, or what threads that should be recorded. The user needs to provide at least one method call for every thread that should be touched (together with some context).
The problem is similar to how to tie JFR events for spans and scopes together in distributed tracers.
This article may help:
http://hirt.se/blog/?p=1081
I see following call stack for finalizer thread. Is it normal to have a call to WaitForSingleObject at top in finalizer? Is there anyway I can determine if its not deadlocked or waiting for really long time?
0:009> k
Child-SP RetAddr Call Site
00000000`0a56e5c8 000007fe`fd5010dc ntdll!NtWaitForSingleObject+0xa
00000000`0a56e5d0 000007fe`fdfabeb5 KERNELBASE!WaitForSingleObjectEx+0x79
00000000`0a56e670 000007fe`fe04a576 rpcrt4!UTIL_GetOverlappedResultEx+0x45
00000000`0a56e6b0 000007fe`fdfaf0dd rpcrt4!WS_SyncRecv+0xf6
00000000`0a56e720 000007fe`fdfe7a29 rpcrt4!OSF_CCONNECTION::TransSendReceive+0x18d
00000000`0a56e780 000007fe`fdfa7f61 rpcrt4!OSF_CCONNECTION::SendBindPacket+0xa5c
00000000`0a56e930 000007fe`fdfa8e27 rpcrt4!OSF_CCONNECTION::ActuallyDoBinding+0xc1
00000000`0a56e9d0 000007fe`fdfa8bb6 rpcrt4!OSF_CCONNECTION::OpenConnectionAndBind+0x207
00000000`0a56ea90 000007fe`fdfa8acd rpcrt4!OSF_CCALL::BindToServer+0xc6
00000000`0a56eb40 000007fe`fdfadaeb rpcrt4!OSF_BINDING_HANDLE::InitCCallWithAssociation+0xa5
00000000`0a56eba0 000007fe`fdfad9d0 rpcrt4!OSF_BINDING_HANDLE::AllocateCCall+0x102
00000000`0a56ecd0 000007fe`fdfc74eb rpcrt4!OSF_BINDING_HANDLE::NegotiateTransferSyntax+0x30
00000000`0a56ed20 000007fe`ff462271 rpcrt4!I_RpcNegotiateTransferSyntax+0x9f
00000000`0a56edb0 000007fe`ff45d185 ole32!CRpcChannelBuffer::NegotiateSyntax+0x69
00000000`0a56ee20 000007fe`fe05ba22 ole32!NdrExtNegotiateTransferSyntax+0xe5
00000000`0a56ee60 000007fe`fe05cbbb rpcrt4!Ndr64pClientSetupTransferSyntax+0x453
00000000`0a56eec0 000007fe`ff4621d0 rpcrt4!NdrpClientCall3+0xcb
00000000`0a56f180 000007fe`ff31d8a2 ole32!ObjectStublessClient+0x11d
00000000`0a56f510 000007fe`ff321bb3 ole32!ObjectStubless+0x42
00000000`0a56f560 000007fe`ff321b22 ole32!RemoteReleaseRifRefHelper+0x57
00000000`0a56f5b0 000007fe`ff3217eb ole32!RemoteReleaseRifRef+0xca
00000000`0a56f620 000007fe`ff321417 ole32!CStdMarshal::DisconnectCliIPIDs+0x4c2
00000000`0a56f720 000007fe`ff3194fa ole32!CStdMarshal::Disconnect+0x40c
00000000`0a56f780 000007fe`ff319428 ole32!CStdIdentity::~CStdIdentity+0xa6 [d:\w7rtm\com\ole32\com\dcomrem\stdid.cxx # 312]
00000000`0a56f7b0 000007fe`ff319b49 ole32!CStdIdentity::`scalar deleting destructor'+0x14
00000000`0a56f7e0 000007fe`f2a79f94 ole32!CStdIdentity::CInternalUnk::Release+0xdc [d:\w7rtm\com\ole32\com\dcomrem\stdid.cxx # 767]
00000000`0a56f810 000007fe`f2ba8bea clr!SafeReleasePreemp+0x74
00000000`0a56f860 000007fe`f2ba8acc clr!RCW::ReleaseAllInterfaces+0xda
00000000`0a56f8b0 000007fe`f2ba8c14 clr!RCW::ReleaseAllInterfacesCallBack+0x54
00000000`0a56f930 000007fe`f2ba937e clr!RCW::Cleanup+0x25
00000000`0a56f980 000007fe`f2ba9214 clr!RCWCleanupList::ReleaseRCWListRaw+0x16
00000000`0a56f9b0 000007fe`f2bb005a clr!RCWCleanupList::ReleaseRCWListInCorrectCtx+0x94
00000000`0a56f9f0 000007fe`f2be326e clr!RCWCleanupList::CleanupAllWrappers+0xe5
00000000`0a56fa70 000007fe`f2be319f clr!SyncBlockCache::CleanupSyncBlocks+0xc2
00000000`0a56fae0 000007fe`f2be47c7 clr!Thread::DoExtraWorkForFinalizer+0xdc
00000000`0a56fb10 000007fe`f2a5458c clr!SVR::GCHeap::FinalizerThreadWorker+0x78
00000000`0a56fb50 000007fe`f2a5451a clr!Frame::Pop+0x50
00000000`0a56fb90 000007fe`f2a54491 clr!COMCustomAttribute::PopSecurityContextFrame+0x192
00000000`0a56fc90 000007fe`f2b31bfe clr!COMCustomAttribute::PopSecurityContextFrame+0xbd
00000000`0a56fd20 000007fe`f2b45020 clr!ManagedThreadBase_NoADTransition+0x3f
00000000`0a56fd80 000007fe`f2ab33de clr!SVR::GCHeap::FinalizerThreadStart+0xb4
00000000`0a56fdc0 00000000`772459ed clr!Thread::intermediateThreadProc+0x7d
00000000`0a56fe80 00000000`7747c541 kernel32!BaseThreadInitThunk+0xd
00000000`0a56feb0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
0:009> !CLRStack
OS Thread Id: 0x1338 (6)
Child SP IP Call Site
000000000b67fcd8 00000000774a12fa [DebuggerU2MCatchHandlerFrame: 000000000b67fcd8]
This is a finalizer worker and it is blocked for a RPC response. In this case if the thread is not waiting for more than a few milliseconds (depending on how long the COM server takes to respond to the request) it is normal. But if we are waiting longer then you might want to investigate what happened to the response packet.
There is no way in user mode to understand what's the tick-count of the thread you may want to examine the same thread using livekd on a running system if the problem is re-producible
It is this indication of a blocked finalizer. A very simple example if you use COM to interact between Excel and C#.
You have a Module of Excel that is being made a member variable of C# and then the issue might arise as the control of that module is within GC's hands but excel will be waiting for it to be released by the C#.
Simple way is to just clear any unmanaged code you have in your C# component before letting the unmanaged code to be cleared.
I had a MSMQ application setup where data was being pushed into one queue. Initially I only had one process reading from it and processing it. Since the volume has increased I started multiple processes to read from it which is basically a new instance of my original process. I do not see any errors but the performance has really dropped. My understanding is that each process will read from a queue and receive a new message that has not yet been processed and continue with that. Is this correct or is it possible that multiple processes could end up processing the same message?
Dim q As MessageQueue
If MessageQueue.Exists(".\private$\MsgsIQueue") Then
q = New MessageQueue(".\private$\MsgsIQueue")
Else
'GS - If there is no queue then we're done here
Console.WriteLine("Queue has not been created!")
Return
End If
While True
Dim message As Message
counter += 1
Try
If q.Transactional = True Then
Thread.Sleep(2000)
End If
q.MessageReadPropertyFilter.ArrivedTime = True
message = q.Peek(TimeSpan.FromSeconds(20.0))
message.UseJournalQueue = True
message = q.Receive(New TimeSpan(0, 0, 60))
message.Formatter = New XmlMessageFormatter
(New [String]() {"System.String"})
ProcessMessage(message)
....
Ok, are you sure that it is the queue reading that is actually causing the performance degradation? I would suspect that there is some other bottleneck in your pipeline as MSMQ is really good at handling reading from multiple processes/threads.
If I take a look at your code I would suggest the following changes:
Why sleep for 2 secs if is a tx queue? Always use tx queues and move the call to Sleep to the catch block to have a wait interval if the queue is empty.
Move the setting of the filter outside of the loop.
Remove the call to Peek as it performs nothing of value.
Use journal queue is only of use when sending messages. So remove it.
Set the formatter on the queue instead and it will be used for all reads.
You should also wrap the call to Read and ProcessMessage within a TransactionScope where you also wrap ProcessMessage in another try/catch block. This way you can commit the read if everything went Ok in ProcessMessage or otherwise choose to abort the read or move the message to a dead letter queue.
ok here is my code :
For i = 0 To 10
Dim tTemp As Threading.Thread = New Threading.Thread(AddressOf dwnld)
tTemp.IsBackground = True
'tTemp.Start(geturl)
lThreads.Add(tTemp)
'MsgBox(lThreads.Item(i).ThreadState)
Next
I create a list of threads with 10 threads, assign them a function, properties and add them to the list.
'While ListBox2.Items.Count > 0
For i = 0 To lThreads.Count - 1
If (lThreads.Item(i).ThreadState = 12) Then
If (ListBox2.Items.Count > 0) Then
lThreads.Item(i).Start(geturl)
If (i = lThreads.Count - 1) Then
i = 0
End If
Else
Exit For
End If
'MsgBox(lThreads.Item(i).ThreadState)
ElseIf (lThreads.Item(i).ThreadState = 16) Then
lThreads.RemoveAt(i)
Dim tTemp As Threading.Thread = New Threading.Thread(AddressOf dwnld)
tTemp.IsBackground = True
lThreads.Add(tTemp)
If (i = lThreads.Count - 1) Then
i = 0
End If
End If
Next
What's happening is, i see the threads stop after the function dwnld is completed. So i first check for the state (12 means background and unstarted). On case 12 start the thread and in case 16 (stopped) remove that particular thread and add a different thread like i add 10 above.
Also there is a check when the i counter reaches last number, restart the whole loop by assigning i=0.
The program downloads some web pages, the url is passed from the listbox2. The geturl will pass the url and remove it from the list. So when the listbox is empty, exit the for loop.
But the above code is running for only 11 times and it does not restart. I tried using a lable and goto but it simple hangs.
Can anyone tell me what to do?
What i want is to maintain 10 threads to keep downloading the web pages and when the list is empty, exit the function.
Trying to manually manage your own custom pool of threads is probably the wrong approach here. Use ThreadPool.QueueUserWorkItem or preferrably the new Task class. The thread pooling is managed for you which greatly simplifies the code. Completely scrap this code and start over using one of the techniques I just mentioned. If you run into problems implementing either of these techniques then post a more specific question.
Micro-management of threads is, well, just a really bad idea. The moment I see anyone trying to maintain a list of threads that are continually created, terminated and destroyed I just know they are doomed. I have seen experienced professionals trying to do it - it's fun looking on, waiting for the inevitable spectacular failure after months of trying to fix the unfixable.
Thread pools are, typically, nothing of the sort. They are usually a pool of tasks - task class instances on a producer-consumer queue - that several threads feed off as and when they are free to do work. The work threads auto-manage themselves by getting new tasks themselves when they have finished with the old one - no need for any higher-level micro management.
Listen to #Brian - forget managing lists of threads, checking their state and all that gunge. It'll just make you ill. Go with ThreadPool.QUWI or Tasks.
So I have a very weird problem.
I am writing some code in VB.Net under .NET 2.0 which interfaces with MS Exchange 2003. Because of the Exchange 2003 "requirement" I am forced to write this code using WEBDAV.
The code itself is replicating, to some degree, a schedule management process. It's creating Appointments on the Exchange Server in response to inputs from the user and managing it's data internally in a SQL Server database.
The problem situation is this: A new person is assigned to be in charge of a meeting. The requirement says the program should generate a meeting cancellation request to the person removed from the meeting (if such a person existed) and a meeting request sent to the new person.
In the case of there being an existing person, what appears to happen is this:
The meeting cancellation request
gets sent
Exchange barfs and returns status code 500 (internal server error) during the set of
requests which send the meeting request to the new person.
However! While debugging this particular scenario, it works just fine for me, if I step through the code in the Visual Studio debugger. Left to it's own devices, it fails every time.
Just for yuk's sake, I added a Thread.Sleep(500) to the part after sending the cancellation request, and Exchange doesn't barf anymore...
So, my question!
If adding a Thread.Sleep to the code causes this error to go away, a race condition is implied, no? But, my code is running under a web application and is a totally single threaded process, from start to finish. The web requests I am sending are all in synchronous mode so this shouldn't be a problem.
What would I do next to try and track down the issue?
Try and divine if the race condition itself is in the .Net 2.0 BCL networking code?
Try and do some debugging on the Exchange server itself?
Ignore it, be glad the Thread.Sleep masks the problem and keep on going?
Any further suggestions would be wonderful.
In response to comment, I can post the failing function:
Private Shared Sub UpdateMeeting(ByVal folder As String, ByVal meetingId As String, ByVal oldAssignedId As String, ByVal newAssignedTo As String, ByVal transaction As DbTransaction)
If String.IsNullOrEmpty(meetingId) Then
Throw New Exception("Outlook ID for that date and time is empty.")
End If
Dim x As New Collections.Generic.List(Of String)
If oldAssignedId <> newAssignedTo AndAlso Not String.IsNullOrEmpty(oldAssignedId) Then
'send cancellation to old person
Dim lGetCounselorEmail1 As String = GetCounselorEmail(oldAssignedId, transaction)
Common.Exchange.SendCancellation(meetingId, New String() {lGetCounselorEmail1})
' Something very weird here. Running this code through the debugger works fine. Running without causes exchange to return 500 - Internal Server Error.
Threading.Thread.Sleep(500)
End If
x.Add(folder)
If Not String.IsNullOrEmpty(newAssignedTo) Then x.Add(GetCounselorEmail(newAssignedTo, transaction))
x.RemoveAll(AddressOf String.IsNullOrEmpty)
If x.Count > 0 Then
If Not Common.Exchange.UpdateMeetingAttendees(meetingId, x.ToArray()) Then
Throw New Exception("Failure during update of calendar")
End If
End If
End Sub
...but a lot of the implementation details are hidden here, as I wrote a set of classes to interface with Exchange WebDAV.
Ended up sticking with the Sleep and calling it a day.
My 'belief' is that I was erroneous in thinking that a WebRequest/WebResponse combo sent to Exchange through WebDav was an atomic operation.