I have a Windows Service application. It is a very busy application. It is supposed to run continuously looking for things to do. After it runs for a while I get
Exception of type 'System.OutOfMemoryException' was thrown.
It can happen at different times but usually a this paragraph:
Private Shared Function GetUnprocessedQueue() As Boolean
Try
Dim l_svcOOA As New svcITGOOA.IsvcITGOOAClient(OOAProcessing.cGlobals.EndPoint_ITGOOA)
Dim l_iFilter As New svcITGOOA.clsFilter
With l_svcOOA
With l_iFilter
.FilingType = OOAProcessing.cGlobals.FilingType
End With
m_ReturnClass = .itgWcfOOA(1, cGlobals.DatabaseIndicator, svcITGOOA.eOOAAction.GetUnprocessedQueue, l_iFilter, 71)
Return CompletedGetUnprocessedQueue(m_ReturnClass)
End With
Catch ex As Exception
ExceptionHandling(ex, "GetUnprocessedQueue " & m_Application)
Return False
End Try
End Function
This is using a wcf service to read a queue. It reads the queue every two minutes to see if new records have been added to it.
Please help me solve this. I don’t know where to start.
The OutOfMemoryException exception occurs when the GC has completed a cycle of collection but the memory is not available even after that. I couldn't make out what the above code snippet does, but I think using Weak References for objects could be useful.
I had a timer that was generated within the same paragraph that I was setting
For example
m_svcTimer = New Timers.Timer With {.Interval = m_Interval, .Enabled = True}
AddHandler m_svcTimer.Elapsed, AddressOf StartTheQueueIfTime
m_svcTimer.Enabled = True
m_svcTimer.Start()
was within the paragraph StartTheQueueIfTime. I thought this would be a way to change the time interval. Instead it kept creating new events. Finally too many caused my crash.
Bob
Related
I have a service that watches 2 email inboxes. One thread is created for each inbox. The new messages are read and marked as Read on startup, then the NewMail event handler handles all new mail after that.
We have been using EWS version 14. We had no issues retrieving emails.
After we moved the emails to Office 365, the problems started.
On startup, the retrieval and handling of existing new messages is still working fine.
After startup, when the NewMail event handler is triggered, the bind/load method times out, regardless of what we set the Timeout property to.
We tried to switch to the newest EWS version (15.0).
With the new version, the startup handling still works fine.
However, trying to bind one email in the NewMail event handler makes no error, but makes the inbox thread quit.
When there is a new email message in both inboxes, the entire service freezes on the Bind line (i let it sit for 40 minutes before having to kill the service).
We tried both EWS versions with the following, with no success:
The below code is used in the NewEmail event handler AND the startup processing of existing new messages. Why would the same code have issues in one place and not the other? How do i fix this?
I have also tried to use EmailMessage.Bind using the NotificationEventArgs object, but i have the same issues.
We tried a SyncLock to get around the lock up and/or timeout, but that did not help.
Thanks!
Private Sub HaveMail(ByVal sender As Object, ByVal e As Microsoft.Exchange.WebServices.Data.NotificationEventArgs)
Dim error_occured As Boolean = False
Try
Try
SyncLock lock
Logger.LogDebug("Just entered the Locked state of the HaveMail event")
Dim ir As FindItemsResults(Of Item) = service.FindItems(WellKnownFolderName.Inbox, New SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, False), New ItemView(20))
Do While ir.Items.Count > 0
For Each i As EmailMessage In ir
Try
Logger.LogDebug("About to load a newly received email message")
i.Load(PropertySet.FirstClassProperties)
Catch ex As Exception
Throw New Exception("Error while loading an email message: " & ex.Message)
End Try
Dim rm As New ReceivedMessage
rm.ReceivedDateTime = i.DateTimeReceived
rm.ToAddress = i.DisplayTo
rm.Subject = i.Subject
rm.Body = i.Body
rm.FromAddress = i.From.Address
Logger.LogDebug("Just loaded a newly received email message from " & rm.FromAddress)
Dim rme As New ReceivedMessageEvent
rme.msg = rm
RaiseEvent GotMail(Me, rme)
Logger.LogDebug("Just processed a newly received email message from " & rm.FromAddress)
i.IsRead = True
i.Update(ConflictResolutionMode.AlwaysOverwrite)
Next
ir = service.FindItems(WellKnownFolderName.Inbox, New SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, False), New ItemView(20))
Loop
End SyncLock
Generally speaking, it's not a good practice to do much of anything from within the actual event handler. So Update, FindItems, and Bind would all make a call back to EWS from within your handler, and that is not good. The ExchangeService object is also not thread safe. How did you get away with it before O365? I have no answer other than just lucky.
Thanks to an answer on my posting on MSN, I was able to fix the problem.
I needed to raise the DefaultConnectionLimit to 100. Apparently, it defaults to 2, which was not enough.
I've got website hosted in azure (as a web service) that is using sql azure as it's backend.
My error logs have been heavily populated with what looks to be transient network and sql connection errors.
As such I've implemented the Enterprise Library Transient Error Handling Block. In tests it would seem that its operating correctly.
The issue I'm encountering is that I would like to log instances where this retry logic is occurring. From the documentation RetryPolicy.Retrying looks to be the event I'm after but in tests it doesn't fire. There's plenty of examples in C# that follow the following pattern to fire this event:
var retryPolicy = new RetryPolicy<SqlAzureTransientErrorDetectionStrategy>(retryStrategy);
// Receive notifications about retries.
retryPolicy.Retrying += (sender, args) =>
{
// Log details of the retry.
var msg = String.Format("Retry - Count:{0}, Delay:{1}, Exception:{2}",
args.CurrentRetryCount, args.Delay, args.LastException);
Trace.WriteLine(msg, "Information");
};
I thought I'd adapted this correctly but in short, what's wrong with the code below?!
Private RetryManager As RetryManager
Private WithEvents RetryPolicy As RetryPolicy
Private Sub RetryPolicy_Retrying(ByVal sender As Object, ByVal args As RetryingEventArgs)
' Log details of the retry.
Dim msg = String.Format("Retry - Count:{0}, Delay:{1}, Exception:{2}", args.CurrentRetryCount, args.Delay, args.LastException)
Trace.TraceInformation(msg)
End Sub
Private Sub SetupRetryPolicy()
'If its already set then lets not do it again
If RetryPolicy Is Nothing Then
RetryManager = EnterpriseLibraryContainer.Current.GetInstance(Of RetryManager)()
RetryPolicy = RetryManager.GetRetryPolicy(Of SqlAzureTransientErrorDetectionStrategy)("Exponential Backoff Retry Strategy")
' connect sub as handler to event when retry occurs
AddHandler RetryPolicy.Retrying, AddressOf RetryPolicy_Retrying
End If
End Sub
Public Sub ExecuteAndDoStuff(ByVal connString As String, ByVal cmdText As String)
SetupRetryPolicy()
'get a connection with retry
Using conn As New ReliableSqlConnection(connString, RetryPolicy, RetryPolicy)
conn.Open()
Using cmd As SqlCommand = conn.CreateCommand
Try
cmd.CommandText = cmdText
' this might be overkill, do I need to pass the retry policy in again for the command?
Dim dr As SqlDataReader = cmd.ExecuteReaderWithRetry(RetryPolicy, RetryPolicy)
'... do something with this datareader
Catch ex As Exception
'log error
Trace.TraceError("Query failed to execute despite retry logic: " & ex.ToString)
'continue to throw the error (picked up higher up the chain)
Throw ex
End Try
End Using
End Using
End Sub
I'm totally new to at least half of what's going on in this block of code but before anyone thows a rtfm at me - I tried!
It's hard to tell if anything is wrong in your code; it could be that there are no transient errors being detected at all. How are you making the determination that there are transient errors? The first thing I would do is to make sure you have a repeatable way of creating a transient error.
The way I would setup the test is to have a 1GB database in Azure SQL Database, fill it up with data up until it has reached its storage limit, and then try to add more data (which will generate a transient error everytime).
There are two things to keep in mind with Azure SQL transient errors:
1) They are very hard to test because many of them depend on variables that are outside of your control; one of the easiest transient error to replicate is out of space (the suggestion above)
2) There are a few other types of errors that can be fired, such as router swicthing conditions in Azure, that are not considered transient; for example IOException errors are not captured by the SQL Transient Strategy. As a result, you either need to account for those kinds of errors seperately, or customize the strategy to include these errors. Your catch block should trap these errors in your current implementation.
Imports System.Web
Imports System.Net
Imports System.Net.ServicePointManager
Public Class GetSource
Function GetHtml(ByVal strPage As String) As String
tryAgain:
ServicePointManager.UseNagleAlgorithm = True
ServicePointManager.Expect100Continue = True
ServicePointManager.CheckCertificateRevocationList = True
ServicePointManager.DefaultConnectionLimit = 100
Dim strReply As String = "NULL"
Try
Dim objhttprequest As System.Net.HttpWebRequest
Dim objhttpresponse As System.Net.HttpWebResponse
objhttprequest = System.Net.HttpWebRequest.Create(strPage)
objhttprequest.Proxy = proxyObject
objhttprequest.AllowAutoRedirect = True
objhttprequest.Timeout = 100000
objhttpresponse = objhttprequest.GetResponse
Dim objstrmreader As New StreamReader(objhttpresponse.GetResponseStream)
strReply = objstrmreader.ReadToEnd()
Catch ex2 As System.Net.WebException
GoTo tryAgain
Catch ex As Exception
strReply = "ERROR! " + ex.Message.ToString
GoTo tryAgain
End Try
Return strReply
End Function
What I got here is a vb.net code where I parse the website for its html
This function works fine.
The question is this...
1.If I run 100 threads with this function at the same time, Will it work?
2.Won't it affect my internet connection as well?
I don't want to waste time creating threads and codes a hundred times so if you know the answer please advice me on what should I do instead
One thing I see that could cause you problems is the goto. You retry if you get an error, but there is no way to break out of the method if an error does occur everytime you request the page, causing an infinite loop. You should put a check in, saying only try again if some cancel flag has not been set. Second, there could be issues with the number of threads you run depending on how much work each thread must do. There is a CPU and memory cost for each thread and it could peg your machine, especially if you get an infinite loop in one of them. Everything else gets a "it depends." Your pc and internet connection will determine everything else. There are tools available to monitor this and I would suggest using them to see what works. I found this page with a lot of information, it might have what you are looking for - http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html. Hope this helps.
Wade
Here is vb.net 2008 code is:
Public Function CheckURL(ByVal URL As String) As Boolean
Try
Dim Response As Net.WebResponse = Nothing
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Response = WebReq.GetResponse
Response.Close()
Return True
Catch ex As Exception
End Try
End Function
when a url is processing in checking it hangs my application for a while. Is this possible it checks smoothly all url list without hanging my application..
Is there any other fastest way to check urls?
Note: I have about 800 urls in file to check all links a valid by website responce or not.
If an exception occurs, the WebResponse object isn't properly disposed of. This can lead to your app running out of connections. Something like this will work better:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
This using the Using keyword ensures that the response is closed and finalized whenever that block exits.
If it's the server itself that's taking awhile to respond, look into the BeginGetResponse method on the HttpWebRequest. Check MSDN for a sample on how to use it. But be warned, that way also lies madness if you are not careful.
The answer is two fold:
Most of the waiting time is due to downloading content you don't need. If you request to only return the header, you will receive substantially less data, which will make your process faster.
As Matt identified, you aren't disposing of your connections, which may slow your process.
Expanding on Matt's answer, do the following:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
WebReq.Method = "HEAD" 'This is the important line.
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
GetResponse delivers you the whole content to your request. If this is what you want, there's not many room to speed up the request on the client side, since it mostly depends on the URLs server how fast to reply and how much data will be send over. If you just want to check if the URL is valid (or responding at all), it might be better to just ping it.
Keep in mind GetResponse isn't disposed when it runs into an error, so use the code posted by Matt to avoid this.
For your other problem, hanging application, you might avoid this be running this code as a thread.
This works basically like this (from here):
rem at the top of the code
Imports System.Threading
...
rem your event handler, p.e. button click or whatever
trd = New Thread(AddressOf ThreadTask)
trd.IsBackground = True
trd.Start()
rem your code
Private Sub ThreadTask()
dim i as long
Do
i += 1
Thread.Sleep(100)
Loop
End Sub
I'm running in to a problem that I was able to fix with Application.DoEvents, but don't want to leave that in because it might introduce all sorts of nasty problems.
Background:
Our app is primarily a desktop app that makes many calls to a web service. We control everything but changes to the overall system design are not going to be seriously considered. One of those calls, Calculate, is used very often, and occasionally can take a few minutes to process all the data to return valid results.
Previously this call to Calculate was done synchronously and thus would block the UI leaving the user to wonder if the app had frozen or not, etc. I've successfully moved all the long wait calls to a BackgroundWorker and then made a simple Waiting screen that would cycle through a "Calculating..." animated message.
Now the problem arises when our UI code tries to call the calculate routine again prior to the first one finishing. I would get a "This BackgroundWorker is currently busy and cannot run multiple instances..." message. Which I thought should be controlled by the resetEvent.WaitOne() calls. It did not so I thought maybe another event controlling access to the entire routine would help, so I added the calcDoneEvent. This still did not fix the problem, but would cause it to block indefinitely on the 2nd call to Calculate's calcDoneEvent.WaitOne() call. Then on a whim I added the Application.DoEvents to the bottom of Calculate and viola, problem solved.
I don't want to leave that .DoEvents in there because I've read it can cause problems that later are very difficult to track down. Is there a better way to handle this situation?
Thanks in advance..
Private WithEvents CalculateBGW As New System.ComponentModel.BackgroundWorker
Dim resetEvent As New Threading.AutoResetEvent(False)
Dim calcDoneEvent As New Threading.AutoResetEvent(True)
Public Sub Calculate()
calcDoneEvent.WaitOne() ' will wait if there is already a calculate running.'
calcDoneEvent.Reset()
' setup variables for the background worker'
CalculateBGW.RunWorkerAsync() ' Start the call to calculate'
Dim nMsgState As Integer = 0
' will block until the backgorundWorker is done'
Do While Not resetEvent.WaitOne(200) ' sleep for 200 miliseconds, then update the status window'
Select Case nMsgState
Case 1
PleaseWait(True, vbNull, "Calculating. ")
Case 2
PleaseWait(True, vbNull, "Calculating.. ")
Case 3
PleaseWait(True, vbNull, "Calculating... ")
Case 4
PleaseWait(True, vbNull, "Calculating....")
Case Else
PleaseWait(True, vbNull, "Calculating ")
End Select
nMsgState = (nMsgState + 1) Mod 5
Loop
PleaseWait(False, vbNull) 'make sure the wait screen goes away'
calcDoneEvent.Set() ' allow another calculate to proceed'
Application.DoEvents() ' I hate using this here'
End Sub
Private Sub CalculateBGW_DoWork(ByVal sender As System.Object, _
ByVal e As System.ComponentModel.DoWorkEventArgs) Handles CalculateBGW.DoWork
Try
'make WS Call, do data processing on it, can take a long time..'
'No Catch inside the DoWork for BGW, or exception handling wont work right...'
'Catch'
Finally
resetEvent.Set() 'unblock the main thread'
End Try
End Sub
Private Sub CalculateBGW_RunWorkerCompleted(ByVal sender As Object, _
ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles CalculateBGW.RunWorkerCompleted
'If an error occurs we must check e.Error prior to touching e.Result, or the BGW'
'will possibly "eat" the exception for breakfast (I hear theyre tasty w/ jam)'
If Not (e.Error Is Nothing) Then
'If a Web Exception timeout, retry the call'
If TypeOf e.Error Is System.Net.WebException And _
e.Error.Message = "The operation has timed out" And _
intRetryCount < intRetryMax Then
' Code for checking retry times, increasing timeout, then possibly recalling the BGW'
resetEvent.Reset()
CalculateBGW.RunWorkerAsync() 'restart the call to the WS'
Else
Throw e.Error ' after intRetryMax times, go ahead and throw the error up higher'
End If
Else
Try
'normal completion stuff'
Catch ex As Exception
Throw
End Try
End If
End Sub
You declared:
Private WithEvents CalculateBGW As New System.ComponentModel.BackgroundWorker
Dim resetEvent As New Threading.AutoResetEvent(False)
Dim calcDoneEvent As New Threading.AutoResetEvent(True)
as private fields of the containing class. Notice that this way, all calls to RunWorkerAsync() are referred to the same object instance of the BackgroundWorker class (that is, to the same object). That is why it is "busy". This code is built to hold only one BackgroundWorker at a given time.
If you mean to allow the UI code to call the Calculate() method whenever it needs to, you should declare CalculateBGW as a local variable within the Calculate() method, thus creating a new instance of the BackgroundWorker class with every call (and they will run asynchronosly). This means you'll have to add and remove the event handlers inside Calculate() as well, using AddHandler and RemoveHandler.
There are several approaches to updating the UI on the progress, but it is suggested to use the BackgroundWorker.ProgressChanged event and BackgroundWorker.ReportProgress method.
Use the BackgroundWorker.RunWorkerCompleted event as a callback trigger, reporting the UI that the calculation is completed, thus triggering the needed code to represent the result. This approach eliminates the need to maintain a thread looping around bossing the calculation thread - thereby eliminating the need for DoEvents(). It lets the calculation thread inform its boss when its done working, instead of having the boss checking the worker's status and going to sleep over and over.