Exchange Web Services Attachment load is slow - vb.net

I'm in the process of writing some code to download and process the attachments of email messages and then process them. The code is working as required in some cases, but still has some major problems.
Whenever the code loads the attachment into a file on the local disk, it takes a very long time to do so and often times out with the following exception as a result of the slow download:
A first chance exception of type 'Microsoft.Exchange.WebServices.Data.ServiceRequestException' occurred in Microsoft.Exchange.WebServices.dll
I may be wrong, but if the exchange server in question is on the same gigabit network as the server running the code and outlook has fast access to emails, attachments, etc then attachments should download considerably faster than they do and much more consistently. Here are some example of download/load times:
800KB Zip - 1m 4s
840KB Zip - 6m 18s
1.33MB Zip - 11m 23s
2.78MB Zip - 17m 3s
I have tried setting the EWS connection timeout setting to 300000ms instead of the default 100000ms to give the attachments more time to download and the number of exceptions has decreased slightly but the waiting time is now way too long.
The code does run in threads, no more than 8 at a time (10 being the throttling limit for EWS i believe), but i cant imagine that would make much of a difference. (It hasn't done when I've been testing single emails at a time).
Here is the threaded code that downloads the attachments (some un-related bits removed for simplicity):
Dim strMessageFolder As String
' Prepare the directory where this emails attachments will be stored
strMessageFolder = g_strFolder_Temp & strMessageID & "\"
' Create a folder to store the attachments for this email
Call FileSystem_CreateFolder(strMessageFolder, True)
' Process the emails attachments
For Each emailAttachment In emailMessage.Attachments
Dim fileattach As FileAttachment
'Dim fileattachStream As FileStream
Dim strAttachmentFile As String
' Prepare for the downloading of the attachment
fileattach = emailAttachment
blnTryFailed = False
intAttempts = 0
strAttachmentFile = strMessageFolder & fileattach.Name
' Handle up to 3 download attempts
Do
Try
' Try to download the attachment - Method 1
fileattach.Load(strAttachmentFile)
' Try to download the attachment - Method 2
'fileattachStream = New FileStream(strAttachmentFile, FileMode.OpenOrCreate, FileAccess.ReadWrite)
'fileattach.Load(fileattachStream)
'fileattachStream.Close()
'fileattachStream.Dispose()
blnTryFailed = False
Catch ex As Exception
blnTryFailed = True
' Ensure the failed download is deleted
Call FileSystem_DeleteFile(strAttachmentFile)
intAttempts += 1
End Try
Loop While blnTryFailed And intAttempts < 3
' If the attachment download was unsuccessful then we cannot process the current email
If blnTryFailed = True Then
emailMessage.IsRead = False
'message.Subject = message.Subject & " - Attachment download failed, skipped"
Try
emailMessage.Update(ConflictResolutionMode.AutoResolve)
Catch ex As Exception
Call Logging_Add("Unable to mark email as skipped", strMessageID, "Debug")
End Try
Exit Sub
End If
As mentioned previously, im aware of the Exchange throttling but cannot find anything related to the speed at which attachments are downloaded.
So my question is what could possibly be causing such slow download speeds?

I have the same issue with my app. The problem was caused by default EWS setting which writes to Console all HttpRequest and HttpResponse messages between EWS and application. Turning off TraceFlags was blessing.
My code in c#:
ExchangeService service = new ExchangeService(ExchangeVersion.Exchange2010_SP2);
service.TraceFlags = TraceFlags.None;

Related

Weird Issue With Windows Service - Service Timing Out

Hi there I have piece of legacy (VS2010) Windows Service code that I have imported into VS2017 and is causing me severe frustration. This code has worked well for about the last 6 years, however when I carry out the install and attempt to start the service the SCM comes back with a timeout error. The OnStart code is as follows:
Protected Overrides Sub OnStart(ByVal args() As String)
'Instaniate the timer for the service
_serviceTimer = New Threading.Timer(New Threading.TimerCallback(AddressOf Tick), Nothing, 60000, 60000)
End Sub
The call back is:
Private Sub Tick(ByVal state As Object)
'Switch off the timer event whilst the code executes
_serviceTimer.Change(System.Threading.Timeout.Infinite, System.Threading.Timeout.Infinite)
If Not _started Then
Startup()
_started = True
End If
Call ServerProcess()
'Re-enable the timer now that application code has completed
_serviceTimer.Change(_longInterval, _longInterval)
End Sub
I originally had the Startup process in the OnStart method, however removed it as an attempt at resolving this issue, however it has not made any difference. Method Startup is as follows:
Public Sub Startup()
Try
'Source these settings from the local config file
_appDataFolder = Utilities.GetSetting("AppDataRoot")
_configPathMapped = _appDataFolder & Utilities.GetSetting("ConfigPathMapped")
_logPath = _appDataFolder & "\" & utl.GetSetting("LogPath")
'Instaniate the timer for the service - Commented out after moving startup code from OnStart method
' _serviceTimer = New Threading.Timer(New Threading.TimerCallback(AddressOf Tick), Nothing, Timeout.Infinite, Timeout.Infinite)
'Initialise logging architecture
_logger = New aslLog.Logger(_configPathMapped & "nlog.config", _logPath, My.Application.Info.ProductName, My.Application.Info.Version.ToString)
_logger.SendLog("Started PSALERTS Schedule Server Service", NLog.LogLevel.Info, _serviceTimer, _checkInterval, Nothing)
'Determine if the cloned config files exists in the mapped config file folder
'We clone these files to a writable destination to allow us to overcome write restrictions ot the C: drive on the SPEN PTI Desktop model
If Not System.IO.File.Exists(_configPathMapped & "psaservermachine.config") Then
'Clone the app.config file in the config folder as psaservermachine.config
Utilities.CloneFile(_programFileLocation & "PSALERTSScheduleServer.exe.config", _configPathMapped & "psaservermachine.config")
End If
If Not System.IO.File.Exists(_configPathMapped & "nlog.config") Then
'Clone the nlog.config file
Utilities.CloneFile(_programFileLocation & "PSALERTSScheduleServer.exe.config", _configPathMapped & "nlog.config")
End If
'Determine the Oracle TNS Environment
'Check for the existence of the environment variable 'TNS_ADMIN'
If Environment.GetEnvironmentVariable("TNS_ADMIN") IsNot Nothing Then
'If TNS_ADMIN exists then we can continue with the application session
Else
Dim oraTnsPath As String = ""
'If it doesn't exist then we need to determine the Oracle information from the PATH environment variable
oraTnsPath = GetOraTnsPath()
If oraTnsPath <> "" Then
'Then create the TNS_ADMIN environment variable
Environment.SetEnvironmentVariable("TNS_ADMIN", oraTnsPath)
Else
'If no oracle client information exists then raise an error to this effect and exit the app
'informing the user that they need to install the Oracle client in order to use PSALERTS
Beep()
Throw New PSALERTSOracleConfigException(
"PSALERTS Oracle Configuration Error. PSALERTS Did not find a valid Oracle Client." & vbCrLf & vbCrLf &
"Please install a valid Oracle Client and try again." & vbCrLf & vbCrLf &
"If a valid Oracle Client is installed then ensure that the PATH environment variable contains an entry for the Oracle Client." & vbCrLf & vbCrLf &
"For example - TNS_ADMIN=C:\oracle\12.1.0\Client_lite\NETWORK\ADMIN"
)
End If
End If
'Register the application
If Not Registered() Then
'Register the application
Register()
End If
If Registered() Then
'Clean/close any stray Excel processes from previous debug session
If _debugModeOn Then
CleanUpRedundantProcesses("EXCEL", "PSALERTS")
End If
'instantiate fresh excel session
_myXLApp = New Excel.Application
'Get the timer interval settings
_longInterval = CType(utl.GetSettingServerMachine(_configPath, "appSettings", "LongIntervalMillis"), Integer)
_initInterval = CType(utl.GetSettingServerMachine(_configPath, "appSettings", "InitialIntervalMillis"), Integer)
_refreshInterval = CType(utl.GetSettingServerMachine(_configPath, "appSettings", "InitialIntervalMillis"), Integer)
'Re-start the timer with periodic signalling as per the specified check interval
_serviceTimer.Change(_initInterval, _initInterval)
Else
_started = False
End If
Catch ex As Exception
_logger.SendLog("PSALERTS Schedule Server startup failure.", NLog.LogLevel.Error, ex)
Finally
End Try
End Sub
I use a similar technique for a number of similar services and they are running fine. Would appreciate some insight from any Windows Service gurus out there. Oh, I use WiX to carry out the install, again this is a well worn template for a number of similar such applications.
Kind Regards
Paul J.
Core: The very most typical errors:
Config problems: connection strings, faulty paths, etc...
Boot startup problem (good list - from FAQ)
Wrong password / login account when running as a real user with password.
Files missing or runtimes missing.
Permission problems (ACL / NT Privilege missing).
Maybe check this answer before the below.
UPDATE: Maybe have a look at this previous answer. Service startup timing issue. Also check my ad-hoc answer there in the same page.
Debugger: Other than that - nothing like stepping through the code with a debugger. I haven't done that in a long time. Deploy debug binaries and try? Windows 10 now hides messages from services - not sure how that affects debuggers: No more switching to Session 0.
I am not a service guru, but a deployment specialist. I'll just provide some links and see if that helps. Maybe I have not fully understood the whole problem. I tend to focus on the deployment side and not so much development side.
Ideas List / Debugging Check List: These are "ideas lists" for what could be wrong for applications in general - not just services (two first lists are similar - created some time apart):
Crash on launch
Desktop application won't launch
General purpose WiX / MSI links
Yes, these lists are very generic - too large to digest. Just skim the first two I think.
Debugging Tools: Also a reminder of the most useful service debugging tools: Event Viewer, Task Manager, Services.msc, Process Explorer (system internals), The NET command and SC.exe.
Good Service FAQ: https://www.coretechnologies.com/WindowsServices/FAQ.html
Your startup method should fire up a background worker and quickly return to the SCM that it has started. There is a system wide default setting of 30 seconds but honestly a proper service should respond in a few seconds.
Looking though your code, your connection to the database is probably the long pole causing the problem.

Download many small files with FTP quickly

I have a working code to download many files (hundreds) from an FTP server, but it's very slow and often a timeout error appears.
This is my current way of doing the downloads:
Using ftpClient As New WebClient()
ftpClient.Credentials = New System.Net.NetworkCredential(ftpuser, ftppassword)
For i As Integer = 0 To directoriesDownload.Count - 1
If directoriesDownload(i).Contains(".") Then
If Sync_BackgroundWorker.CancellationPending = True Then
Exit Sub
End If
Dim path As String = "ftp://" & ftpserver & "Datenbank/" + directoriesDownload(i).ToString()
Dim trnsfrpth As String = config.rootpath & "ServerDownload\" + directoriesDownload(i).ToString()
ftpClient.DownloadFile(path, trnsfrpth)
filenametodownload = directoriesDownload(i).ToString()
filesdownloaded += 1
Sync_BackgroundWorker.ReportProgress(filesdownloaded)
End If
Next
ftpClient.Dispose()
End Using
Is there any faster way of downloading hundreds of small files (up to 10 KB) from an FTP server in VB.NET?
It would be the best if there is an option to sign into the FTP only once instead of logging in and out for every file.
I found somebody else having the same problem but without a working result:
Using FTP to download each file *WHILE* getting the file list
I also tried multithreading with a Parallel.For loop, but WebClient does not work with multithreading. Same thing if I try with ftpClient.DownloadFileAsync(New Uri(path), trnsfrpth).
Is there any faster way of downloading hundreds of small files (up to 10 KB) from an FTP server in VB.NET?
...
I also tried multithreading with a Parallel.For loop, but WebClient does not work with multithreading. Same thing if I try with ftpClient.DownloadFileAsync(New Uri(path), trnsfrpth).
Multithreading is the way to go. It's not true that WebClient does not support multithreading. Why wouldn't it?
If you have a problem with implementing multithreaded FTP transfers, you should ask a question about that, rather than asking a question about other (and probably non-existent) ways.
It would be the best if there is an option to sign into the FTP only once instead of logging in and out for every file.
Your code does sign into FTP only once.
See C# - FtpWebRequest - Multiple requests over the same connection/login - What is written there about FtpWebRequest is equally true for WebClient, as WebClient uses FtpWebRequest internally.

SendKeys.SendWait in Visual Basic program not working properly

I am developing my first VB program. The program's main task is to send Skype messages using SendKeys.
I managed to send messages successfully but it is kinda slow as I need to use System.Threading.Thread.Sleep(1000) to properly send messages. If I remove the Sleep(1000) function, the program starts to do weird things like partial messages, some dialogs not showing up and at the end Skype crashed and I couldn't type anything.
Problem: How to remove Sleep(1000) to send messages without such large pause? If I have more than 500 contacts it takes lot of time.
My code:
Dim all() As String = System.IO.File.ReadAllLines(appPath)
Dim message_text As String = TextBox1.Text
If all.Length > 0 Then
For i = 0 To all.Length - 1
System.Threading.Thread.Sleep(1000) '' < ---this line
Skype.Client.OpenMessageDialog(all(i))
Skype.Client.Focus()
SendKeys.SendWait(message_text + " - " + DateTime.Now.ToString())
SendKeys.SendWait("{ENTER}")
System.Threading.Thread.Sleep(100)
Next
MessageBox.Show("Ziņa nosūtīta Skype lietotājiem")
Else
End If
I don't understand why it sometimes sends just partial messages as I am using SendKeys.SendWait that should wait till message text is copied in dialog and then Sending "Enter" key.

Invalid Mail Attachment

I have a windows service that sends out emails by retriving the records from the database. The service account Windows Service is running under, is a Local admin on that machine. The issue i am having is somehow the windows service manages to send out some emails with pdf attachements that exist on the local server where windows service is located and for most of them i get an error message: Invalid Mail Attachment and having real difficulties what could be causing it? any idea on where should i start looking please?
Thanks
Try
' Initialize new mail message class
Dim objEmailMessage As New System.Net.Mail.MailMessage(EmailOutboxEntity.EmailFrom, EmailOutboxEntity.EmailTo)
' Set the email parameters
objEmailMessage.Subject = EmailOutboxEntity.EmailSubject
objEmailMessage.Body = EmailOutboxEntity.EmailBody
' Check if attachments have been specified
If Not EmailOutboxEntity.TestOriginalFieldValueForNull(EmailOutboxFieldIndex.EmailAttachements) Then
' Get an array of attachment file names to send
Dim arrAttachment As String() = Split(EmailOutboxEntity.EmailAttachements, CONST_AttachmentSeparator)
' Step through each filename and attach it to the email
For Each strAttachment As String In arrAttachment
' Attach the current file to the email
objEmailMessage.Attachments.Add(New System.Net.Mail.Attachment(strAttachment))
Next
End If
' Set the SMTP server
Dim mailServer As New System.Net.Mail.SmtpClient(pstrSMTPServer)
' Send the message
mailServer.Send(objEmailMessage)
Catch errException As Exception
' Throw an exception indicating the email failed to send
Throw New EmailOutboxManagerException(errException.Message, "Test Failed to send email, got the following error instead:")
End Try
Sorry Guys, as i have figured out the answer and it was my fault because i was running two different versions of the windows service on different servers and they were both picking up the attachments from the database and the path existed on one server where the files were saved and not on the other server and that's why some of the emails were managed to be sent.

Testing if I have full read/write access to a file in VB .NET 4.5 before processing it

I'm writing a windows service which runs as the local system account. I'm trying to make sure if I have full read/write access to a file beginning to process it any further. Here is my code:
Dim FullPath As String
FullPath = "C:\directory\file.txt"
Dim ps As Security.PermissionSet
ps = New Security.PermissionSet(Security.Permissions.PermissionState.Unrestricted)
ps.AddPermission(New Security.Permissions.FileIOPermission(Security.Permissions.FileIOPermissionAccess.AllAccess, FullPath))
ps.AddPermission(New Security.Permissions.FileIOPermission(Security.Permissions.FileIOPermissionAccess.AllAccess, IO.Path.GetDirectoryName(FullPath)))
Try
ps.Demand()
Catch ex As Security.SecurityException
System.Diagnostics.EventLog.WriteEntry("ShopLink", "File " + FullPath + " will not be parsed. " + ex.Message)
Exit Sub
Catch ex As Exception
System.Diagnostics.EventLog.WriteEntry("ShopLink", "File " + FullPath + " will not be parsed. " + ex.Message)
Exit Sub
End Try
Then I set the full access permissions for the file to "Deny" for the user account my service is running as. After executing, the code above doesn't throw any exceptions and allows file processing to begin. When the service later tries to change and/or delete the file, I get an "Access Denied" exception.
Any suggestions?
For this purpose i use thise small function:
Private Function HasAccess(ByVal ltFullPath As String)
Try
Using inputstreamreader As New StreamReader(ltFullPath)
inputstreamreader.Close()
End Using
Using inputStream As FileStream = File.Open(ltFullPath, FileMode.Open, FileAccess.ReadWrite, FileShare.None)
inputStream.Close()
Return True
End Using
Catch ex As Exception
Return False
End Try
End Function
In your case then:
If HasAccess(FullPath) ...
I have solved the problem by using My.Computer.FileSystem.DeleteFile to delete the file instead of Kill. My.Computer.FileSystem.DeleteFile was executed without problems after successfully demanding full read/write access to the file in the way described above, while Kill consistently threw an "Access denied" exception.
Using "Kill"... I know this is a very old thread but I'll add this in case anyone stumbles on it like I did. I was working on some old VB6 legacy code. One of my clients users was getting a runtime exception during a file open after a kill. The code was "Killing" the file and then rebuilding it from scratch with binary data held in memory. It tuns out that the "Kill" function triggered the user's anti-virus software which locked the file long enough to cause the next "Open" statement to fail. I discovered this using an error logging utility (the name escapes me at the moment). The line in the error log file on the failed "Open" statement was that the file's status was "Delete pending" due to the user's anti-virus software.