I'm using HttpListener as a simple http server. I've been benchmarking req/sec performance by having HttpListener return the string "OK" to each request - this has hit a limit at 16,000 requests per second. Based on the following code, is there anything that I can do to further optimize performance? Or are we hitting the limits of Windows Http.sys?
Image uploading isn't working, here's a link to the Visual Studio performance trace:
VS Performance Trace
Public Class HTTPServer
Shared Listener As HttpListener = New HttpListener
Public Shared Sub Start()
ServicePointManager.DefaultConnectionLimit = 500
ServicePointManager.Expect100Continue = False
ServicePointManager.MaxServicePoints = 500
Listener.Prefixes.Add("http://localhost/")
Listener.Start()
For i As Integer = 1 To (System.Environment.ProcessorCount * 2)
Dim NewThread As New System.Threading.Thread(AddressOf ListenerThread)
NewThread.Priority = ThreadPriority.Normal
NewThread.IsBackground = True
NewThread.Start()
Next
End Sub
Private Shared Sub ListenerThread()
Dim SyncResult As IAsyncResult
While True
SyncResult = Listener.BeginGetContext(New AsyncCallback(AddressOf ListenerCallback), Listener)
SyncResult.AsyncWaitHandle.WaitOne()
End While
End Sub
Private Shared Sub ListenerCallback(ByVal StateObject As IAsyncResult)
Dim Listener As HttpListener = DirectCast(StateObject.AsyncState, HttpListener)
Dim Context As HttpListenerContext = Listener.EndGetContext(StateObject)
Dim Request As HttpListenerRequest = Context.Request
Dim Response As HttpListenerResponse = Context.Response
Dim ResponseString As String = "OK"
Dim Buffer As Byte() = System.Text.Encoding.UTF8.GetBytes(ResponseString)
Response.ContentLength64 = Buffer.Length
Dim OutputStream As System.IO.Stream = Response.OutputStream
OutputStream.Write(Buffer, 0, Buffer.Length)
OutputStream.Close()
OutputStream.Dispose()
End Sub
End Class
Well one thing you can do is only call Encoding.UTF8.GetBytes("OK") once, instead of on every request. I doubt that it'll make much difference, but if you're doing exactly the same thing on every request, you might as well have that response in the form you need it.
I would also use a Using statement rather than calling Close and Dispose explicitly. That's not a performance thing - just general good practice to avoid problems with unclosed streams when there are exceptions.
Realistically, do you need to hit more than 16K QPS? Have you worked out whether you're being CPU-bound or IO-bound? As soon as your program needs to actually do some real work on each request, I would imagine that will dwarf any micro-optimizations you make here anyway.
Related
I am writing a TCP listener (server) that needs to receive messages and send back an acknowledgement. Pretty basic stuff. There are literally dozens of examples out there, including MSDN, from which I copied much of my code. I can receive the message no problem. The problem comes when I try to send back the response. The sending client (Corepoint HL7 engine) reports the following error:
The connection was closed before a response was received
I have tested my service with my own TCP sending test app (written using code copied from MSDN) and it works. But when I receive messages from Corepoint, the response does not go back.
Below is my code. Does anyone have any idea why the NetworkStream.Write method is not actually sending the data (or why the client is not receiving it)? I've tried every idea I've found in other posts that are similar to my problem, and nothing is working. Am I doing something wrong, or is something wrong in the configuration of Corepoint?
Sub Main()
listenThread.Start()
End Sub
Private serverSocket As TcpListener
Dim listenThread As New Thread(New ThreadStart(AddressOf ListenForClients))
Private Sub ListenForClients()
Dim port As Int32 = '(pick a port #)
Dim localIP As IPAddress = 'enter your IP
serverSocket = New TcpListener(localIP, port)
serverSocket.Start()
While True 'blocks until a client has connected to the server
Dim client As TcpClient
If serverSocket.Pending Then
client = serverSocket.AcceptTcpClient
'tried these 2 settings with no effect
'client.NoDelay = True
client.Client.NoDelay = True
ProcessIncomingMessageSocketTCPClient(client) 'I was doing this in a separate thread but temporarily kept it on this thread to eliminate threading as the possible cause (but no luck)
client.Close()
Else
Threading.Thread.Sleep(1000) 'wait 1 second and poll again
End If
End While
End Sub
Private Sub ProcessIncomingMessageSocketTCPClient(ByRef objClient As TcpClient)
Dim strMessageText As String
Dim clientStream As NetworkStream
Dim msgBuffer(4096) As Byte
Dim numberOfBytesRead As Integer
Dim strChunk As String
Dim strCompleteMessage As New Text.StringBuilder
Dim sendBytes As Byte()
clientStream = objClient.GetStream()
Do
numberOfBytesRead = clientStream.Read(msgBuffer, 0, msgBuffer.Length)
strChunk = Encoding.ASCII.GetString(msgBuffer, 0, numberOfBytesRead)
strCompleteMessage.AppendFormat("{0}", strChunk)
Loop While clientStream.DataAvailable
strMessageText = strCompleteMessage.ToString
sendBytes = Encoding.ASCII.GetBytes("I received a message from you")
clientStream.Write(sendBytes, 0, sendBytes.Length)
objClient.Close() 'tried it with and without this line
End Sub
It turns out that nothing is wrong with my code. The TCP was and is working correctly. This application is an HL7 listener and I was missing the MLP wrapping around my ACK. As soon as I added that, the sending application accepted my ACK and all is good.
I’m after some help with a problem i have been working on for the past few days but i can't seem to get it to work correctly.
I have multiple clients connecting to a server, the server needs to keep a list of connected clients and remove them from the list if the client disconnects, I have this bit working correctly.
Once a client is connected the server may call upon the client for some information about its state by sending a few bytes of data to which the client responds with 2 bytes of data. Based on this response the server will complete any one of number of different tasks.
I have had this working synchronously, though now I am trying to make the function asynchronous and I’m running in to an issue. The best I can tell my function that request the information from the client is completing before the client responds and is returning incorrect data, if I print the data received from the client I can see it is the correct information.
This is my 1st time working with asynchronous functions and connections so it’s highly likely I’ve got it completely wrong, I have looked at lots of example code but I can seem to find any that sheads any light on my issue.
This is what I have at the moment:
'create the collection instance to store connected clients
Private clients As New List(Of TcpClient)
'declare a variable to hold the listener instance
Private listener As TcpListener
'declare a variable to hold the cancellation token source instance
Private tokenSource As CancellationTokenSource
Private recvTsk As Task
Private Rreply As New List(Of Byte)
Private Async Sub startTCPListenerServer()
'get port from ini file
Dim netPort = SettingsIniFile.GetString("Setup", "NetworkPort", "")
While True
'create a new cancellation token source instance
tokenSource = New CancellationTokenSource
'create a new listener instance bound to the desired address and port
listener = New TcpListener(IPAddress.Any, netPort)
'start the listener
listener.Start()
While True
Try
Dim client As TcpClient = Await listener.AcceptTcpClientAsync
clients.Add(client)
Dim clientIP As String = client.Client.RemoteEndPoint.ToString
ListBox1.Items.Add(clientIP)
Try
'begin reading from the client's data stream
Using stream As NetworkStream = client.GetStream
Dim buffer(client.ReceiveBufferSize - 1) As Byte
Dim read As Integer = 1
'if read is 0 client has disconnected
While read > 0
recvTsk = New Task(Sub()
For i = 0 To read - 1
'data recived at this point is correct
Rreply.Add(buffer(i))
Next
End Sub, tokenSource.Token)
read = Await stream.ReadAsync(buffer, 0, buffer.Length, tokenSource.Token)
recvTsk.Start()
End While
'client gracefully closed the connection on the remote end
End Using
Catch ocex As OperationCanceledException
'the expected exception if this routines's async method calls honor signaling of the cancelation token
'*NOTE: NetworkStream.ReadAsync() will not honor the cancelation signal
Catch odex As ObjectDisposedException
'server disconnected client while reading
Catch ioex As IOException
'client terminated (remote application terminated without socket close) while reading
Finally
'ensure the client is closed - this is typically a redundant call, but in the
'case of an unhandled exception it may be necessary
'remove the client from the list of connected clients
clients.Remove(client)
client.Close()
ListBox1.Items.Remove(clientIP)
'remove the client's task from the list of running tasks
'clientTasks.Remove(client.Task)
End Try
Catch odex As ObjectDisposedException
'listener stopped, so server is shutting down
Exit While
End Try
End While
For i As Integer = clients.Count - 1 To 0 Step -1
clients(i).Close()
Next
tokenSource.Dispose()
End While
'signal any processing of current clients to cancel (if listening)
tokenSource.Cancel()
'abort the current listening operation/prevent any new connections
listener.Stop()
'End If
End Sub
Async Function sendToPod(message() As Byte, podNum As Integer) As Task(Of Byte)
If clients.Count = 0 Then
Else
Dim podIP As String
'get ip address as string from ini file
podIP = SettingsIniFile.GetString("NetworkSettings", "Pod" & podNum & "IP", "")
Dim currentClient As TcpClient = Nothing
For Each client As TcpClient In clients
Dim clientIP As String = (CType(client.Client.RemoteEndPoint, IPEndPoint).Address.ToString())
If clientIP = podIP Then
currentClient = client
End If
Next
If currentClient IsNot Nothing Then
'get the current client, stream, and data to write
Dim stream As NetworkStream = currentClient.GetStream
Dim buffer() As Byte = message
'wait for the data to be sent to the remote
Await stream.WriteAsync(buffer, 0, buffer.Length)
recvTsk.Wait()
Return Rreply(1)
End If
End If
End Function
Private Async Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Try
Me.TextBox1.Text = Await sendToPod({"&HAB", "&HAD", "&HFF", "&HFF"}, 1)
Catch
End Try
End Sub
End Class
I am working on a program and uploads a shipping manifest to a the shippers website. When I try to upload, I get a nondescript error back from their server, and when checking with the shipper, they tell me that "there is an issue with the SSL" I am using.
I've spent quite a bit of time piecing together code that, from what I seem to understand, is supposed to work, but I'm not making any progress. As far as I know everything else is fine with the upload, but there is a problem with my SSL certificate
If I understand what this code is supposed to do correctly, I should get a certificate from the shippers website, which allows certification to my program for a space of time during which I can upload the data. I'm really not sure that this is what my code is doing at all, but the only code examples I have seen show it something like this.
Here's my code with the URLs changed:
'This references a custom class that compiles the manifest I'm going to upload
Dim StringToUpload As String = Compile_Manifest(MyDate, UseTestDB)
Dim webClient As New System.Net.WebClient
webClient.Credentials = System.Net.CredentialCache.DefaultCredentials
'From what I understand,
'this is supposed to set up properties used in next section of code
System.Net.ServicePointManager.SecurityProtocol = Net.SecurityProtocolType.Ssl3
System.Net.ServicePointManager.ServerCertificateValidationCallback = _
AddressOf AcceptAllCertifications
'I can see that this reaches the server,
'but I don't know how it relates to the next section of code
'that actually uploads the manifest
Dim ServerRequest As System.Net.WebRequest = _
System.Net.WebRequest.Create("https://www.certify.some-shippper.com:443/somefolder")
Dim ServerResponse As System.Net.WebResponse
ServerResponse = ServerRequest.GetResponse()
ServerResponse.Close()
'This code works for the upload of the manifest,
'and it seems the above code is unrelated and does not use a SSL certificate.
'When this code runs I get the same error back from the shippers server,
'indicating an issue with my SSL, with or without the two sections of code above.
Dim StrResult As String = ""
Dim WrappedString As String = TransmitPLD.WrapPldFile(StringToUpload)
'This references a custom class that wraps the data to upload
'in information from the shipper.
Dim ByesToUpload As Byte() = _
System.Web.HttpUtility.UrlEncodeToBytes(WrappedString, _
System.Text.ASCIIEncoding.ASCII)
Dim Result As Byte() = _
webClient.UploadData("https://www.certify.some-shippper.com:443/somefolder", _
ByesToUpload)
StrResult = System.Web.HttpUtility.UrlDecode(Result, _
System.Text.ASCIIEncoding.ASCII)
MessageBox.Show(StrResult)
So it turns out I went about it the wrong way. I needed to upload my data through System.Net.WebRequest and it takes care of the certificates for me. Not implementing all the parts of the code I needed, it didn't handle the retrieval of the shipper's certificate.
In case anyone else gets confused about the matter like I did, here's my working code for anyone to see, adapt and use. My resource for fixing the code (and by that I mean starting from scratch) was the MSDN page for the WebRequest class, and it has code examples much the same as what I have below in C++, C#, and VB.NET and here is the link.
First there are some global variables that need to be set and class that needs to be created for to store the upload response:
' This is set in the function that Upload function
' and uploads the data in the ReadCallback sub
Private Shared WrappedString As String
' This is used to wait for the callback in the Upload function
Private Shared allDone As New Threading.ManualResetEvent(False)
Friend Class RequestState
' This class stores the request state of the request.
Public request As Net.WebRequest
Public Sub New()
request = Nothing
End Sub ' New
End Class ' RequestState
Then there is a sub needed for the upload part web request which will be called further below in the upload function:
Private Shared Sub ReadCallback(asynchronousResult As IAsyncResult)
Try
Dim myRequestState As RequestState = CType(asynchronousResult.AsyncState, RequestState)
Dim myWebRequest As Net.WebRequest = myRequestState.request
' End the request.
Dim streamResponse As IO.Stream = myWebRequest.EndGetRequestStream(asynchronousResult)
' Convert the string into a byte array.
Dim byteArray As Byte() = System.Text.Encoding.ASCII.GetBytes(WrappedString)
' Write the data to the stream.
streamResponse.Write(byteArray, 0, byteArray.Length)
streamResponse.Close()
' Allow the main thread to resume.
allDone.Set()
Catch ex As Exception
Throw New Exception("Error in " & Reflection.MethodBase.GetCurrentMethod.Name.ToString & " **" & ex.Message, ex)
End Try
End Sub ' ReadCallback
Finally, this is the function that should be called to upload the data, which uses all the code above:
Public Shared Function Upload(ByVal MyDate As Date) As String
Dim StrResult As String = ""
UploadSucess = False
Try
' This is my code that builds the manifest that I want to upload
Dim StringToUpload As String = Compile_PLD200(MyDate)
WrappedString = TransmitPLD.WrapPldFile(StringToUpload)
Dim myWebRequest As Net.WebRequest
myWebRequest = Net.WebRequest.Create("https://www.some.website.com:443/someplace")
' Create an instance of the RequestState and assign
' myWebRequest to it's request field.
Dim myRequestState As New RequestState()
myRequestState.request = myWebRequest
myWebRequest.ContentType = "multipart/mixed; boundary=BOUNDARY"
myRequestState.request.Method = "POST"
' Start the asynchronous 'BeginGetRequestStream' method call.
Dim r As IAsyncResult = CType(myWebRequest.BeginGetRequestStream(AddressOf ReadCallback, myRequestState), IAsyncResult)
' Pause the current thread until the async operation completes.
allDone.WaitOne()
' Send the Post and get the response.
Dim myWebResponse As Net.WebResponse = myWebRequest.GetResponse()
Dim streamResponse As IO.Stream = myWebResponse.GetResponseStream()
Dim streamRead As New IO.StreamReader(streamResponse)
Dim readBuff(256) As [Char]
Dim count As Integer = streamRead.Read(readBuff, 0, 256)
While count > 0
Dim outputData As New [String](readBuff, 0, count)
Console.WriteLine(outputData)
count = streamRead.Read(readBuff, 0, 256)
StrResult += outputData
End While
' Close the Stream Object.
streamResponse.Close()
streamRead.Close()
' Release the HttpWebResponse Resource.
myWebResponse.Close()
Catch ex As Exception
Throw New Exception("Error in " & Reflection.MethodBase.GetCurrentMethod.Name.ToString & " **" & ex.Message, ex)
End Try
Return StrResult
End Function ' Upload
Again here is the MSDN page for the WebRequest class which has a code example too.
Hope this helps anyone who was stuck like I was. And any criticisms as to the implementation of the code are welcome. This just happen to do what I want, I can't say it is the most efficient implementation.
I am making a program that must process about 5000 strings as quickly as possible. about 2000 of these strings must be translated via a webrequest to mymemory.translated.net. (see code below, JSON part removed since not needed here)
Try
url = "http://api.mymemory.translated.net/get?q=" & Firstpart & "!&langpair=de|it&de=somemail#christmas.com"
request = DirectCast(WebRequest.Create(url), HttpWebRequest)
response = DirectCast(request.GetResponse(), HttpWebResponse)
myreader = New StreamReader(response.GetResponseStream())
Dim rawresp As String
rawresp = myreader.ReadToEnd()
Debug.WriteLine("Raw:" & rawresp)
Catch ex As Exception
MessageBox.Show(ex.ToString)
End Try
the code itself is working fine, problem is it is a blocking code and needs about 1 second per string. Thats more then half an hour for all my strings. i would need to convert this code to a non blocking one and make multiple calls on the same time. Could somebody please tell me how i could do that? I was thinking of a background worker but that wouldnt speed things up.. it would just execute the code on a different thread...
thanks!
The problem is you aren't just being held back by the maximum number of concurrent operations. HttpWebRequests are throttled by nature (I believe the default policy allows only 2 at any given time), so you have to override that behaviour too. Please refer to the code below.
Imports System.Diagnostics
Imports System.IO
Imports System.Net
Imports System.Threading
Imports System.Threading.Tasks
Public Class Form1
''' <summary>
''' Test entry point.
''' </summary>
Private Sub Form1_Load() Handles MyBase.Load
' Generate enough words for us to test thoroughput.
Dim words = Enumerable.Range(1, 100) _
.Select(Function(i) "Word" + i.ToString()) _
.ToArray()
' Maximum theoretical number of concurrent requests.
Dim maxDegreeOfParallelism = 24
Dim sw = Stopwatch.StartNew()
' Capture information regarding current SynchronizationContext
' so that we can perform thread marshalling later on.
Dim uiScheduler = TaskScheduler.FromCurrentSynchronizationContext()
Dim uiFactory = New TaskFactory(uiScheduler)
Dim transformTask = Task.Factory.StartNew(
Sub()
' Apply the transformation in parallel.
' Parallel.ForEach implements clever load
' balancing, so, since each request won't
' be doing much CPU work, it will spawn
' many parallel streams - likely more than
' the number of CPUs available.
Parallel.ForEach(words, New ParallelOptions With {.MaxDegreeOfParallelism = maxDegreeOfParallelism},
Sub(word)
' We are running on a thread pool thread now.
' Be careful not to access any UI until we hit
' uiFactory.StartNew(...)
' Perform transformation.
Dim url = "http://api.mymemory.translated.net/get?q=" & word & "!&langpair=de|it&de=somemail#christmas.com"
Dim request = DirectCast(WebRequest.Create(url), HttpWebRequest)
' Note that unless you specify this explicitly,
' the framework will use the default and you
' will be limited to 2 parallel requests
' regardless of how many threads you spawn.
request.ServicePoint.ConnectionLimit = maxDegreeOfParallelism
Using response = DirectCast(request.GetResponse(), HttpWebResponse)
Using myreader As New StreamReader(response.GetResponseStream())
Dim rawresp = myreader.ReadToEnd()
Debug.WriteLine("Raw:" & rawresp)
' Transform the raw response here.
Dim processed = rawresp
uiFactory.StartNew(
Sub()
' This is running on the UI thread,
' so we can access the controls,
' i.e. add the processed result
' to the data grid.
Me.Text = processed
End Sub, TaskCreationOptions.PreferFairness)
End Using
End Using
End Sub)
End Sub)
transformTask.ContinueWith(
Sub(t As Task)
' Always stop the stopwatch.
sw.Stop()
' Again, we are back on the UI thread, so we
' could access UI controls if we needed to.
If t.Status = TaskStatus.Faulted Then
Debug.Print("The transformation errored: {0}", t.Exception)
Else
Debug.Print("Operation completed in {0} s.", sw.ElapsedMilliseconds / 1000)
End If
End Sub,
uiScheduler)
End Sub
End Class
If you want to send 10 parallel requests, you must create 10 BackgroundWorkers. Or manually create 10 threads. Then iterate, and whenever a worker/thread is done, give it a new task.
I do not recommend firing 5000 parallel threads/workers, you must be careful:
A load like that could be interpreted as spamming or an attack by the server. Don't overdo it, maybe talk to translated.net and ask them about the workload they accept.
Also think about what your machine and your internet upstream can handle.
I would create a Task for every request, so you can have a Callback for every call using ContinueWith:
For Each InputString As String In myCollectionString
Tasks.Task(Of String).Factory.StartNew(Function(inputString)
Dim request As HttpWebRequest
Dim myreader As StreamReader
Dim response As HttpWebResponse
Dim rawResp As String = String.Empty
Try
Dim url As String = "http://api.mymemory.translated.net/get?q=" & inputString & "!&langpair=de|it&de=somemail#christmas.com"
request = DirectCast(WebRequest.Create(url), HttpWebRequest)
response = DirectCast(request.GetResponse(), HttpWebResponse)
myreader = New StreamReader(response.GetResponseStream())
rawResp = myreader.ReadToEnd()
Debug.WriteLine("Raw:" & rawResp)
Catch ex As Exception
MessageBox.Show(ex.ToString)
End Try
Return rawResp
End Function, CancellationToken.None, _
Tasks.TaskCreationOptions.None).ContinueWith _
(Sub(task As Tasks.Task(Of String))
'Dom something with result
Console.WriteLine(task.Result)
End Sub)
Next
Here is vb.net 2008 code is:
Public Function CheckURL(ByVal URL As String) As Boolean
Try
Dim Response As Net.WebResponse = Nothing
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Response = WebReq.GetResponse
Response.Close()
Return True
Catch ex As Exception
End Try
End Function
when a url is processing in checking it hangs my application for a while. Is this possible it checks smoothly all url list without hanging my application..
Is there any other fastest way to check urls?
Note: I have about 800 urls in file to check all links a valid by website responce or not.
If an exception occurs, the WebResponse object isn't properly disposed of. This can lead to your app running out of connections. Something like this will work better:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
This using the Using keyword ensures that the response is closed and finalized whenever that block exits.
If it's the server itself that's taking awhile to respond, look into the BeginGetResponse method on the HttpWebRequest. Check MSDN for a sample on how to use it. But be warned, that way also lies madness if you are not careful.
The answer is two fold:
Most of the waiting time is due to downloading content you don't need. If you request to only return the header, you will receive substantially less data, which will make your process faster.
As Matt identified, you aren't disposing of your connections, which may slow your process.
Expanding on Matt's answer, do the following:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
WebReq.Method = "HEAD" 'This is the important line.
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
GetResponse delivers you the whole content to your request. If this is what you want, there's not many room to speed up the request on the client side, since it mostly depends on the URLs server how fast to reply and how much data will be send over. If you just want to check if the URL is valid (or responding at all), it might be better to just ping it.
Keep in mind GetResponse isn't disposed when it runs into an error, so use the code posted by Matt to avoid this.
For your other problem, hanging application, you might avoid this be running this code as a thread.
This works basically like this (from here):
rem at the top of the code
Imports System.Threading
...
rem your event handler, p.e. button click or whatever
trd = New Thread(AddressOf ThreadTask)
trd.IsBackground = True
trd.Start()
rem your code
Private Sub ThreadTask()
dim i as long
Do
i += 1
Thread.Sleep(100)
Loop
End Sub