How to Host a webpage on my Localhost and intercept responses in VB.NET - vb.net

So at the moment I am hosting some HTML code on my localhost using a httplistener in VB.NET. Here is the code for that:
Shared httpListener As HttpListener = New HttpListener()
Private Shared Sub Main()
Debug.WriteLine("Starting server...")
httpListener.Prefixes.Add("localhost:5000/")
httpListener.Start()
Debug.WriteLine("Server started.")
Dim _responseThread As Thread = New Thread(AddressOf ResponseThread)
_responseThread.Start()
End Sub
Private Shared Sub ResponseThread()
While True
Dim context As HttpListenerContext = httpListener.GetContext()
Dim responseArray As Byte() = Encoding.UTF8.GetBytes("<html><script src=""https://www.google.com/recaptcha/api.js"">async defer</script><div class=""g-recaptcha"" data-sitekey=""sitekey""></div></html>")
context.Response.OutputStream.Write(responseArray, 0, responseArray.Length)
context.Response.KeepAlive = False
context.Response.Close()
Debug.WriteLine("Respone given to a request.")
End While
End Sub
Now this works perfectly and notifies me whenever a request is made to the localhost, but I want to be able to receive response data when a user (me) does the recaptcha. I am not sure how to do this and have looked everywhere. So any help is appreciated, I tried seeing if I could get responses using the httplistenerresponse but had no look and even looked into TCP clients and listeners. There is probably an easier way to do this, and any help is appreciated.
Fiddler is a prime example of what I need my program to do, but just listen to the response after completing the captcha.
Thanks!

Related

TCP Listener connection is closed before acknowledgement/response can be sent

I am writing a TCP listener (server) that needs to receive messages and send back an acknowledgement. Pretty basic stuff. There are literally dozens of examples out there, including MSDN, from which I copied much of my code. I can receive the message no problem. The problem comes when I try to send back the response. The sending client (Corepoint HL7 engine) reports the following error:
The connection was closed before a response was received
I have tested my service with my own TCP sending test app (written using code copied from MSDN) and it works. But when I receive messages from Corepoint, the response does not go back.
Below is my code. Does anyone have any idea why the NetworkStream.Write method is not actually sending the data (or why the client is not receiving it)? I've tried every idea I've found in other posts that are similar to my problem, and nothing is working. Am I doing something wrong, or is something wrong in the configuration of Corepoint?
Sub Main()
listenThread.Start()
End Sub
Private serverSocket As TcpListener
Dim listenThread As New Thread(New ThreadStart(AddressOf ListenForClients))
Private Sub ListenForClients()
Dim port As Int32 = '(pick a port #)
Dim localIP As IPAddress = 'enter your IP
serverSocket = New TcpListener(localIP, port)
serverSocket.Start()
While True 'blocks until a client has connected to the server
Dim client As TcpClient
If serverSocket.Pending Then
client = serverSocket.AcceptTcpClient
'tried these 2 settings with no effect
'client.NoDelay = True
client.Client.NoDelay = True
ProcessIncomingMessageSocketTCPClient(client) 'I was doing this in a separate thread but temporarily kept it on this thread to eliminate threading as the possible cause (but no luck)
client.Close()
Else
Threading.Thread.Sleep(1000) 'wait 1 second and poll again
End If
End While
End Sub
Private Sub ProcessIncomingMessageSocketTCPClient(ByRef objClient As TcpClient)
Dim strMessageText As String
Dim clientStream As NetworkStream
Dim msgBuffer(4096) As Byte
Dim numberOfBytesRead As Integer
Dim strChunk As String
Dim strCompleteMessage As New Text.StringBuilder
Dim sendBytes As Byte()
clientStream = objClient.GetStream()
Do
numberOfBytesRead = clientStream.Read(msgBuffer, 0, msgBuffer.Length)
strChunk = Encoding.ASCII.GetString(msgBuffer, 0, numberOfBytesRead)
strCompleteMessage.AppendFormat("{0}", strChunk)
Loop While clientStream.DataAvailable
strMessageText = strCompleteMessage.ToString
sendBytes = Encoding.ASCII.GetBytes("I received a message from you")
clientStream.Write(sendBytes, 0, sendBytes.Length)
objClient.Close() 'tried it with and without this line
End Sub
It turns out that nothing is wrong with my code. The TCP was and is working correctly. This application is an HL7 listener and I was missing the MLP wrapping around my ACK. As soon as I added that, the sending application accepted my ACK and all is good.

EWS Connection to Office365 fails - 401 Unauthorized

I need to make a VB .net tool to read email and save attachments. My company recently migrated from on-premise Exchange to Office 365. I have been reading EWS tutorials for 2 days now, and searching StackOverflow, but cannot get past the most basic step of authorized access to the O365 mailbox.
I took the original code from this article: htp://www.c-sharpcorner.com/UploadFile/jj12345678910/reading-email-and-attachment-from-microsoft-exchange-server/. I had some trouble converting it to VB using the Telerik converter but I think I have it right. Each time I try using the FindItemsResults method it halts with "(401) Unauthorized."
The instructions for asking a question state that I should include links to what I have already found and why it did not work, but my SO reputation only allows me 2 links. Here is what I have tried:
I have tried every possible usercode and domain combination I can think after reading this page: htps://stackoverflow.com/questions/10107872/ews-connections-issues-401-unauthorized
I am trying to read my own mailbox, so this one does not help: htps://stackoverflow.com/questions/43346498/401-unauthorized-access-when-using-ews-to-connect-to-mailbox
My connection looks identical to the connection on this page, but using it in my project did not get past the same Unauthorized error as before:htps://stackoverflow.com/questions/29009295/ews-managed-api-retrieving-e-mails-from-office365-exchange-server
Here is my code:
Public Class Form1
Public Exchange As ExchangeService
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
lstMsg.Clear()
lstMsg.View = View.Details
lstMsg.Columns.Add("Date", 150)
lstMsg.Columns.Add("From", 250)
lstMsg.Columns.Add("Subject", 400)
lstMsg.Columns.Add("Has Attachment", 50)
lstMsg.Columns.Add("Id", 100)
lstMsg.FullRowSelect = True
End Sub
Public Sub ConnectToExchangeServer()
Me.lblMsg.Text = "Connecting to Exchange Server"
lblMsg.Refresh()
Try
'Exchange = New ExchangeService(ExchangeVersion.Exchange2013)
Exchange = New ExchangeService()
Exchange.TraceEnabled = True
'Exchange.UseDefaultCredentials = True
Exchange.Credentials = New WebCredentials("DoeJohn", "mypasswd", "mycorp.com")
'Exchange.AutodiscoverUrl("DoeJohn#mycorp.mail.onmicrosoft.com", AddressOf MyRedirectionURLValidationCallback)
'Exchange.AutodiscoverUrl("John.Doe#mycorp.com", AddressOf MyRedirectionURLValidationCallback)
Exchange.Url = New System.Uri("https://outlook.office365.com/ews/exchange.asmx")
lblMsg.Text = "Connected to Exchange Server"
lblMsg.Refresh()
Catch ex As Exception
MsgBox("Fatal Error in Connect: " & ex.Message)
End
End Try
End Sub
Public Function MyRedirectionURLValidationCallback(RedirectionURL As String) As Boolean
Dim Result As Boolean = False
Dim RedirectionURI As Uri = New Uri(RedirectionURL)
If RedirectionURI.Scheme = "https" Then
Return True
End If
Return False
End Function
Private Sub btnRead_Click(sender As Object, e As EventArgs) Handles btnRead.Click
Call ConnectToExchangeServer()
Dim ts As TimeSpan = New TimeSpan(0, -1, 0, 0)
Dim MyDate As DateTime = DateTime.Now.Add(ts)
Dim MyFilter As SearchFilter.IsGreaterThanOrEqualTo = New SearchFilter.IsGreaterThanOrEqualTo(ItemSchema.DateTimeReceived, MyDate)
If Exchange IsNot Nothing Then
Dim FindResults As FindItemsResults(Of Item) =
Exchange.FindItems(WellKnownFolderName.Inbox, MyFilter, New ItemView(50))
Dim NewRow As ListViewItem
For Each MyItem As Item In FindResults
Dim Message As EmailMessage = EmailMessage.Bind(Exchange, MyItem.Id)
NewRow = New ListViewItem(Message.DateTimeReceived.ToString())
NewRow.SubItems.Add(Message.From.Name.ToString())
NewRow.SubItems.Add(Message.Subject)
NewRow.SubItems.Add(Message.HasAttachments.ToString())
NewRow.SubItems.Add(Message.Id.ToString())
lstMsg.Items.Add(NewRow)
Next
Else
End If
End Sub
I have confirmed that AutoDiscover is correctly finding the server, comparing to Test Email AutoConfiguration in Outlook.
AutoConfig
An interesting sidenote -- after my company moved to Office 365, I noticed that I have two new SMTP mail addresses. If I open Outlook properties on myself, I see this:
Props
This means someone can send mail to me now either at the old address john.doe#mycorp.com, and also now at doejohn#mycorp.mail.onmicrosoft.com. The new address is based on my domain usercode. I tested the microsoft one from a gmail account and it works.
To sum up, here are my questions:
1. Why am I getting the (401) Unauthorized errors when I try to read my Inbox?
2. Does Office 365 expect me to use my domain account or my mailbox name for user credentials?
3. In the domain part of the WebCredentials statement, do I use my company's mycorp.com or instead do I use Office 365's domain outlook.office365.com?
If you have read this far, many thanks!
I found the answer to the problem above, so I'll share here in case anyone needs it. The problem is down to security protocols added by my IT organization that were not documented because of recent phishing attacks on our company.
The first clue was that IT stated if I want to check company email on my personal 4G device like smartphone or ipad, I must also install Microsoft InTune for MFA connection to the O365 mail server. Since I am unaware of how to integrate InTune into my .Net app, I had to look for another answer.
I requested and was approved to move my functional mailbox from O365 to an on-premise Exchange server for this app. That solved the authentication problem.

"KeepAlive" for TCPClient connection to keep connection open?

Is there a way to keep a TcpClient connection open constantly? I have an application which lets our users scan a carton, some database updates are performed, and a shipping label is sent to and printed from a wireless hip printer (model of printer is Zebra QLn420) which the user is using.
The application attempts to keep a connection to the wireless printer via the TcpClient connection, and multiple checks are made throughout processing to make sure the connection is good, before sending a generated ZPL to the printer for printing.
We have been having an issue with an occasional label missing, and it seems to be whenever the user stops scanning for a few minutes, then resumes. However, it is a semi-rare occurrence when a label is skipped, and as such is rather hard to reproduce (I haven't been able to replicate it myself, but I have seen it happen out in the warehouse).
I would like to know either if there is a way to make sure that the connection is always open (by "pinging" the device every so often), or if there is a way to get feedback that the data has been received and printed.
This is the code I'm calling to ensure a connection:
Public Function Connect(strIP As String, intPort As Integer) As Boolean
Try
'connect to printer via TcpClient, need ip address and port number
'connects without thread, hangs program for 10-20 seconds if printer is not turned on, replaced with code below to thread the connection and set timeout
For i As Integer = 1 To 2
If Not (client IsNot Nothing AndAlso client.Connected) Then
'uses ClientSocketParameters structure to pass to recursive function ConnectionReturned()
clntSockParams = New ClientSocketParameters
clntSockParams.addrs = strIP
clntSockParams.prt = intPort
'create client and call BeginConnect (attempts to connect on separate thread until TimeoutTime has elapsed)
client = New System.Net.Sockets.TcpClient
client.SendTimeout = 5000
client.ReceiveTimeout = 5000
'setup timer with timeout length and start, if timer goes past intTimeoutLength, the Timeout() function is called which closes everything and leaves client = Nothing
AddHandler TimeoutTime.Elapsed, AddressOf Timeout
TimeoutTime.Interval = intTimeoutLength
TimeoutTime.Start()
client.BeginConnect(strIP, intPort, New AsyncCallback(AddressOf ConnectionReturned), clntSockParams)
'keeps the program from doing anything else until BeginConnect either succeeds or fails (due to connect on separate thread)
Do While TimeoutTime.Enabled
System.Threading.Thread.Sleep(500)
Loop
End If
'if TimeoutTime is elapsed and client is Nothing, connection didn't happen, throw an error
If client Is Nothing Then
blnConnected = False
Else
blnConnected = True
Exit For
End If
Next
Catch ex As Exception
blnConnected = False
End Try
Return blnConnected
End Function
Private Sub ConnectionReturned(ByVal ar As System.IAsyncResult)
'this method is called from the client.BeginConnect line in Connect(), make sure timer is running
If TimeoutTime.Enabled Then
'ensure client is initialized
If client Is Nothing Then client = New System.Net.Sockets.TcpClient
'keep calling ConnectionReturned until client.Connected is true
If client.Connected Then
TimeoutTime.Stop()
Else
Dim actualParameters As ClientSocketParameters = DirectCast(ar.AsyncState, ClientSocketParameters)
client.BeginConnect(actualParameters.addrs, actualParameters.prt, New AsyncCallback(AddressOf ConnectionReturned), clntSockParams)
End If
End If
End Sub
Private Sub Timeout(ByVal sender As Object, ByVal e As EventArgs)
'this method is only called if TimeoutTime elapsed, which means no connection was made. close the client object if needed, set to Nothing, and stop TimeoutTime
If TimeoutTime.Enabled Then
Try
client.Close()
Catch ex As Exception
End Try
client = Nothing
TimeoutTime.Stop()
End If
End Sub
According to this question:
tcp client in vb.net not receiving the entire data response data from server
TcpClient is not always guaranteed to deliver all data to the other end of a connection, so if a more reliable connection method is available, that would be worth a try as well.
Please let me know if more information is needed. Thank you!
Originally I got the code for connecting via this link. I've modified it since because it would hang the application for 10-20 seconds if it took longer to connect. Code here is in C# and I translated to VB:
Send ZPL Commands via TCP/IP in C#
This is the link to the docs for the class:
TcpClient Class
Zebra printers have a timeout setting on TCP that I think has a 3 or 5 minute default. The first thing to do is to turn that timeout off. There will still be other reasons that the printer would disconnect so you will need to handle that as well.
embed this into your program:
! U1 setvar "wlan.ip.timeout.enable" "off"
Make sure you send a CR/LF before and after that line.
if you send a query after your format you can know that the whole format made it to the printer. Something like the following would work:
! U1 getvar "device.uptime"
That's not a sane solution. A sane solution is this: If a label isn't completely sent to the printer because the connection failed, make a new connection and send the label.

My url checker function is hanging application in vb.net

Here is vb.net 2008 code is:
Public Function CheckURL(ByVal URL As String) As Boolean
Try
Dim Response As Net.WebResponse = Nothing
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Response = WebReq.GetResponse
Response.Close()
Return True
Catch ex As Exception
End Try
End Function
when a url is processing in checking it hangs my application for a while. Is this possible it checks smoothly all url list without hanging my application..
Is there any other fastest way to check urls?
Note: I have about 800 urls in file to check all links a valid by website responce or not.
If an exception occurs, the WebResponse object isn't properly disposed of. This can lead to your app running out of connections. Something like this will work better:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
This using the Using keyword ensures that the response is closed and finalized whenever that block exits.
If it's the server itself that's taking awhile to respond, look into the BeginGetResponse method on the HttpWebRequest. Check MSDN for a sample on how to use it. But be warned, that way also lies madness if you are not careful.
The answer is two fold:
Most of the waiting time is due to downloading content you don't need. If you request to only return the header, you will receive substantially less data, which will make your process faster.
As Matt identified, you aren't disposing of your connections, which may slow your process.
Expanding on Matt's answer, do the following:
Try
Dim WebReq As Net.HttpWebRequest = Net.HttpWebRequest.Create(URL)
WebReq.Method = "HEAD" 'This is the important line.
Using Response = WebReq.GetResponse()
Return True
End Using
Catch ex as WebException
Return False
End Try
GetResponse delivers you the whole content to your request. If this is what you want, there's not many room to speed up the request on the client side, since it mostly depends on the URLs server how fast to reply and how much data will be send over. If you just want to check if the URL is valid (or responding at all), it might be better to just ping it.
Keep in mind GetResponse isn't disposed when it runs into an error, so use the code posted by Matt to avoid this.
For your other problem, hanging application, you might avoid this be running this code as a thread.
This works basically like this (from here):
rem at the top of the code
Imports System.Threading
...
rem your event handler, p.e. button click or whatever
trd = New Thread(AddressOf ThreadTask)
trd.IsBackground = True
trd.Start()
rem your code
Private Sub ThreadTask()
dim i as long
Do
i += 1
Thread.Sleep(100)
Loop
End Sub

HttpListener Performance Optimization

I'm using HttpListener as a simple http server. I've been benchmarking req/sec performance by having HttpListener return the string "OK" to each request - this has hit a limit at 16,000 requests per second. Based on the following code, is there anything that I can do to further optimize performance? Or are we hitting the limits of Windows Http.sys?
Image uploading isn't working, here's a link to the Visual Studio performance trace:
VS Performance Trace
Public Class HTTPServer
Shared Listener As HttpListener = New HttpListener
Public Shared Sub Start()
ServicePointManager.DefaultConnectionLimit = 500
ServicePointManager.Expect100Continue = False
ServicePointManager.MaxServicePoints = 500
Listener.Prefixes.Add("http://localhost/")
Listener.Start()
For i As Integer = 1 To (System.Environment.ProcessorCount * 2)
Dim NewThread As New System.Threading.Thread(AddressOf ListenerThread)
NewThread.Priority = ThreadPriority.Normal
NewThread.IsBackground = True
NewThread.Start()
Next
End Sub
Private Shared Sub ListenerThread()
Dim SyncResult As IAsyncResult
While True
SyncResult = Listener.BeginGetContext(New AsyncCallback(AddressOf ListenerCallback), Listener)
SyncResult.AsyncWaitHandle.WaitOne()
End While
End Sub
Private Shared Sub ListenerCallback(ByVal StateObject As IAsyncResult)
Dim Listener As HttpListener = DirectCast(StateObject.AsyncState, HttpListener)
Dim Context As HttpListenerContext = Listener.EndGetContext(StateObject)
Dim Request As HttpListenerRequest = Context.Request
Dim Response As HttpListenerResponse = Context.Response
Dim ResponseString As String = "OK"
Dim Buffer As Byte() = System.Text.Encoding.UTF8.GetBytes(ResponseString)
Response.ContentLength64 = Buffer.Length
Dim OutputStream As System.IO.Stream = Response.OutputStream
OutputStream.Write(Buffer, 0, Buffer.Length)
OutputStream.Close()
OutputStream.Dispose()
End Sub
End Class
Well one thing you can do is only call Encoding.UTF8.GetBytes("OK") once, instead of on every request. I doubt that it'll make much difference, but if you're doing exactly the same thing on every request, you might as well have that response in the form you need it.
I would also use a Using statement rather than calling Close and Dispose explicitly. That's not a performance thing - just general good practice to avoid problems with unclosed streams when there are exceptions.
Realistically, do you need to hit more than 16K QPS? Have you worked out whether you're being CPU-bound or IO-bound? As soon as your program needs to actually do some real work on each request, I would imagine that will dwarf any micro-optimizations you make here anyway.