I need to capture the response content after the request is sent from webBrowser control.
When I have httpAnalyzer running, I am able to capture some important data (raw data) in response content that is passed back to webBrowser control.
Any ideas?
More info:
I have an internal site which uses flash to generate reports. I was able to automate page loading and simulate button clicking that produce the report. When I use the httpAnalyzer, I am able to see raw data that I need to capture; but I don't know how to actually get to response object.
I used FiddlerCore: http://fiddler2.com/fiddlercore
private void button_click()
{
Thread myThread = new Thread(delegate()
{
fiddlerRun();
});
myThread.Start();
}
private void fiddlerRun()
{
#region AttachEventListeners
Fiddler.FiddlerApplication.BeforeResponse += delegate(Fiddler.Session oS)
{
String s1 = oS.GetResponseBodyAsString(); // this is where response is catched
//I know that response must contain specific word - if it is there- catch it!
if (s1.Contains("License") )
{
//do my stuff with response
//...
//stop Fiddler
Fiddler.FiddlerApplication.Shutdown();
System.Threading.Thread.Sleep(750);
}
};
#endregion AttachEventListeners
System.Diagnostics.Debug.WriteLine("Starting FiddlerCore...");
Fiddler.CONFIG.IgnoreServerCertErrors = false;
try
{
Fiddler.FiddlerApplication.Startup(8877, true, true);
}
catch { }
}
Related
I'm trying to write an infinite length response body and detect when a client disconnects so I can stop writing. I'm used to getting socket exceptions or similar when a client closes the connection but that doesn't seem to be happening when writing directly to Response.Body. I can close the client applications and the server side just keeps on writing. I've included the relevant code below. It's entirely possible there is a better way to do it but this came to mind. Basically I have a live video feed which should go on forever. I'm writing to ResponseBody as chunked content (No content length, flushing after each video frame). The video frames are received via an event callback from elsewhere in the program so I'm subscribing to the events in the controller method and then forcing it to stay open with the await Task.Delay loop so the Response stream isn't closed. The callback for H264PacketReceived is formatting the data as a streaming mp4 file and writing it to the Response Stream. This all seems to work fine, I can play the live stream with ffmpeg or chrome, but when I close the client application I don't get an exception or anything. It just keeps writing to the stream without any errors.
public class LiveController : ControllerBase
{
[HttpGet]
[Route("/live/{cameraId}/{stream}.mp4")]
public async Task GetLiveMP4(Guid cameraId, int stream)
{
try
{
Response.StatusCode = 200;
Response.ContentType = "video/mp4";
Response.Headers.Add("Cache-Control", "no-store");
Response.Headers.Add("Connection", "close");
ms = Response.Body;
lock (TCPVideoReceiver.CameraStreams)
{
TCPVideoReceiver.CameraStreams.TryGetValue(cameraId, out cameraStream);
}
if (this.PacketStream == null)
{
throw new KeyNotFoundException($"Stream {cameraId}_{stream} not found");
}
else
{
connected = true;
this.PacketStream.H264PacketReceived += DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed += PacketStream_StreamClosed;
}
while(connected)
{
await Task.Delay(1000);
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
connected = false;
this.PacketStream.H264PacketReceived -= DefaultStream_H264PacketReceived;
this.PacketStream.StreamClosed -= PacketStream_StreamClosed;
}
}
private bool connected = false;
private PacketStream PacketStream;
private Mp4File mp4File;
private Stream ms;
private async void PacketStream_StreamClosed(PacketStream source)
{
await Task.Run(() =>
{
try
{
Console.WriteLine($"Closing live stream");
connected = false;
ms.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
});
}
private async void DefaultStream_H264PacketReceived(PacketStream source, H264Packet packet)
{
try
{
if (mp4File == null && packet.IsIFrame)
{
mp4File = new Mp4File(null, packet.sps, packet.pps);
var _p = mp4File.WriteHeader(0);
await ms.WriteAsync(mp4File.buffer, 0, _p);
}
if (mp4File != null)
{
var _p = mp4File.WriteFrame(packet, 0);
var start = mp4File._moofScratchIndex - _p;
if (_p > 0)
{
await ms.WriteAsync(mp4File._moofScratch, start, _p);
await ms.FlushAsync();
}
}
return;
}
catch (Exception ex)
{
connected = false;
Console.WriteLine(ex.ToString());
}
}
Answering my own question.
When the client disconnects mvc core sets the cancellation token HttpContext.RequestAborted
By monitoring and/or using that cancellation token you can detect a disconnect and clean everything up.
That said, the entire design can be improved by creating a custom stream which encapsulates the event handling (producer/consumer). Then the controller action can be reduced to.
return File(new MyCustomStream(cameraId, stream), "video/mp4");
The File Method already monitors the cancellation token and everything works as you'd expect.
I am encountering the same error described in Selenium Error - The HTTP request to the remote WebDriver timed out after 60 seconds.
In order to better understand the problem, I need to create a minimal sample that reproduces the bug - an html page and a console app that uses Selenium to open it.
My question is: How exactly can I reproduce the bug, i.e. create an experimental program that intentionally triggers this bug?
Edit: In case it helps, according to IEDriver. Download. HTTP request to the remote WebDriver server timed out after 60 seconds:
The problem here is that when IE is in the process of downloading a file, the readyState of the browser never moves from interactive to complete
You could try to add a web page which includes a button control, in the button click event, you can call a web API to get data. In the web API method, add Thread. Sleep () method to stop the executing thread for a given amount of time (more than the request time). Then, if you trigger the button click event using Selenium WebDriver, it will show this error.
Code like this:
Code in mvc view:
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.0/jquery.min.js"></script>
<script>
$(function () {
$("#buttonSearchPro").click(function () {
$.ajax({
url: "#Url.Action("GetData", "Home")",
async: false,
success: function (data) {
alert(data);
}
});;
});
});
</script>
<input type="button" id="buttonSearchPro" class="btn btnAction" value="Download" />
Code in MVC controller:
public ActionResult GetData()
{
Thread.Sleep(70000000);
return Json("OK", JsonRequestBehavior.AllowGet);
}
Code in console application:
private const string URL = #"http://localhost:65330/Home/Index";
private const string IE_DRIVER_PATH = #"D:\Downloads\webdriver\IEDriverServer_x64_3.14.0";
static void Main(string[] args)
{
//EdgeWebDriver();
InternetExplorerTest();
}
public static void InternetExplorerTest()
{
try{
var options = new InternetExplorerOptions()
{
InitialBrowserUrl = URL,
IntroduceInstabilityByIgnoringProtectedModeSettings = true
};
var driver = new InternetExplorerDriver(IE_DRIVER_PATH, options);
driver.Navigate();
//find the button and trigger click event.
driver.FindElementById("buttonSearchPro").Click() ;
driver.Close(); // closes browser
driver.Quit(); // closes IEDriverServer process
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
Console.WriteLine("OK");
Console.ReadKey();
}
the result like this:
I'm trying to access a request's raw input body/stream in ASP.net 5. In the past, I was able to reset the position of the input stream to 0 and read it into a memory stream but when I attempt to do this from the context the input stream is either null or throws an error (System.NotSupportedException => "Specified method is not supported.").
In the first example below I can access the raw request in a controller if I declare the controller method's parameter object type as dynamic. For various reasons, this is not a solution and I need to access the raw request body in an authentication filter anyways.
This Example Works, But Is Not a Reasonable Solution:
[HttpPost("requestme")]
public string GetRequestBody([FromBody] dynamic body)
{
return body.ToString();
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
var m = new MemoryStream();
Request.Body.CopyTo(m);
var contentLength = m.Length;
var b = System.Text.Encoding.UTF8.GetString(m.ToArray());
return b;
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
Request.Body.Position = 0;
var input = new StreamReader(Request.Body).ReadToEnd();
return input;
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
Request.Body.Position = 0;
var input = new MemoryStream();
Request.Body.CopyTo(input);
var inputString = System.Text.Encoding.UTF8.GetString(input.ToArray());
return inputString;
}
I need to access the raw request body of every request that comes in for an API that I am building.
Any help or direction would be greatly appreciated!
EDIT:
Here is the code that I would like to read the request body in.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNet.Mvc;
using Microsoft.AspNet.Http;
namespace API.Filters
{
public class CustomAuthorizationAttribute : Attribute, IAuthorizationFilter
{
public CustomAuthorizationAttribute()
{ }
public void OnAuthorization(AuthorizationContext context)
{
if (context == null)
throw new ArgumentNullException("OnAuthorization AuthorizationContext context can not be null.");
else
{
if (this.AuthorizeCore(context.HttpContext) == false)
{
// Do Other Stuff To Check Auth
}
else
{
context.Result = new HttpUnauthorizedResult();
}
}
}
protected virtual bool AuthorizeCore(HttpContext httpContext)
{
var result = false;
using (System.IO.MemoryStream m = new System.IO.MemoryStream())
{
try
{
if (httpContext.Request.Body.CanSeek == true)
httpContext.Request.Body.Position = 0;
httpContext.Request.Body.CopyTo(m);
var bodyString = System.Text.Encoding.UTF8.GetString(m.ToArray());
return CheckBody(bodyString); // Initial Auth Check returns true/false <-- Not Shown In Code Here on Stack Overflow
}
catch (Exception ex)
{
Logger.WriteLine(ex.Message);
}
}
return false;
}
}
}
This code would be accessed when a call is made to a controller method marked with the CustomAuthorization attribute like so.
[Filters.CustomAuthorizationAuthorization]
[HttpPost]
public ActionResult Post([FromBody]UserModel Profile)
{
// Process Profile
}
Update
The information below is pretty outdated by now. Due to performance reasons this is not possible by default, but fortunately can be changed. The latest solution should be to enable request buffering with EnableBuffering:
Request.EnableBuffering();
See also this blog post for more information: https://devblogs.microsoft.com/aspnet/re-reading-asp-net-core-request-bodies-with-enablebuffering/.
Old, outdated answer for reference
The implementation of Request.Body depends on the controller action.
If the action contains parameters it's implemented by Microsoft.AspNet.WebUtilities.FileBufferingReadStream, which supports seeking (Request.Body.CanSeek == true). This type also supports setting the Request.Body.Position.
However, if your action contains no parameters it's implemented by Microsoft.AspNet.Loader.IIS.FeatureModel.RequestBody, which does not support seeking (Request.Body.CanSeek == false). This means you can not adjust the Position property and you can just start reading the stream.
This difference probably has to do with the fact that MVC needs to extract the parameters values from the request body, therefore it needs to read the request.
In your case, your action does not have any parameters. So the Microsoft.AspNet.Loader.IIS.FeatureModel.RequestBody is used, which throws an exception if you try to set the Position property.
**Solution**: either do not set the position or check if you actually _can_ set the position first:
if (Request.Body.CanSeek)
{
// Reset the position to zero to read from the beginning.
Request.Body.Position = 0;
}
var input = new StreamReader(Request.Body).ReadToEnd();
The exceptions you see in your three last snippets are the direct consequence of trying to read the request body multiple times - once by MVC 6 and once in your custom code - when using a streamed host like IIS or WebListener. You can see this SO question for more information: Read body twice in Asp.Net 5.
That said, I'd only expect this to happen when using application/x-www-form-urlencoded, since it wouldn't be safe for MVC to start reading the request stream with lengthy requests like file uploads. If that's not the case, then it's probably a MVC bug you should report on https://github.com/aspnet/Mvc.
For workarounds, you should take a look at this SO answer, that explains how you can use context.Request.ReadFormAsync or add manual buffering: Read body twice in Asp.Net 5
app.Use(next => async context => {
// Keep the original stream in a separate
// variable to restore it later if necessary.
var stream = context.Request.Body;
// Optimization: don't buffer the request if
// there was no stream or if it is rewindable.
if (stream == Stream.Null || stream.CanSeek) {
await next(context);
return;
}
try {
using (var buffer = new MemoryStream()) {
// Copy the request stream to the memory stream.
await stream.CopyToAsync(buffer);
// Rewind the memory stream.
buffer.Position = 0L;
// Replace the request stream by the memory stream.
context.Request.Body = buffer;
// Invoke the rest of the pipeline.
await next(context);
}
}
finally {
// Restore the original stream.
context.Request.Body = stream;
}
});
I just had this same issue. Remove the parameters from the method signature, and then read the Request.Body Stream how you want to.
You need to call Request.EnableRewind() to allow the stream to be rewound so you can read it.
string bodyAsString;
Request.EnableRewind();
using (var streamReader = new StreamReader(Request.Body, Encoding.UTF8))
{
bodyAsString = streamReader.ReadToEnd();
}
I Know this my be late but in my case its Just I had a problem in routing as bellow
At startup.cs file I was beginning the routing with /api
app.MapWhen(context => context.Request.Path.StartsWithSegments(new PathString("/api")),
a =>
{
//if (environment.IsDevelopment())
//{
// a.UseDeveloperExceptionPage();
//}
a.Use(async (context, next) =>
{
// API Call
context.Request.EnableBuffering();
await next();
});
}
//and I was putting in controller
[HttpPost]
[Route("/Register", Name = "Register")]
//Just Changed the route to start with /api like my startup.cs file
[HttpPost]
[Route("/api/Register", Name = "Register")]
//and now the params are not null and I can ready the body request multiple
To begin with, I checked the discussions regarding this issue and couldn't find an answer to my problem and that's why I'm opening this question.
I've set up a web service using restlet 2.0.15.The implementation is only for the server. The connections to the server are made through a webpage, and therefore I didn't use ClientResource.
Most of the answers to the exhaustion of the thread pool problem suggested the inclusion of
#exhaust + #release
The process of web service can be described as a single function.Receive GET requests from the webpage, query the database, frame the results in XML and return the final representation. I used a Filter to override the beforeHandle and afterHandle.
The code for component creation code:
Component component = new Component();
component.getServers().add(Protocol.HTTP, 8188);
component.getContext().getParameters().add("maxThreads", "512");
component.getContext().getParameters().add("minThreads", "100");
component.getContext().getParameters().add("lowThreads", "145");
component.getContext().getParameters().add("maxQueued", "100");
component.getContext().getParameters().add("maxTotalConnections", "100");
component.getContext().getParameters().add("maxIoIdleTimeMs", "100");
component.getDefaultHost().attach("/orcamento2013", new ServerApp());
component.start();
The parameters are the result of a discussion present in this forum and modification by my part in an attempt to maximize efficiency.
Coming to the Application, the code is as follows:
#Override
public synchronized Restlet createInboundRoot() {
// Create a router Restlet that routes each call to a
// new instance of HelloWorldResource.
Router router = new Router(getContext());
// Defines only one route
router.attach("/{taxes}", ServerImpl.class);
//router.attach("/acores/{taxes}", ServerImplAcores.class);
System.out.println(router.getRoutes().size());
OriginFilter originFilter = new OriginFilter(getContext());
originFilter.setNext(router);
return originFilter;
}
I used an example Filter found in a discussion here, too. The implementation is as follows:
public OriginFilter(Context context) {
super(context);
}
#Override
protected int beforeHandle(Request request, Response response) {
if (Method.OPTIONS.equals(request.getMethod())) {
Form requestHeaders = (Form) request.getAttributes().get("org.restlet.http.headers");
String origin = requestHeaders.getFirstValue("Origin", true);
Form responseHeaders = (Form) response.getAttributes().get("org.restlet.http.headers");
if (responseHeaders == null) {
responseHeaders = new Form();
response.getAttributes().put("org.restlet.http.headers", responseHeaders);
responseHeaders.add("Access-Control-Allow-Origin", origin);
responseHeaders.add("Access-Control-Allow-Methods", "GET,POST,DELETE");
responseHeaders.add("Access-Control-Allow-Headers", "Content-Type");
responseHeaders.add("Access-Control-Allow-Credentials", "true");
response.setEntity(new EmptyRepresentation());
return SKIP;
}
}
return super.beforeHandle(request, response);
}
#Override
protected void afterHandle(Request request, Response response) {
if (!Method.OPTIONS.equals(request.getMethod())) {
Form requestHeaders = (Form) request.getAttributes().get("org.restlet.http.headers");
String origin = requestHeaders.getFirstValue("Origin", true);
Form responseHeaders = (Form) response.getAttributes().get("org.restlet.http.headers");
if (responseHeaders == null) {
responseHeaders = new Form();
response.getAttributes().put("org.restlet.http.headers", responseHeaders);
responseHeaders.add("Access-Control-Allow-Origin", origin);
responseHeaders.add("Access-Control-Allow-Methods", "GET,POST,DELETE"); //
responseHeaders.add("Access-Control-Allow-Headers", "Content-Type");
responseHeaders.add("Access-Control-Allow-Credentials", "true");
}
}
super.afterHandle(request, response);
Representation requestRepresentation = request.getEntity();
if (requestRepresentation != null) {
try {
requestRepresentation.exhaust();
} catch (IOException e) {
// handle exception
}
requestRepresentation.release();
}
Representation responseRepresentation = response.getEntity();
if(responseRepresentation != null) {
try {
responseRepresentation.exhaust();
} catch (IOException ex) {
Logger.getLogger(OriginFilter.class.getName()).log(Level.SEVERE, null, ex);
} finally {
}
}
}
The responseRepresentation does not have a #release method because it crashes the processes giving the warning WARNING: A response with a 200 (Ok) status should have an entity (...)
The code of the ServerResource implementation is the following:
public class ServerImpl extends ServerResource {
String itemName;
#Override
protected void doInit() throws ResourceException {
this.itemName = (String) getRequest().getAttributes().get("taxes");
}
#Get("xml")
public Representation makeItWork() throws SAXException, IOException {
DomRepresentation representation = new DomRepresentation(MediaType.TEXT_XML);
DAL dal = new DAL();
String ip = getRequest().getCurrent().getClientInfo().getAddress();
System.out.println(itemName);
double tax = Double.parseDouble(itemName);
Document myXML = Auxiliar.getMyXML(tax, dal, ip);
myXML.normalizeDocument();
representation.setDocument(myXML);
return representation;
}
#Override
protected void doRelease() throws ResourceException {
super.doRelease();
}
}
I've tried the solutions provided in other threads but none of them seem to work. Firstly, it does not seem that the thread pool is augmented with the parameters set as the warnings state that the thread pool available is 10. As mentioned before, the increase of the maxThreads value only seems to postpone the result.
Example: INFO: Worker service tasks: 0 queued, 10 active, 17 completed, 27 scheduled.
There could be some error concerning the Restlet version, but I downloaded the stable version to verify this was not the issue.The Web Service is having around 5000 requests per day, which is not much.Note: the insertion of the #release method either in the ServerResource or OriginFilter returns error and the referred warning ("WARNING: A response with a 200 (Ok) status should have an entity (...)")
Please guide.
Thanks!
By reading this site the problem residing in the server-side that I described was resolved by upgrading the Restlet distribution to the 2.1 version.
You will need to alter some code. You should consult the respective migration guide.
I have a WCF service that is processing a call, sending that processed data onto another service, and alerting the caller and any other instances of that application by firing a callback. Originally the callbacks were being called at the end but I found that if the second service was not running that there would be a twenty second delay while we attempted to discover it. Only then were the callbacks called. I moved the callback notification before the call to the second service but it still had the delay. I even tried firing the callbacks on a background process but that didn't work either. Is there a way to get around this delay, outside of changing the timeout of the discovery? Here is a code snippet.
// Alert the admins of the change.
if (alertPuis)
{
ReportBoxUpdated(data.SerialNumber);
}
// Now send the change to the box if he's online.
var scope = new Uri(string.Format(#"net.tcp://{0}", data.SerialNumber));
var boxAddress = DiscoveryHelper.DiscoverAddress<IAtcBoxService>(scope);
if (boxAddress != null)
{
var proxy = GetBoxServiceProxy(boxAddress);
if (proxy != null)
{
proxy.UpdateBox(boxData);
}
else
{
Log.Write("AtcSystemService failed on call to update toool Box: {0}",
data.SerialNumber);
}
}
else if (mDal.IsBoxDataInPendingUpdates(data.SerialNumber) == false)
mDal.AddPendingUpdate(data.SerialNumber, null, true, null);
}
and
private static void ReportBoxUpdated(string serialNumber)
{
var badCallbacks = new List<string>();
Action<IAtcSystemServiceCallback> invoke = callback =>
callback.OnBoxUpdated(serialNumber);
foreach (var theCallback in AdminCallbacks)
{
var callback = theCallback.Value as IAtcSystemServiceCallback;
try
{
invoke(callback);
}
catch (Exception ex)
{
Log.Write("Failed to execute callback for admin instance {0}: {1}",
theCallback.Key, ex.Message);
badCallbacks.Add(theCallback.Key);
}
}
foreach (var bad in badCallbacks) // Clean out any stale callbacks from the list.
{
AdminCallbacks.Remove(bad);
}
}
Have you considered caching the result?