I am creating a nodejs program that sends HTTP requests. I want the HTTP requests to reach the server at an exact time. For example, I might want to send an HTTP requeset and have it reach the server at exactly Tuesday, August 6, 2019 1:00:00 PM, or 1565096400000 (epoch timestamp in milliseconds).
How would I do this?
One thing I've considered is using setTimeout() to time my requests at a certain time, but the time at which they reach the server is inconsistent.
Here is an example of a using setTimeout to time my requests.
In this example, I would want the request to reach the server at exactly 1565096400000 (epoch timestamp).
var options = {
url = 'someurl.com';
}
function callback(error, response, body) {
if(!error) {
console.log('Error.');
} else {
console.log(body);
}
}
var time_until_request = 1565096400000 - Date.now();
//We want the request to reach the server at exactly 1565096400000 (epoch //timestamp).
var timeout = setTimeout(function(){
request(options, callback);
},time_until_request);
Of course, with what I am doing, the request won't reach the server at the time I want it to. It will reach the server some time before or after - I believe due to inconsistencies with latency.
Related
I am debugging/testing the part of my app that sends an HTTP POST, with a file to upload, to a 3rd party server and need to be sure of the order of the info that server sends back.
Below is a very trimmed down sample of how I handle what is returned by the server.
At this moment, am debugging for files sent that exceed the LimitRequestBody size set in Apache. Yes, I do check file size in my app before sending, but am trying to debug for anything possible, i.e. malicious bot sending data outside of my app.
What I can't seem to find online is the lifecycle of what a server will send back in terms of the response, data, and error, and need to be sure I will get them back in this order:
Response
Data
and if there's an error:
Error (and then nothing else)
uploadSession.dataTask(with: upFile.toUrl!)
{ (data, response, error) in
if let response = response {
upLoadInvClass.upResp(resp: response)
}
if let error = error {
upLoadInvClass.upErr(error: error)
}
if let data = data {
upLoadInvClass.upData(data: data)
}
}.resume()
I'm sending a post request to the same url multiple times in a row. Some of the requests return 200 status code and some return 204 saying no content. the request updates some content in a mongo database. I don't know if this is relevant. What could cause this problem?
My index.js:
app.post('/updatetrialsession',authenticateJWT ,(req,res)=>{
User.findOne({username:req.user.username}).then(user=>{
var trialIdx=user.examTrials.findIndex(it=>it.trialId===req.body.trialId)
var questionIdx=req.body.questionIdx
if(trialIdx!==-1){
user.examTrials[trialIdx].questions[questionIdx]=req.body.question
user.examTrials[trialIdx].currentQuestion=questionIdx+1
user.examTrials[trialIdx].countDown=req.body.countDown
user.examTrials[trialIdx].numOfSolved=req.body.numOfSolved
var filter={
'username':user.username
}
var update={
$set:{
'examTrials':user.examTrials
}
}
User.findOneAndUpdate(filter,update).then(user=>{
console.log('updated')
res.json({
status:"success"
})
})
}
})
})
Note: all requests go to this url "/epdatetrialsession".
when there's a 20 sec gap between each request everything works just fine. but when the server get flooded with say a request every 2 seconds or so, some return with 204 status code
I have had the same issue once. In my experience, When sending multiple number of requests to the server within a very short period of time, sometimes the server fails to answer all of them. Depends on the server capability. Did you try to check this by, sending the request for only the ones which returns 204 one at a time and see whether they still returns a 204? If so, the server has responded correctly even when sending the requests in a row. But if the server return a content with 200 when sending one at a time, then the issue might be with the server when handling multiple requests at a time.
I am making a nodejs program that sends HTTP requests. How could I determine how quickly the request reaches the website?
I can determine the amount of time between sending the request and receiving a response from the server. I'm not sure if I can take this time and just divide it by 2 to determine the amount of time between sending the request and the server receiving the request.
Here is an example of how I would determine the amount of time between sending the request and receiving a response from the server.
var request = require('request');
var options = {
url = 'someurl.com';
}
var date1, date2;
function callback(error, response, body) {
if(!error) {
console.log('Error.');
} else {
date2 = new Date().getTime();
console.log(date2-date1);
}
}
date1 = new Date().getTime();
request(options, callback);
This would give me the total time (the amount of time between sending request and receiving response), but not the time that I want (the amount of time between sending request and the website receiving request).
It depends on whether you have access to the API, which you are sending your requests.
If that is the case, then you can keep your date1 and generate date2 on the server upon receiving the request. Then include that in your response and calculate the difference.
For that synchronise your client<->server timeserver (ntp).
Otherwise you can always keep your approach and divide by 2 as you mentioned.
Hope it helps !
We are using Masstransit with RabbitMq for making RPCs from one component of our system to others.
Recently we faced the limit of throughput on client side, measured about 80 completed responses per second.
While trying to investigate where the problem was, I found that requests were processed fast by the RPC server, then responses were put to callback queue, and then, the queue processing speed was 80 M\s
This limit is only on client side. Starting another process of the same client app on the same machine doubles requests throughput on the server side, but then I see two callback queues, filled with messages, are being consumed each with the same 80 M\s
We are using single instance of IBus
builder.Register(c =>
{
var busSettings = c.Resolve<RabbitSettings>();
var busControl = MassTransitBus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(busSettings.Host), h =>
{
h.Username(busSettings.Username);
h.Password(busSettings.Password);
});
cfg.UseSerilog();
cfg.Send<IProcessorContext>(x =>
{
x.UseCorrelationId(context => context.Scope.CommandContext.CommandId);
});
}
);
return busControl;
})
.As<IBusControl>()
.As<IBus>()
.SingleInstance();
The send logic looks like this:
var busResponse = await _bus.Request<TRequest, TResult>(
destinationAddress: _settings.Host.GetServiceUrl<TCommand>(queueType),
message: commandContext,
cancellationToken: default(CancellationToken),
timeout: TimeSpan.FromSeconds(_settings.Timeout),
callback: p => { p.WithPriority(priority); });
Has anyone faced the problem of that kind?
My guess that there is some program limit in the response dispatch logic. It might be the Max thread pool size, or the size of the buffer, also the prefetch count of response queue.
I tried to play with .Net thread pool size, but nothing helped.
I'm kind of new to Masstransit and will appreciate any help with my problem.
Hope it can be fixed in configuration way
There are a few things you can try to optimize the performance. I'd also suggest checking out the MassTransit-Benchmark and running it in your environment - this will give you an idea of the possible throughput of your broker. It allows you to adjust settings like prefetch count, concurrency, etc. to see how they affect your results.
Also, I would suggest using one of the request clients to reduce the setup for each request/response. For example, create the request client once, and then use that same client for each request.
var serviceUrl = yourMethodToGetIt<TRequest>(...);
var client = Bus.CreateRequestClient<TRequest>(serviceUrl);
Then, use that IRequestClient<TRequest> instance whenever you need to perform a request.
Response<Value> response = await client.GetResponse<TResponse>(new Request());
Since you are just using RPC, I'd highly recommend settings the receive endpoint queue to non-durable, to avoid writing RPC requests to disk. And adjust the bus prefetch count to a higher value (higher than the maximum number of concurrent requests you may have by 2x) to ensure that responses are always delivered directly to your awaiting response consumer (it's an internal thing to how RabbitMQ delivers messages).
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.PrefetchCount = 1000;
}
I have a client server application. The server side is completely in java. The client side has a few lines of html and the rest in plain javascript (I don't use ajax or jquery etc).
The client receives user inputs through an input box and when the user clicks a button, the client sends a POST request to the server using XmlHtpRequest. The server then runs a process that can take several minutes. The server periodically (every 5 seconds) sends a progress update message with HTTP status code 202 and with content "Processing.... completed 5 / 100". The numbers 5 and 100 are a sample. Instead of 100, it is the actual final counter of the process and instead of 5 it is the current counter indicating the progress. When the server finishes the processing it sends a 200 OK with the final output as the content. I have an output text area where I want to display the progress message during the processing and later the final output is also placed in the same output text area.
My client side javascript code is:
<script>
var xhr = new XmlHttpRequest();
function sendCommandToServer() {
window.xhr.onreadystatechange = getServerResponse;
window.xhr.open("POST", (document.domain + "/" + "executeCommand"), true);
window.xhr.send("arguments and data for the command");
}
</script>
<script>
function getServerResponse() {
if(window.xhr.status === 202) {
myTextArea.value = window.xhr.responseText;
window.xhr.onreadystatechange = getServerResponse;
return;
}
if(window.xhr.status === 200) {
myTextArea.value = window.xhr.responseText;
return;
}
alert("processing error");
}
</script>
The server correctly gets the command and runs the process and finishes and produces the output. It also correctly sends the progress messages. But my client side javascript does not get all of them. I am able to get the 1st progress message from the server - something like - "Processing .... 5 / 100" - but after that I am not able to get further calls to getServerResponse() to continue updating the progress as and when the server sends messages. If I modify the server side to avoid sending progress and send only the final output result, it works correctly and I get the final output result displayed at the client. I run this on IE10 and Chrome, with the same behavior. Can xmlHttpRequest() be used for this use-case? Am I doing something wrong? Any help will be appreciated. Thanks.
I'm new to this myself and trying to work out a method of achieving something similar.
I think the 202 status code is the only response you will get, ie the server will send a single response when it has accepted the job (202), or a 200 response when it has completed.
Source:
http://100pulse.com/http-statuscode/202.jsp