socket http lua to set timeout - api

I am trying to create a function that can call REST with the http socket lua.
And I tried to set the timeout this way. But, when I run this function, the timeout is not running. How should I set the timeout?
local http = require "socket.http"
local socket = require "socket"
local respbody = {}
http.request {
method = req_method,
url = req_url,
source = ltn12.source.string(req_body),
headers =
{
["Content-Type"] = req_content_type,
["content-length"] = string.len(req_body),
["Host"] = host,
},
sink = ltn12.sink.table(respbody),
create = function()
local req_sock = socket.tcp()
req_sock:settimeout(3, 't')
return req_sock
end,
}

You may want to check lua-http. I use it to call REST and works like a charm. I am not an expert but, as far as I can tell, it is a good LUA http implementation.
You can set a two seconds timeout as simple as:
local http_client = require "http.client"
local myconnection = http_client.connect {
host = "myrestserver.domain.com";
timeout = 2;
}
Full documentation in here.

if I implement the example with my requirements, will it be like this? cmiiw
local http_client = require "http.client"
local req_body = "key1=value1&key2=value2"
local myconnection = http_client.connect {
method = "POST";
url = "myrestserver.domain.com/api/example";
host = "myrestserver.domain.com";
source = req_body
headers = {
["Content-Type"] = "application/x-www-form-urlencoded",
["content-length"] = string.len(req_body),
},
timeout = 2;
}

LuaSocket implicitly set http.TIMEOUT to the socket object.
Also you have to remember that socket timeout is not the same as request timeout.
Socket timeout means timeout for each operation independently. For simple case you can wait connection up to timeout seconds and then each read operation can take up to timeout seconds. And because of HTTP client read response line by line you get timeout seconds for each header plus for each body chunk. Also, there may be redirecions where each redirection is a separate HTTP request/response. If you use TLS there also will be hendshake after connection which also took several send/receive operation.
I did not use lua-http module and do not know how timeout implemented there.
But I prefer use modules like cURL if I really need to restrict request timeout.

Related

Blazor WebAssembly MQTT over websockets not working

I'm trying to implement an mqtt over websocket client subscriber in Blazor using Paho. The problem is it insists on using wss instead of ws and throws an ERR_SSL_PROTOCOL_ERROR error upon connection.
Here's a simplified code block:
var mqtt;
var host = "api.mydomainexample.com";
var port = 1884;
function onConnect(){
console.log("connected ! Now listening for messages ..");
mqtt.subscribe("someTopic");
}
function onFailure(message){
console.log("connection to host failed: " + message);
}
function onMessageArrived(msg){
var message = "Message received on topic '"+ msg.destinationName +"': "+ msg.payloadString;
console.log(message);
}
function mqttConnect() {
console.log("connecting to " + host + " ..");
mqtt = new Paho.MQTT.Client(host, port, clientid);
var options = {
timeout: 3,
onSuccess: onConnect,
onFailure: onFailure,
useSSL: false
};
mqtt.onMessageArrived = onMessageArrived;
mqtt.connect(options);
}
I copied this code into an html page created in notepad, called the function from the html body and ran the file in browser. It worked and subscribed well.
Also I added useSSL: false in the connection options although I didnt have it before but still didnt work.
here's the error I'm having from console:
WebSocket connection to 'wss://api.mydomainexample:1884/mqtt' failed: Error in connection establishment: net::ERR_SSL_PROTOCOL_ERROR
I also changed my projects launch settings so that it launches as http and not https because based on this answer, I cannot use a ws from a page loaded through https.
Any ideas ? Can't I just connect to a websocket without certificate in blazor?
Ok it turns out that when creating the blazor application, there is an option to 'configure on https' where this option causes requests redirection from http to https and consequently asks for secure wss instead of ws.
Hope this helps someone!

S3 presigned URL: Generate on server, use from client?

Is it possible to generate an S3 presigned URL in a Lambda function and return that URL to a client, so the client can use it to do an unauthenticated HTTP PUT?
I'm finding that S3 is unexpectedly closing my HTTPS connection when I try to PUT to the URL I get from the lambda function, and I don't know if it's because the server and client are different entities.
Can I do what I want to do? Is there a secret step I'm missing here?
EDIT: per Anon Coward's request, the server-side code is:
presigned_upload_parts = []
for part in range(num_parts):
resp = s3.generate_presigned_url(
ClientMethod = 'upload_part',
Params = {
'Bucket': os.environ['USER_UPLOADS_BUCKET'],
'Key': asset_id,
'UploadId': s3_upload_id,
'PartNumber': part
}
)
presigned_upload_parts.append({"part": part, "url": resp})
return custom_http_response_wrapper(presigned_upload_parts)
The client-side code is:
for idx, part in enumerate(urls):
startByte = idx * bytes_per_part
endByte = min(filesize, ((idx + 1) * bytes_per_part))
f.seek(startByte, 0)
bytesBuf = f.read(endByte - startByte)
print(f"Buffer is type {type(bytesBuf)} with length {len(bytesBuf):,}")
print(f"Part {str(idx)}: bytes {startByte:,} to {endByte:,} as {part['url']}")
#resp = requests.post(part['url'], data = bytesBuf, headers = self.get_standard_headers())
resp = requests.put(
url = part['url'],
data = bytesBuf
)
The error I'm getting is:
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
The presigned URL looks like:
https://my-bucket-name.s3.amazonaws.com/my/item/key?uploadId=yT2W....iuiggs-&partNumber=0&AWSAccessKeyId=ASIAR...MY&Signature=i6duc...Mmpc%3D&x-amz-security-token=IQoJ...%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F...SWHC&Expires=1657135314
There was a bug in my code somewhere. I ran the code under WSL as a test, and in the Linux environment got a more friendly error that helped me find and fix a minor bug, and now it's running as expected in the Windows environment. Whether that's because of the bugfix or some other environmental change I'll never know.

Is there a way to specify a connection timeout for the client?

I'd imagine this should be a property of a channel but the channel doesn't have anything related to the connection configuration.
You can set the request timeout like this:
from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import service_pb2_grpc, service_pb2
stub = service_pb2_grpc.V2Stub(ClarifaiChannel.get_grpc_channel())
if __name__ == '__main__':
YOUR_CLARIFAI_API_KEY = 'addyourclarifaikeyhere'
auth_metadata = (('authorization', f'Key {YOUR_CLARIFAI_API_KEY}'),)
resp = stub.ListModels(service_pb2.ListModelsRequest(),
metadata=auth_metadata,
timeout=0.0001)
print(resp)
Looking at the list of configurations for a grpc channel, there doesn't seem to be a global connection timeout.

Make curl call to grpc service

I'm trying to learn grpc using kotlin and make a simple grpc service with following proto definition :
syntax = "proto3";
option java_multiple_files = true;
option java_package = "br.bortoti";
option java_outer_classname = "StockProto";
option objc_class_prefix = "HLW";
package br.bortoti;
import "google/api/annotations.proto";
service StockService {
rpc GetStock (GetStockRequest) returns (Stock) {
option(google.api.http) = {
get: "V1/stocks/{stock}"
body: "*"
};
}
}
message Stock {
string ticker = 1;
}
message GetStockRequest {
string ticker = 1;
}
message GetStockReply {
string ticker = 1;
}
so, i'm basically mapping a service to a get request.
but when i try to call this url from curl like :
curl http://localhost:8088/V1/stocks/1
i get the error :
curl: (1) Received HTTP/0.9 when not allowed
and from the server side i have :
INFO: Transport failed
io.netty.handler.codec.http2.Http2Exception: Unexpected HTTP/1.x request: GET /V1/stocks/1
how can i make the server accept http 1.1 calls? is it even possible?
Maybe this is two question.
The gRPC use HTTP2 and need lots of headers so it is diffcult request by curl. Maybe you need grpcurl
And the path V1/stocks/{stock} need use grpc-gateway toghter, you can reference grpc-gateway for more detail.
Since you are learn how to use gRPC, maybe you can reference this project: helloworlde/grpc-java-sample, feel free to translate chinese.

Socket Hang Up when using https.request in node.js

When using https.request with node.js v04.7, I get the following error:
Error: socket hang up
at CleartextStream.<anonymous> (http.js:1272:45)
at CleartextStream.emit (events.js:61:17)
at Array.<anonymous> (tls.js:617:22)
at EventEmitter._tickCallback (node.js:126:26)
Simplified code that will generate the error:
var https = require('https')
, fs = require('fs')
var options = {
host: 'localhost'
, port: 8000
, key: fs.readFileSync('../../test-key.pem')
, cert: fs.readFileSync('../../test-cert.pem')
}
// Set up server and start listening
https.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'})
res.end('success')
}).listen(options.port, options.host)
// Wait a second to let the server start up
setTimeout(function() {
var clientRequest = https.request(options, function(res) {
res.on('data', function (chunk) {
console.log('Called')
})
})
clientRequest.write('')
clientRequest.end()
}, 1000)
I get the error even with the server and client running on different node instances and have tested with port 8000, 3000, and 443 and with and without the SSL certificates. I do have libssl and libssl-dev on my Ubuntu machine.
Any ideas on what could be the cause?
In
https.createServer(function (req, res) {
you are missing options when you create the server, should be:
https.createServer(options, function (req, res) {
with your key and cert inside
I had a very similar problem where the response's end event never fired.
Adding this line fixed the problem:
// Hack to emit end on close because of a core bug that never fires end
response.on('close', function () {response.emit('end')});
I found an example of this in the request library mentioned in the previous answer.
Short answer: Use the the latest source code instead of the one you have. Store it where you will and then require it, you are good to go.
In the request 1.2.0 source code, main.js line 76, I see
http.createClient(options.uri.port, options.uri.hostname, options.uri.protocol === 'https:');
Looking at the http.js source code, I see
exports.createClient = function(port, host) {
var c = new Client();
c.port = port;
c.host = host;
return c;
};
It is requesting with 3 params but the actual function only has 2. The functionality is replaced with a separate module for https.
Looking at the latest main.js source code, I see dramatic changes. The most important is the addition of require('https').
It appears that request has been fixed but never re-released. Fortunately, the fix seems to work if you just copy manually from the raw view of the latest main.js source code and use it instead.
I had a similar problem and i think i got a fix. but then I have another socket problem.
See my solution here: http://groups.google.com/group/nodejs/browse_thread/thread/9189df2597aa199e/b83b16c08a051706?lnk=gst&q=hang+up#b83b16c08a051706
key point: use 0.4.8, http.request instead of http.createClient.
However, the new problem is, if I let the program running for long time, (I actually left the program running but no activity during weekend), then I will get socket hang up error when I send a request to http Server. (not even reach the http.request). I don't know if it is because of my code, or it is different problem with http Server