Our Ktor application clients are sending headers larger than 8KB. Those requests are being rejected by KTor server engines with a HTTP 400. I've tried Netty and Tomcat so far and they both fail with a HTTP 400 status. With Tomcat engine, the error is more obvious as the response from Tomcat contains the text: java.lang.IllegalArgumentException: Request header is too large . I have tried the configuration suggested in https://youtrack.jetbrains.com/issue/KTOR-27 . The config parameter maxHeaderLength doesn't work. This was suggested in the related PR https://github.com/ktorio/ktor/pull/2490 . I am using Ktor version 1.6.8. How can I get my Ktor apps working with large headers, like headers with almost 16000 characters?
You can configure the Netty engine to provide a HttpServerCodec object with the desired value for maximum header size. Here is an example:
embeddedServer(Netty, applicationEngineEnvironment {
connector {
port = 3333
}
module {
routing {
get("/") {
call.respondText { "Hello" }
}
}
}
}) {
httpServerCodec = {
HttpServerCodec(
HttpObjectDecoder.DEFAULT_MAX_INITIAL_LINE_LENGTH,
32 * 1024, // max header size
HttpObjectDecoder.DEFAULT_MAX_CHUNK_SIZE
)
}
}.start(true)
Related
The WebSocket server is a online testing one
The Website
Something goes wrong And I don't know how to fix it.
val client = HttpClient(CIO) { install(WebSockets) }
GlobalScope.launch {
client.webSocket("ws://82.157.123.54:9010/ajaxchattest") {}
}
the error printStackTrace
java.lang.IllegalStateException: Failed to parse request body: request body length
should be specified,
chunked transfer encoding should be used or
keep-alive should be disabled (connection: close)
not knowing how to enable encoding or disable keep-alive or specify body length.
The 82.157.123.54:9010/ajaxchattest endpoint responds with 403 Forbidden instead of 101 Switching Protocols if the Origin header is absent or invalid. So to make it work just append the Origin header with a well-formed value:
val client = HttpClient(CIO) { install(WebSockets) }
client.webSocket("ws://82.157.123.54:9010/ajaxchattest", request = {
header(HttpHeaders.Origin, "http://example")
}) {}
I'm trying to learn grpc using kotlin and make a simple grpc service with following proto definition :
syntax = "proto3";
option java_multiple_files = true;
option java_package = "br.bortoti";
option java_outer_classname = "StockProto";
option objc_class_prefix = "HLW";
package br.bortoti;
import "google/api/annotations.proto";
service StockService {
rpc GetStock (GetStockRequest) returns (Stock) {
option(google.api.http) = {
get: "V1/stocks/{stock}"
body: "*"
};
}
}
message Stock {
string ticker = 1;
}
message GetStockRequest {
string ticker = 1;
}
message GetStockReply {
string ticker = 1;
}
so, i'm basically mapping a service to a get request.
but when i try to call this url from curl like :
curl http://localhost:8088/V1/stocks/1
i get the error :
curl: (1) Received HTTP/0.9 when not allowed
and from the server side i have :
INFO: Transport failed
io.netty.handler.codec.http2.Http2Exception: Unexpected HTTP/1.x request: GET /V1/stocks/1
how can i make the server accept http 1.1 calls? is it even possible?
Maybe this is two question.
The gRPC use HTTP2 and need lots of headers so it is diffcult request by curl. Maybe you need grpcurl
And the path V1/stocks/{stock} need use grpc-gateway toghter, you can reference grpc-gateway for more detail.
Since you are learn how to use gRPC, maybe you can reference this project: helloworlde/grpc-java-sample, feel free to translate chinese.
I have a Vue.js project where I have configured a webpack dev-server to proxy all requests to the UI to my backend server. Here is the relevant part of vue.config.js:
devServer: {
contentBase: PATHS.build,
port: 9000,
https: false,
hot: true,
progress: true,
inline: true,
watchContentBase: true,
proxy: {
'^/': {
target: 'http://127.0.0.1:8089',
secure: false
},
}
},
I've noticed that if the HTTP response code from http://127.0.0.1:8089 is anything other than 2xx then the proxy fails with the following error:
Proxy error: Could not proxy request /api/test from localhost:9000 to http://127.0.0.1:8089.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (HPE_INVALID_CHUNK_SIZE).
This also causes the HTTP response code from the request to localhost:9000 to be 500 for any error and all the information about what went wrong on the server side is lost. This is problematic as I want to be able to extract information from error responses to display to the user.
I know it's possible to do because I had it working on an older Angular project which I think was using Webpack 3 (am now using Webpack 4). I tried copying all the dev-server config from this project but it just doesn't seem to work here!
EDIT: I was wrong. The Proxy error does not occur on every bad response but only for one of the requests which is a multipart file upload. Still unable to reproduce this in a smaller example to put on github though so struggling to pinpoint the cause.
This error message comes from node_modules/#vue/cli-service/lib/util/prepareProxy.js, which define a onError callback for node-http-proxy;
So I did some experiment, make back-end api generate 400 404 500 response, but I didn't got this error.
After I happen to close back-end api, error arise:
Proxy error: Could not proxy request /hello from localhost:8080 to http://localhost:8081 (ECONNREFUSED).
I search in the doc and find these:
The error event is emitted if the request to the target fail. We do not do any error handling of messages passed between client and proxy, and messages passed between proxy and target, so it is recommended that you listen on errors and handle them
So the onError do not handle error code, is called only when request fail (500 response is still treated as a complete request, connection refuse is not)
Go back to your error message, [HPE_INVALID_CHUNK_SIZE] means bad request to the back-end api. In this issue, it gives an solution: add a keep-alive header:
devServer: {
publicPath: 'http://localhost:9090/front/static-dev/build/',
port: 9090,
proxy: {
'/**': {
target: 'http://localhost:8080',
secure: false,
changeOrigin: true,
headers: {
Connection: 'keep-alive'
}
},
open: true
}
I have finally found the problem, and I apologise, it was a lot more of a specific issue than I originally thought when I wrote the question.
Issue was to do with a request which was proxied to another server using the Spring RestTemplate:
e.g.
#PostMapping("/upload")
public ResponseEntity upload(#RequestParam("file") MultipartFile file)
throws Exception {
String baseUrl = serviceProperties.getAddress();
HttpEntity<MultiValueMap<String, Object>> request = createMultipartRequest(file.getBytes());
return restTemplate.postForEntity(baseUrl + "/api/upload", filterRequest, String.class);
}
The ResponseEntity returning from the rest template proxy contained the header "Connection: close" when the response was anything other than 200 which cause the connection to close and caused this request to fail to return anything which subsequently made the dev-server proxy fail on the UI.
Fixed this by not passing the response headers from the rest template proxy to the response:
#PostMapping("/upload")
public ResponseEntity upload(#RequestParam("file") MultipartFile file)
throws Exception {
String baseUrl = serviceProperties.getAddress();
HttpEntity<MultiValueMap<String, Object>> request = createMultipartRequest(file.getBytes());
ResponseEntity response = restTemplate.postForEntity(baseUrl + "/api/upload", filterRequest, String.class);
return new ResponseEntity<>(response.getBody(), response.getStatusCode());
}
I have a requirement in Nifi where I have cycle through different HTTPS REST Endpoints and provide different certificates for some endpoints and different username / password for some other endpoints.
I used InvokeHTTP processor to send the requests, although URL takes an expression language, I cannot setup SSLContextService with an expression.
Alternatively, I thought on using ExecuteScript to call those Endpoints, however as listed here in StackOverflow post; I still don't know how to programmatically call an external service through a script.
Any help appreciated.
just for fun created the groovy script that calls http.
for sure you can avoid using it. and I believe InvokeHTTP processor covers almost all needs.
However.. going to call test rest service: /post at https://httpbin.org
the flow: GenerateFlowFile (generates body) -> EcecuteGroovyScript (call service)
The body generated by GenerateFlowFile : {"id":123, "txt":"aaabbbccc"}
In ExecuteGroovyScript 1.5.0 declare the CTL.ssl1 property and link it to StandardSSLContextService
and now the script:
#Grab(group='acme.groovy', module='acmehttp', version='20180301', transitive=false)
import groovyx.acme.net.AcmeHTTP
import org.apache.nifi.ssl.SSLContextService.ClientAuth
def ff=session.get()
if(!ff)return
def http
ff.write{ffIn, ffOut->
http = AcmeHTTP.post(
url: "https://httpbin.org/post", //base url
query: [aaa:"hello", bbb:"world!"], //query parameters
// send flowfile content (stream) as a body
body: ffIn,
headers:[
//assign content-type from flowfile `mime.type` attribute
"content-type":ff.'mime.type'
],
// you can declare `CTX.ssl1`, `CTX,.ssl2`,... processor properties and map them to SSLContextService
// then depending on some condition create different SSLContext
// in this case let's take `CTL.ssl1` service to create context
ssl: CTL["ssl"+1].createSSLContext(ClientAuth.WANT),
// the next commented line creates trust all ssl context:
//ssl: AcmeHTTP.getNaiveSSLContext(),
// the receiver that transfers url response stream to flowfile stream
receiver:{respStream, httpCtx-> ffOut << respStream }
)
}
//set response hesders as flow file attributes with 'http.header.' prefix
http.response.headers.each{ k,v-> ff['http.header.'+k]=v }
//status code and message
ff.'http.status.code' = http.response.code
ff.'http.status.message' = http.response.message
if( http.response.code < 400){
//transfer to success if response was ok
REL_SUCCESS << ff
}else{
//transfer to failure when response code is 400+
REL_FAILURE << ff
}
I'm looking for a way to customize StrongLoop LoopBack HTTP response code and headers.
I would like to conform to some company business rules regarding REST API.
Typical case is, for a model described in JSON, to have HTTP to respond to POST request with a code 201 + header Content-Location (instead of loopback's default response code 200 without Content-Location header).
Is it possible to do that using LoopBack ?
Unfortunately the way to do this is a little difficult because LoopBack does not easily have hooks to modify all responses coming out of the API. Instead, you will need to add some code to each model in a boot script which hooks in using the afterRemote method:
Inside /server/boot/ add a file (the name is not important):
module.exports = function(app) {
function modifyResponse(ctx, model, next) {
var status = ctx.res.statusCode;
if (status && status === 200) {
status = 201;
}
ctx.res.set('Content-Location', 'the internet');
ctx.res.status(status).end();
}
app.models.ModelOne.afterRemote('**', modifyResponse);
app.models.ModelTwo.afterRemote('**', modifyResponse);
};