apache benchmark: fails with lengthy custom header for authorization - apache
Apache benchmark is failing when using lengthy custom authorization header -
ab.exe -n 1 -c 1 -S -H "Authorization: SAML PHRyaWJ1dGUgQXR0cmlidXRlTmFtZT0iaGFzX3NlbGVjdCIgQXR0cmlidXRlTmFtZXNwYWNlPSJodHRwOi8vc2NoZW1hcy5iZW50bGV5LmNvbS93cy8yMDExLzAzL2lkZW50aXR5L2NsYWltcyI+PHNhbWw6QXR0cmlidXRlVmFsdWU+dHJ1ZTwvc2FtbDpBdHRyaWJ1dGVWYWx1ZT48L3NhbWw6QXR0cmlidXRlPjwvc2FtbDpBdHRyaWJ1dGVTdGF0ZW1lbnQ+PGRzOlNpZ25hdHVyZSB4bWxuczpkcz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnIyI+PGRzOlNpZ25lZEluZm8+PGRzOkNhbm9uaWNhbGl6YXRpb25NZXRob2QgQWxnb3JpdGhtPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxLzEwL3htbC1leGMtYzE0biMiIC8+PGRzOlNpZ25hdHVyZU1ldGhvZCBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMDQveG1sZHNpZy1tb3JlI3JzYS1zaGEyNTYiIC8+PGRzOlJlZmVyZW5jZSBVUkk9IiNfZDA4MTc0NWItOTU2ZC00NmE4LTg4MzItMTQ5MjBmNmY4ZTZmIj48ZHM6VHJhbnNmb3Jtcz48ZHM6VHJhbnNmb3JtIEFsZ29yaXRobT0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnI2VudmVsb3BlZC1zaWduYXR1cmUiIC8+PGRzOlRyYW5zZm9ybSBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMTAveG1sLWV4Yy1jMTRuIyIgLz48L2RzOlRyYW5zZm9ybXM+PGRzOkRpZ2VzdE1ldGhvZCBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMDQveG1sZW5jI3NoYTI1NiIgLz48ZHM6RGlnZXN0VmFsdWU+Q1hISDVsWCtZTTROVDhFUU1tVHIyN2lBdEk1dE9XajhuQ0pSTFEzYnFQbz08L2RzOkRpZ2VzdFZhbHVlPjwvZHM6UmVmZXJlbmNlPjwvZHM6U2lnbmVkSW5mbz48ZHM6U2lnbmF0dXJlVmFsdWU+alhVNG5Ud1lQYVZWazJhcUhjNW1DMk5FWWRSdXBPV1A2Nis4bmNlbndFQmlseG81S0RmdldBNldUaDdHSGlVNUpSdCtzbWsyY1dWajRPTElnUEhIWFlFZjNkaVNYNkd2ejBqVGh4VDBzVDNmSDNob081dkt6S2dITVdNWGM5cEswU2xrQVRSenhPZmxCTHFkclQ3bnBBd3lrbEtlTFhhaTRIVlJ6MU5aN3pFNXFVaHArVmJ3Y3FCZXpkZ1pPMWlCMHkyRHdzWFRJSFhmN2hOUmFLVUYzenF2bm5IYlBNZG95cGZNbzhxU0hwcFhWNzRuN0lLQ3hHakhlYzdJak1JMWlrZVNFaTE2ODJUN0Y0cHRESXNjYmcxWW5LTTRhdHZxMWZMSi83QWx0ck90SGFMSkZBSU1jeG43TXhCckNuNXhSR2I2ZDN2c0U0VzVHN2E3MmY2OUpBPT08L2RzOlNpZ25hdHVyZVZhbHVlPjxLZXlJbmZvIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwLzA5L3htbGRzaWcjIj48WDUwOURhdGE+PFg1MDlDZXJ0aWZpY2F0ZT5NSUlHampDQ0JYYWdBd0lCQWdJUUFxQzZUbmZlVFJQeUQrN0Q2YlE4RERBTkJna3Foa2lHOXcwQkFRVUZBREJJTVFzd0NRWURWUVFHRXdKVlV6RVZNQk1HQTFVRUNoTU1SR2xuYVVObGNuUWdTVzVqTVNJd0lBWURWUVFERXhsRWFXZHBRMlZ5ZENCVFpXTjFjbVVnVTJWeWRtVnlJRU5CTUI0WERURXpNVEF4TlRBd01EQXdNRm9YRFRFME1UQXlNREV5TURBd01Gb3dkekVMTUFrR0ExVUVCaE1DVlZNeEZUQVRCZ05WQkFnVERGQmxibTV6ZVd4MllXNXBZVEVPTUF3R0ExVUVCeE1GUlhoMGIyNHhKakFrQmdOVkJBb1RIVUpsYm5Sc1pYa2dVM2x6ZEdWdGN5d2dTVzVqYjNKd2IzSmhkR1ZrTVJrd0Z3WURWUVFERXhCaGRYUm9MbUpsYm5Sc1pYa3VZMjl0TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFwemdYV29tVlpObS81NnBiUnlaeThWYlhyVUNsUWpPWU13N3dGYUZUTE9XRU9iOWhncUVWWnZOVUxLNFBmV2dmTUo4cUxxZkhodnFXbG8xSkFxRFdiUUpybFJIQVNLdUNjOWVPa3JQMzdpMW1WenNtNWxQUlBWb242R1RhUkE5SVRBRitiNW5OMDB5MHczOThNWnFJZ2xJdDRQRmNlM1NXd3lYQkdOSUpyWURNYndmRjFFQU5hU1V4Skh3eUd0eVVLVWlxeFB3SllFZ3JacHJoZHlMQnRRdXJaZGJwNkViSEduQVZJWHpKR2Y0TC9WN2tpdWExNWFmeW9Ycm5tZ2d0SGJBMEFOVkY5eXFBcG5oc0FxR2pDdHpSSGpvWG9FMFNnZFY1U3hvUm1lb3hXZWowOEZYdW96bHRLc2d5Z25ZbXZoZFB5VGl4SjVqYSttZlVxUjZCeXdJREFRQUJvNElEUXpDQ0F6OHdId1lEVlIwakJCZ3dGb0FVa0hIYk4rdHp5Ty9jMVI0U3RqUzZLMXFncHBJd0hRWURWUjBPQkJZRUZKaHM1L1d2bkYwbHFSdGhMYklwOUlLTUVwMXRNQnNHQTFVZEVRUVVNQktDRUdGMWRHZ3VZbVZ1ZEd4bGVTNWpiMjB3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CQmdnckJnRUZCUWNEQWpCaEJnTlZIUjhFV2pCWU1DcWdLS0FtaGlSb2RIUndPaTh2WTNKc015NWthV2RwWTJWeWRDNWpiMjB2YzNOallTMW5OQzVqY213d0txQW9vQ2FHSkdoMGRIQTZMeTlqY213MExtUnBaMmxqWlhKMExtTnZiUzl6YzJOaExXYzBMbU55YkRDQ0FjUUdBMVVkSUFTQ0Fic3dnZ0czTUlJQnN3WUpZSVpJQVliOWJBRUJNSUlCcERBNkJnZ3JCZ0VGQlFjQ0FSWXVhSFIwY0RvdkwzZDNkeTVrYVdkcFkyVnlkQzVqYjIwdmMzTnNMV053Y3kxeVpYQnZjMmwwYjNKNUxtaDBiVENDQVdRR0NDc0dBUVVGQndJQ01JSUJWaDZDQVZJQVFRQnVBSGtBSUFCMUFITUFaUUFnQUc4QVpnQWdBSFFBYUFCcEFITUFJQUJEQUdVQWNnQjBBR2tBWmdCcEFHTUFZUUIwQUdVQUlBQmpBRzhBYmdCekFIUUFhUUIwQUhVQWRBQmxBSE1BSUFCaEFHTUFZd0JsQUhBQWRBQmhBRzRBWXdCbEFDQUFid0JtQUNBQWRBQm9BR1VBSUFCRUFHa0Fad0JwQUVNQVpRQnlBSFFBSUFCREFGQUFMd0JEQUZBQVV3QWdBR0VBYmdCa0FDQUFkQUJvQUdVQUlBQlNBR1VBYkFCNUFHa0FiZ0JuQUNBQVVBQmhBSElBZEFCNUFDQUFRUUJuQUhJQVpRQmxBRzBBWlFCdUFIUUFJQUIzQUdnQWFRQmpBR2dBSUFCc0FHa0FiUUJwQUhRQUlBQnNBR2tBWVFCaUFHa0FiQUJwQUhRQWVRQWdBR0VBYmdCa0FDQUFZUUJ5QUdVQUlBQnBBRzRBWXdCdkFISUFjQUJ2QUhJQVlRQjBBR1VBWkFBZ0FHZ0FaUUJ5QUdVQWFRQnVBQ0FBWWdCNUFDQUFjZ0JsQUdZQVpRQnlBR1VBYmdCakFHVUFMakI0QmdnckJnRUZCUWNCQVFSc01Hb3dKQVlJS3dZQkJRVUhNQUdHR0doMGRIQTZMeTl2WTNOd0xtUnBaMmxqWlhKMExtTnZiVEJDQmdnckJnRUZCUWN3QW9ZMmFIUjBjRG92TDJOaFkyVnlkSE11WkdsbmFXTmxjblF1WTI5dEwwUnBaMmxEWlhKMFUyVmpkWEpsVTJWeWRtVnlRMEV1WTNKME1Bd0dBMVVkRXdFQi93UUNNQUF3RFFZSktvWklodmNOQVFFRkJRQURnZ0VCQUtQQUhQcGNnd0FiSXNjT2JKYm1jbVA4RzJRWkI4VVVzeVIxOFd3OTdIWUlxRXM3MExScXFyVGpza1BsZnBYanRKSm1pVjgydzZOYWZrQ3dBbWxycy94TTJFako3UkVacDZ2YmVJZHlGTjIwM3dlQkViRm9RYjg0Wk9COWRHQmR4MktZdzBuUHJSM0F1YkRQVTZJNjlhWTBFNGJIRUM2Y09nOGxjeUhkWXFyZDNSRlByUjJiN1hGWVdyaXRXeElXbHFNRUVzNVU3dkRsYjkrby9HRlZqbE1QSW4xcG0yNDBUYW41WkR0Si9NcXZZRTYrQzh1ZUtEZW1lcndiR0ZCNzFJTVI3TUU1SHYzayt3N0hFMm1ETjF0RnVjb1ZyQVVFZndYVUNzTUdzMkJJbzJsVVNGL3FIYUY3akhHdnJPUmloVFlKQnBtQzRpWmJRTGpLQmZBRXBTVT08L1g1MDlDZXJ0aWZpY2F0ZT48L1g1MDlEYXRhPjwvS2V5SW5mbz48L2RzOlNpZ25hdHVyZT48L3NhbWw6QXNzZXJ0aW9uPg==" -v 2 http://localhost:55495/rest/1
Please note that I am testing .net rest api which needs lengthy user token in authorization header to proceed.
Apache Benchmark has a maximum request size of 2048 characters. Your header alone is more than 4KB in size, so that's why your test is failing. The maximum size of the request is given in ab.c. Look for char _request[2048]; in that file and try increasing it. Note that the passed-in headers are also parsed into a header structure that may have limits as well.
You can install the ab from httpd source code, which now sets char _request[8192];.
The code can be found in httpd-2.4.53/support/ab.c
Related
Broken pipe (Write failed) when testing > max allowed Content-Length [duplicate]
This question already has an answer here:
Is possible to execute curl in Karate tests?
(1 answer)
Closed 2 years ago.
I am trying to write a test which validates my server rejects requests larger than 1MB:
Scenario: large requests are rejected
Given url 'https://my.server.com/anything'
And request "x".repeat(1048577)
When method post
Then status 413
This test fails with an javax.net.ssl.SSLException: Broken pipe (Write failed) exception because the server reads the Content-Length header and immediately rejects the request / responds with a 413 before reading the payload.
I verified the server behavior via cURL:
$> printf 'x%.0s' {1..1048577} | curl -i --data #- https://my.server.com/anything
HTTP/1.1 413 Request Entity Too Large
Is it possible to test this feature using Karate?
It certainly can be a case which Karate is not designed for. You may not have full control over some "special" headers like the Content-Length - and we are limited by the underlying Apache HTTP client. I'm not sure if the upcoming 1.0 series will support this and allow you to over-write the Content-Length header: https://github.com/intuit/karate/wiki/1.0-upgrade-guide But you are welcome to investigate and submit a PR if needed. As a workaround, you can use cURL from Karate: https://stackoverflow.com/a/64352676/143475 And also see this answer: https://stackoverflow.com/a/73230200/143475
How to configure Varnish in an API-platform project? [Response size limit issue]
Sometimes in my preproduction and production environment, the varnish container send me this error:
Error (null) Backend fetch failed
Backend fetch failed
Guru Meditation:
XID: (null)
This is due to the size of the body response.
So I did implement this test in my Postman test collection:
pm.test("Size is under 3Ko", function () {
pm.expect(pm.response.responseSize).to.be.below(3000);
});
To be sure that this error does not not appear again.
But I am wondering how can I configure it properly to accept a reasonable size of response?
This my configuration:
Api Platform 2.5.1
VCL 4.0
Varnish documentation states that the default maximum size of an HTTP response is 32 KB. You can tune this by setting the http_resp_size runtime parameter. Here's an example of an increased http_resp_size value: varnishd -p http_resp_size=1M If that doesn't help, please share the varnishlog output for that specific page, as well as the associated VCL code. If you're unsure whether or not your http_resp_size was set to the correct value, you can run the following command on your Varnish server: $ varnishadm param.show http_resp_size Hope this helps.
NiFi- how to http post a PDF document
I wanted to use NiFi's posthttp/invokeHttp processor to post a PDF to an API. But considering the following cURL request to replicate in NiFi: curl -X POST "http://ipaddress:port/api/" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "pdf_file=#sample.pdf; type=application/pdf" Which property takes the -F information in nifi attributes? Configuration for invokehttp right now: error: "400 Bad Request: The browser (or proxy) sent a request that this server could not understand." Configration for posthttp right now: error: server logs: readv() failed (104: Connection reset by peer) while reading upstream
In older version of nifi you will have to use your own script to build a multipart request and then use invoke to create post request. You can refer to this post for a ExecuteGroovyScript example. https://stackoverflow.com/a/57204862 Since Nifi 1.12 you can directly use invokeHTTP by setting content-type https://stackoverflow.com/a/69284300
When you use PostHttp/InvokeHttp you wouldn't be referencing an external file, you would be sending the content of the flow file. So you would first need to bring sample.pdf into NiFi by using GetFile or ListFile/FetchFile and then flow file coming out of those processors represents the PDF, and you would route that flow file to InvokeHttp which would POST the content of the flow file (the pdf).
What does '--user' mean with curl
I'm working with an API and I have to send a POST request. I know how to set a header (-H) and (-d) is the body, but what is "--user".
If I submit this with Postman, or in a text editor with axios or just regular XMLRequest, where do I add this?
The docs say it is for regular http auth.
curl -X POST -H "Content-Type: application/json" \
--user "<client_id>:<client_secret>" \
-d '{"grant_type": "client_credentials", "scope": "public"}' \
...
Late to the party, but here goes... You can use curl with the -v (verbose) parameter to see the headers sent. You will then see that the information provided with --user is transformed into a header, such as: Authorization: Basic YWxhZGRpbjpvcGVuc2VzYW1l The text after the Basic keyword is a base64 encoded text string of the username:password combination provided with the --user parameter To manually generate the base64 encoded credentials on Linux, you can simply call: echo -n "username:password" | base64 -w0 For windows, save the "username:password" to a file, then use certutil.exe to create a base64 encoded file: certutil -encode credentials.txt credentials.asc To test this end to end, you can remove --user username:password and substitute with --header Authorization: Basic YWxhZGRpbjpvcGVuc2VzYW1l and it will still authenticate just fine. In summary, to do this manually without curl, you would need to base64 encode username:password combination. You would then need to set the HTTP Authorization header with the type as Basic along with the base64 encoded string.
--user parameter in curl used for server authentication. So if you don't define authentication type via other parameters like --digest or --negotiate, it means USER parameter for http basic authentication, it also could be combined with :PASSWORD chunk to set a password as well. The full answer on your question depends on what kind authentication is used behind API you are sending request to, and maybe curl would not be enough for it, as it support a limited set of authentication schemes ...
--user (or -u) in curl provides a basic auth to your request. In Postman you can achieve the same result with a choice in Authorization tab. --user "<client_id>:<client_secret>" becomes Type: Basic Auth Username: client_id Password: client_secret
Specify the user name and password to use for server authentication. If you simply specify the user name, curl will prompt for a password. If your curl request does not have any -- user, then server that requires authentication sends back a 401 response code and an associated WWW-Authenticate: header that lists all the authentication methods that the server supports. < HTTP/1.1 401 < WWW-Authenticate: Basic realm="oauth2/client" Then you will know the server is using Basic authentication You can add --basic to explicitly tell it is Basic authentication Please refer to HTTP authentication for more information
Sometimes (depending on server implementation) the --user will negotiate a digest authenticated session. The headers for digest users are a one-time use. I believe a request to the server will first fail with a 401, but include a WWW-Authenticate response, including the digest realm, and the nonce secret. With these, a second request can be made with a new header Authorization value. example: Authorization: Digest username="LXAIQKBC", realm="MMS Public API", nonce="rE3sYnLXEhVMbh72JyUK7kfLIb+bAbKj", uri="/api/atlas/v1.0/groups", cnonce="YTVhM4YwMDB3ZjZjMTkxbCNiODA1ODnxZDFjOGMyMzE=", nc=00000001, qop=auth, response="7a5fcb8e4f92a665315bf62cdd87a67d", algorithm="MD5"
As an addition to Jahmic's answer, Nodejs programmers can do this to convert to base64 string:
const cryptoJS = require("crypto-js");
const base64Str = cryptoJS.enc.Base64.stringify(cryptoJS.enc.Utf8.parse(`${username}:${password}`))
Can I set multiple headers with Siege?
I want to use siege to target a number of URLs on my app, each with different headers. I can set headers for one request siege -u http://localhost/xyz -d1 -r1000 -c25 --header="Token: f2840fc1" (this appears to be undocumented) I can specify a list of URLs in the URL file, with custom headers for each URL. But I can't see a way in the docs.
I suggest using two concurrent calls to siege. Write a URL file that uses Header-A, and another for Header-B. For my API testing, I've got a get_urls.txt file and post_json_urls.txt file, which I call on two instances of Siege at once. This way one gets called with Content-Type: text/json and the other doesn't. Short of rewriting the Siege url parser, this is the only way I know to do it. For example: siege -f get_urls.txt & siege -H 'Content-Type: text/json' -f post_json_urls.txt
As far as I can see from the man page and reading around I think you are right. The only way to specify headers is on the command line using the -H --header options not in the URL file.
You can try this example siege --concurrent=5 --reps=100 --header='sdk:3.0, config:3.0,zid:0' 'https://google.com/api/REGME POST uid=a8qn&aid=43ZK0'