This question already has an answer here:
Is possible to execute curl in Karate tests?
(1 answer)
Closed 2 years ago.
I am trying to write a test which validates my server rejects requests larger than 1MB:
Scenario: large requests are rejected
Given url 'https://my.server.com/anything'
And request "x".repeat(1048577)
When method post
Then status 413
This test fails with an javax.net.ssl.SSLException: Broken pipe (Write failed) exception because the server reads the Content-Length header and immediately rejects the request / responds with a 413 before reading the payload.
I verified the server behavior via cURL:
$> printf 'x%.0s' {1..1048577} | curl -i --data #- https://my.server.com/anything
HTTP/1.1 413 Request Entity Too Large
Is it possible to test this feature using Karate?
It certainly can be a case which Karate is not designed for. You may not have full control over some "special" headers like the Content-Length - and we are limited by the underlying Apache HTTP client.
I'm not sure if the upcoming 1.0 series will support this and allow you to over-write the Content-Length header: https://github.com/intuit/karate/wiki/1.0-upgrade-guide
But you are welcome to investigate and submit a PR if needed.
As a workaround, you can use cURL from Karate: https://stackoverflow.com/a/64352676/143475
And also see this answer: https://stackoverflow.com/a/73230200/143475
Related
The app that I am trying to test has an SFTP server that can be queried via API. Swagger shows the following sample cURL request to get domain files information, and I am having a hard time making this call with Karate:
curl -X GET -H 'Accept: application/json' 'https://{host}:{port}/api/{clientId}/'
I do the following where baseUrl is defined as https://sftp.mydomain.com:22 where 22 is the port number that I can successfully use to connect to the SFTP server via Cyberduck:
Feature:
Background:
* url baseUrl
* def moduleBase = '/api/12345/'
Scenario:
* path moduleBase
When method get
Then status 200
The error that I get is this:
ERROR com.intuit.karate - src/test/java/mytest.feature:9
When method get
http call failed after 815 milliseconds for url: https://sftp.mydomain.com:22/api/12345/
What am I doing wrong?
Clearly it may not be HTTP so I don't think Swagger and all is legit.
Maybe you can just delegate to the OS. Refer: https://stackoverflow.com/a/64352676/143475
I wanted to use NiFi's posthttp/invokeHttp processor to post a PDF to an API.
But considering the following cURL request to replicate in NiFi:
curl -X POST "http://ipaddress:port/api/" -H "accept: application/json" -H
"Content-Type: multipart/form-data" -F "pdf_file=#sample.pdf;
type=application/pdf"
Which property takes the -F information in nifi attributes?
Configuration for invokehttp right now:
error:
"400 Bad Request: The browser (or proxy) sent a request that this server could not understand."
Configration for posthttp right now:
error:
server logs: readv() failed (104: Connection reset by peer) while reading upstream
In older version of nifi you will have to use your own script to build a multipart request and then use invoke to create post request. You can refer to this post for a ExecuteGroovyScript example.
https://stackoverflow.com/a/57204862
Since Nifi 1.12 you can directly use invokeHTTP by setting content-type
https://stackoverflow.com/a/69284300
When you use PostHttp/InvokeHttp you wouldn't be referencing an external file, you would be sending the content of the flow file. So you would first need to bring sample.pdf into NiFi by using GetFile or ListFile/FetchFile and then flow file coming out of those processors represents the PDF, and you would route that flow file to InvokeHttp which would POST the content of the flow file (the pdf).
How would I go about translating the following curl command to a Jitterbit operation?
curl -i -u username:password -X POST -F file=#/path/to/file.csv
https://website.com/api/filepost
Currently I have my Operation structured as follows:
Script:
$jitterbit.target.http.form_data = true;
$jitterbit.target.http.form_data.filename = "file.csv";
$jitterbit.target.http.form_data.name = "file";
Source
A CSV file without headers, which matches the API's specifications (sent the same file successfully via curl)
Transformation:
Text to Text - both source and target use the same file format as the Source file
API Endpoint
Currently I authenticate successfully, but I get a 400/Bad Request error message saying "No file attached".
Full error message:
The operation "2. POST Preapplicants - CSV to API" failed.
Fatal Error
Failed to post to the url 'https://website.com/api/filepost’.
The last (and probably most relevant) error was: The server
returned HTTP Status Code : 400 Bad Request Error is: The
request could not be processed by the server due to invalid
syntax. Headers sent by the server: HTTP/1.1 400 Bad Request
Server: nginx/1.10.3 (Ubuntu) Content-Type: application/json
Transfer-Encoding: chunked Connection: keep-alive Cache-Control:
no-cache Date: Tue, 12 Sep 2017 18:55:38 GMT The response was:
{"message":"No file attached."}
I solved this problem by doing the following:
1. Changing from a transformation operation to an archive operation (using the same source, target, and script)
2. Changing the content-type of my HTTP connection to multipart/form-data (the default content-type passed by curl)
you can do it that way. what I did was use a file list operation stored that into an array then uses a Base64EncodeFile() function to upload the file
Apache benchmark is failing when using lengthy custom authorization header -
ab.exe -n 1 -c 1 -S -H "Authorization: SAML PHRyaWJ1dGUgQXR0cmlidXRlTmFtZT0iaGFzX3NlbGVjdCIgQXR0cmlidXRlTmFtZXNwYWNlPSJodHRwOi8vc2NoZW1hcy5iZW50bGV5LmNvbS93cy8yMDExLzAzL2lkZW50aXR5L2NsYWltcyI+PHNhbWw6QXR0cmlidXRlVmFsdWU+dHJ1ZTwvc2FtbDpBdHRyaWJ1dGVWYWx1ZT48L3NhbWw6QXR0cmlidXRlPjwvc2FtbDpBdHRyaWJ1dGVTdGF0ZW1lbnQ+PGRzOlNpZ25hdHVyZSB4bWxuczpkcz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnIyI+PGRzOlNpZ25lZEluZm8+PGRzOkNhbm9uaWNhbGl6YXRpb25NZXRob2QgQWxnb3JpdGhtPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxLzEwL3htbC1leGMtYzE0biMiIC8+PGRzOlNpZ25hdHVyZU1ldGhvZCBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMDQveG1sZHNpZy1tb3JlI3JzYS1zaGEyNTYiIC8+PGRzOlJlZmVyZW5jZSBVUkk9IiNfZDA4MTc0NWItOTU2ZC00NmE4LTg4MzItMTQ5MjBmNmY4ZTZmIj48ZHM6VHJhbnNmb3Jtcz48ZHM6VHJhbnNmb3JtIEFsZ29yaXRobT0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnI2VudmVsb3BlZC1zaWduYXR1cmUiIC8+PGRzOlRyYW5zZm9ybSBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMTAveG1sLWV4Yy1jMTRuIyIgLz48L2RzOlRyYW5zZm9ybXM+PGRzOkRpZ2VzdE1ldGhvZCBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMDQveG1sZW5jI3NoYTI1NiIgLz48ZHM6RGlnZXN0VmFsdWU+Q1hISDVsWCtZTTROVDhFUU1tVHIyN2lBdEk1dE9XajhuQ0pSTFEzYnFQbz08L2RzOkRpZ2VzdFZhbHVlPjwvZHM6UmVmZXJlbmNlPjwvZHM6U2lnbmVkSW5mbz48ZHM6U2lnbmF0dXJlVmFsdWU+alhVNG5Ud1lQYVZWazJhcUhjNW1DMk5FWWRSdXBPV1A2Nis4bmNlbndFQmlseG81S0RmdldBNldUaDdHSGlVNUpSdCtzbWsyY1dWajRPTElnUEhIWFlFZjNkaVNYNkd2ejBqVGh4VDBzVDNmSDNob081dkt6S2dITVdNWGM5cEswU2xrQVRSenhPZmxCTHFkclQ3bnBBd3lrbEtlTFhhaTRIVlJ6MU5aN3pFNXFVaHArVmJ3Y3FCZXpkZ1pPMWlCMHkyRHdzWFRJSFhmN2hOUmFLVUYzenF2bm5IYlBNZG95cGZNbzhxU0hwcFhWNzRuN0lLQ3hHakhlYzdJak1JMWlrZVNFaTE2ODJUN0Y0cHRESXNjYmcxWW5LTTRhdHZxMWZMSi83QWx0ck90SGFMSkZBSU1jeG43TXhCckNuNXhSR2I2ZDN2c0U0VzVHN2E3MmY2OUpBPT08L2RzOlNpZ25hdHVyZVZhbHVlPjxLZXlJbmZvIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwLzA5L3htbGRzaWcjIj48WDUwOURhdGE+PFg1MDlDZXJ0aWZpY2F0ZT5NSUlHampDQ0JYYWdBd0lCQWdJUUFxQzZUbmZlVFJQeUQrN0Q2YlE4RERBTkJna3Foa2lHOXcwQkFRVUZBREJJTVFzd0NRWURWUVFHRXdKVlV6RVZNQk1HQTFVRUNoTU1SR2xuYVVObGNuUWdTVzVqTVNJd0lBWURWUVFERXhsRWFXZHBRMlZ5ZENCVFpXTjFjbVVnVTJWeWRtVnlJRU5CTUI0WERURXpNVEF4TlRBd01EQXdNRm9YRFRFME1UQXlNREV5TURBd01Gb3dkekVMTUFrR0ExVUVCaE1DVlZNeEZUQVRCZ05WQkFnVERGQmxibTV6ZVd4MllXNXBZVEVPTUF3R0ExVUVCeE1GUlhoMGIyNHhKakFrQmdOVkJBb1RIVUpsYm5Sc1pYa2dVM2x6ZEdWdGN5d2dTVzVqYjNKd2IzSmhkR1ZrTVJrd0Z3WURWUVFERXhCaGRYUm9MbUpsYm5Sc1pYa3VZMjl0TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFwemdYV29tVlpObS81NnBiUnlaeThWYlhyVUNsUWpPWU13N3dGYUZUTE9XRU9iOWhncUVWWnZOVUxLNFBmV2dmTUo4cUxxZkhodnFXbG8xSkFxRFdiUUpybFJIQVNLdUNjOWVPa3JQMzdpMW1WenNtNWxQUlBWb242R1RhUkE5SVRBRitiNW5OMDB5MHczOThNWnFJZ2xJdDRQRmNlM1NXd3lYQkdOSUpyWURNYndmRjFFQU5hU1V4Skh3eUd0eVVLVWlxeFB3SllFZ3JacHJoZHlMQnRRdXJaZGJwNkViSEduQVZJWHpKR2Y0TC9WN2tpdWExNWFmeW9Ycm5tZ2d0SGJBMEFOVkY5eXFBcG5oc0FxR2pDdHpSSGpvWG9FMFNnZFY1U3hvUm1lb3hXZWowOEZYdW96bHRLc2d5Z25ZbXZoZFB5VGl4SjVqYSttZlVxUjZCeXdJREFRQUJvNElEUXpDQ0F6OHdId1lEVlIwakJCZ3dGb0FVa0hIYk4rdHp5Ty9jMVI0U3RqUzZLMXFncHBJd0hRWURWUjBPQkJZRUZKaHM1L1d2bkYwbHFSdGhMYklwOUlLTUVwMXRNQnNHQTFVZEVRUVVNQktDRUdGMWRHZ3VZbVZ1ZEd4bGVTNWpiMjB3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CQmdnckJnRUZCUWNEQWpCaEJnTlZIUjhFV2pCWU1DcWdLS0FtaGlSb2RIUndPaTh2WTNKc015NWthV2RwWTJWeWRDNWpiMjB2YzNOallTMW5OQzVqY213d0txQW9vQ2FHSkdoMGRIQTZMeTlqY213MExtUnBaMmxqWlhKMExtTnZiUzl6YzJOaExXYzBMbU55YkRDQ0FjUUdBMVVkSUFTQ0Fic3dnZ0czTUlJQnN3WUpZSVpJQVliOWJBRUJNSUlCcERBNkJnZ3JCZ0VGQlFjQ0FSWXVhSFIwY0RvdkwzZDNkeTVrYVdkcFkyVnlkQzVqYjIwdmMzTnNMV053Y3kxeVpYQnZjMmwwYjNKNUxtaDBiVENDQVdRR0NDc0dBUVVGQndJQ01JSUJWaDZDQVZJQVFRQnVBSGtBSUFCMUFITUFaUUFnQUc4QVpnQWdBSFFBYUFCcEFITUFJQUJEQUdVQWNnQjBBR2tBWmdCcEFHTUFZUUIwQUdVQUlBQmpBRzhBYmdCekFIUUFhUUIwQUhVQWRBQmxBSE1BSUFCaEFHTUFZd0JsQUhBQWRBQmhBRzRBWXdCbEFDQUFid0JtQUNBQWRBQm9BR1VBSUFCRUFHa0Fad0JwQUVNQVpRQnlBSFFBSUFCREFGQUFMd0JEQUZBQVV3QWdBR0VBYmdCa0FDQUFkQUJvQUdVQUlBQlNBR1VBYkFCNUFHa0FiZ0JuQUNBQVVBQmhBSElBZEFCNUFDQUFRUUJuQUhJQVpRQmxBRzBBWlFCdUFIUUFJQUIzQUdnQWFRQmpBR2dBSUFCc0FHa0FiUUJwQUhRQUlBQnNBR2tBWVFCaUFHa0FiQUJwQUhRQWVRQWdBR0VBYmdCa0FDQUFZUUJ5QUdVQUlBQnBBRzRBWXdCdkFISUFjQUJ2QUhJQVlRQjBBR1VBWkFBZ0FHZ0FaUUJ5QUdVQWFRQnVBQ0FBWWdCNUFDQUFjZ0JsQUdZQVpRQnlBR1VBYmdCakFHVUFMakI0QmdnckJnRUZCUWNCQVFSc01Hb3dKQVlJS3dZQkJRVUhNQUdHR0doMGRIQTZMeTl2WTNOd0xtUnBaMmxqWlhKMExtTnZiVEJDQmdnckJnRUZCUWN3QW9ZMmFIUjBjRG92TDJOaFkyVnlkSE11WkdsbmFXTmxjblF1WTI5dEwwUnBaMmxEWlhKMFUyVmpkWEpsVTJWeWRtVnlRMEV1WTNKME1Bd0dBMVVkRXdFQi93UUNNQUF3RFFZSktvWklodmNOQVFFRkJRQURnZ0VCQUtQQUhQcGNnd0FiSXNjT2JKYm1jbVA4RzJRWkI4VVVzeVIxOFd3OTdIWUlxRXM3MExScXFyVGpza1BsZnBYanRKSm1pVjgydzZOYWZrQ3dBbWxycy94TTJFako3UkVacDZ2YmVJZHlGTjIwM3dlQkViRm9RYjg0Wk9COWRHQmR4MktZdzBuUHJSM0F1YkRQVTZJNjlhWTBFNGJIRUM2Y09nOGxjeUhkWXFyZDNSRlByUjJiN1hGWVdyaXRXeElXbHFNRUVzNVU3dkRsYjkrby9HRlZqbE1QSW4xcG0yNDBUYW41WkR0Si9NcXZZRTYrQzh1ZUtEZW1lcndiR0ZCNzFJTVI3TUU1SHYzayt3N0hFMm1ETjF0RnVjb1ZyQVVFZndYVUNzTUdzMkJJbzJsVVNGL3FIYUY3akhHdnJPUmloVFlKQnBtQzRpWmJRTGpLQmZBRXBTVT08L1g1MDlDZXJ0aWZpY2F0ZT48L1g1MDlEYXRhPjwvS2V5SW5mbz48L2RzOlNpZ25hdHVyZT48L3NhbWw6QXNzZXJ0aW9uPg==" -v 2 http://localhost:55495/rest/1
Please note that I am testing .net rest api which needs lengthy user token in authorization header to proceed.
Apache Benchmark has a maximum request size of 2048 characters. Your header alone is more than 4KB in size, so that's why your test is failing. The maximum size of the request is given in ab.c. Look for char _request[2048]; in that file and try increasing it. Note that the passed-in headers are also parsed into a header structure that may have limits as well.
You can install the ab from httpd source code, which now sets char _request[8192];.
The code can be found in httpd-2.4.53/support/ab.c
Currently we have an Apache 2.2.3 server with mod_ssl 2.2.3 running Django, with users authenticating by using a x509 certificate.
So far the system is running perfectly except for a single user, who when trying to upload a file receives 400 Bad Request error, and the contents of the ssl_error_log regarding this operation are:
[<date>] [error] [client <client ip>] request failed: error reading the headers, referer: <referrer url>
The contents of the ssl_access_log are:
<client ip> - - [<date>] "POST <target page> HTTP/1.1" 400 321
Also, the user's browser is Firefox as far as I know.
I am completely unable to reproduce this bug and so far none of the other users have experienced it. Could you point out some reasons for this to happen?
I've experienced connectivity that stops the upstream after an X amount of bytes is sent. X was a pretty low value, as in enough to request some simple pages, but not to deal with ajax requests much less upload files. As far as I recall, this connectivity problem occurred only when tethering (from a specific Android phone, but I didnt even test other phones).
So if the upstream gets interrupted and the upload stalls, it makes sense apache would return this error, according to this post: "Apache waits a time equal to the Timeout directive (defaults to 5 minutes if not defined) for a response from the client. It is likely Apache is waiting for the CRLF that indicates the end of the headers, yet it is never received.."