Falcon - Difference in stream type between unittests and actual API on post - falconframework

I'm trying to write unittests for my falcon api, and I encountered a really weird issue when I tried reading the body I added to the unittests.
This is my unittest:
class TestDetectionApi(DetectionApiSetUp):
def test_valid_detection(self):
headers = {"Content-Type": "application/x-www-form-urlencoded"}
body = {'test': 'test'}
detection_result = self.simulate_post('/environments/e6ce2a50-f68f-4a7a-8562-ca50822b805d/detectionEvaluations',
body=urlencode(body), headers=headers)
self.assertEqual(detection_result.json, None)
and this is the part in my API that reads the body:
def _get_request_body(request: falcon.Request) -> dict:
request_stream = request.stream.read()
request_body = json.loads(request_stream)
validate(request_body, REQUEST_VALIDATION_SCHEMA)
return request_body
Now for the weird part, my function for reading the body is working without any issue when I run the API, but when I run the unittests the stream type seems to be different which affect the reading of it.
The stream type when running the API is gunicorn.http.body.Body and using unittests: wsgiref.validate.InputWrapper.
So when reading the body from the api all I need to do it request.stream.read() but when using the unittests I need to do request.stream.input.read() which is pretty annoying since I need to change my original code to work with both cases and I don't want to do it.
Is there a way to fix this issue? Thanks!!

It seems like issue was with how I read it. instead of using stream I used bounded_stream which seemed to work, also I removed the headers and just decoded my body.
my unittest:
class TestDetectionApi(DetectionApiSetUp):
def test_valid_detection(self):
body = '''{'test': 'test'}'''
detection_result = self.simulate_post('/environments/e6ce2a50-f68f-4a7a-8562-ca50822b805d/detectionEvaluations',
body=body.encode(), headers=headers)
self.assertEqual(detection_result.json, None)
how I read it:
def _get_request_body(request: falcon.Request) -> dict:
request_stream = request.bounded_stream.read()
request_body = json.loads(request_stream)
validate(request_body, REQUEST_VALIDATION_SCHEMA)
return request_body

Related

get the params value from a variable

i have one feature file as
Feature: Getting the Token
Background:
header Content-Type 'application/json'
def CookieGenerator = Java.type('com.ade.Helpers.CookiesGenerator');
def endpoints read('classpath: src/test/java/com/ade/resources/endpoints.json')
Given url endpoints.token
Scenario: To check the Schema of the response
Given cookies (new CookieGenerator().getCookieValue())
When method GET
Then status 200
def txnToken = response
#print token
from above code i am getting Token's value as something like this "gdjsgjshjhsjfhsg646"
now i have another feature file where i have to use above Token's value in my query parameter value as
Feature: Testing datent Name and Client
Background:
header Content-Type 'application/json""
def endpoints read('classpath:src/test/java/com/ade/resources/endpoints.json") def CookieGenerator Java.type('com.ade.Helpers.CookiesGenerator");
call read('Token.feature')
Given url baseUrl+endpoints.dit.Client.path
Scenario: To check the Schema of the response
Given def head read('classpath:src/test/java/com/ade/resources/reqpay.json") =
def req head.data[1]
And cookies (new CookieGenerator().getCookieValue())
And request req
And param {txntoken = txnToken}
When method post
Then status 200
from above my endpoint should be like https://something.com/clients?txntoken='gdjsgjshjhsjfhsg646'
but i am getting as https://something.com/clients?txntoken=txnToken
https://something.com/clients?txntoken='gdjsgjshjhsjfhsg646'
Your post is hard to read, as #peter-thomas said, please try formatting it better in the future, or edit the post if I haven't answered your question.
I believe what you're looking for is described in the documentation here
* def signIn = call read('classpath:my-signin.feature') { username: 'john', password: 'secret' }
* def authToken = signIn.authToken
you can see how information can be passed
I also asked a similar question fairly recently here
relevant bit here:
* def key = karate.call('ReadRoundUpSubscription.feature');
* def keyvalue = key.acckey
i prefer to call features like this, and not defining things in the reusable feature.

How do you make HTTP requests with Raku?

How do you make HTTP requests with Raku? I'm looking for the equivalent of this Python code:
import requests
headers = {"User-Agent": "python"}
url = "http://example.com/"
payload = {"hello": "world"}
res = requests.get(url, headers=headers)
res = requests.post(url, headers=headers, json=payload)
You may want to try out the recent HTTP::Tiny module.
use HTTP::Tiny;
my $response = HTTP::Tiny.new.get( 'https://example.com/' );
say $response<content>.decode
After searching around a bit, I found an answer in the Cro docs.
use Cro::HTTP::Client;
my $resp = await Cro::HTTP::Client.get('https://api.github.com/');
my $body = await $resp.body;
# `$body` is a hash
say $body;
There's more information on headers and POST requests in the link.
I want to contribute a little more. There is a fantastic module named WWW.
It's very convenient to make 'gets' that receive json because it can be parsed automagically.
In their example:
use WWW;
my $response = jget('https://httpbin.org/get?foo=42&bar=x');
You can examine the objects using the basic functionalities of arrays and hashes, for example to extract the values of my response you can use:
$response<object_you_want_of_json><other_nested_object>[1]<the_last_level>
Here the number [1] are a nested list inside a hash, and the properties are the same. Welcome to the raku community !!!

Authenticated api call to VALR - Python 3.8

I'm trying to make an authenticated api call to VALR crypto exchange as first step towards automated trading. They provide most of the code so I thought it would be easy even as a non coding techie. The code below does actually create the correct HMAC SHA512 signature using the API Secret provided for testing but I have a problem in passing this result along to the next section of code to request balances (starting at line 17). If I cut and paste the result/displayed 'signature' and 'timestamp' (after running the code) back into the code it does in fact work. So what changes do I need to make the code automatically pick up the signature and timestamp. The user defined function appears to keep all parameters "secret" from the rest of the code, especially after using return.
import time
import hashlib
import hmac
def sign_request( api_key_secret,timestamp, verb,path,body=""):
payload = "{}{}{}{}".format(timestamp, verb.upper(), path, body)
message = bytearray(payload, 'utf-8')
signature = hmac.new(bytearray(api_key_secret, 'utf-8'), message, digestmod=hashlib.sha512).hexdigest()
print("Signature=",signature)
print ("Timestamp",timestamp)
return signature
sign_request( verb = "GET", timestamp = int(time.time()*1000),path="/v1/account/balances",api_key_secret="4961b74efac86b25cce8fbe4c9811c4c7a787b7a5996660afcc2e287ad864363" )
import requests
url = "https://api.valr.com/v1/account/balances"
payload = {}
headers = {
'X-VALR-API-KEY': '2589fb273e86aeee10bac1445232aa302feb37e27d32c1c599abc3757599139e',
'X-VALR-SIGNATURE': 'signature',
'X-VALR-TIMESTAMP': 'timestamp'
}
response = requests.request("GET", url, headers=headers, data = payload)
print(response.text.encode('utf8'))
Well after some hard thinking I decided to change to using global variables. The hmac still worked fine and gave me a signature. Then I removed the quotes around signature and timestamp and realised they were both integers. I was then able to convert that signature and timestamp to a string and everything started to work perfectly. Maybe someone else will make use of this. If you want to make a POST request remember to put single quotes around anything in the {body} statement to make it a string.
Here is the code that I am currently using for a GET request from VALR. It's been working fine for many months. You will need to change the path and the url to correspond to whatever you are trying to get, and obviously you will need to add your_api_key and your_api_secret.
If you need to send through other request parameters like transaction types etc. then you will ned to include them in the path and the url e.g. https://api.valr.com/v1/account/transactionhistory?skip=0&limit=100&transactionTypes=MARKET_BUY&currency=ZAR
def get_orders(): # to get open orders from valr
timestamp = int(time.time()*1000)
verb = "GET"
path = "/v1/orders/open"
body = ""
api_key_secret = 'your_api_secret'
payload = "{}{}{}".format(timestamp, verb.upper(), path)
message = bytearray(payload, 'utf-8')
signature = hmac.new(bytearray(api_key_secret, 'utf-8'), message, digestmod=hashlib.sha512).hexdigest()
timestamp_str = str(timestamp)
url = "https://api.valr.com/v1/orders/open"
headers = {
'Content-Type': 'application/json',
'X-VALR-API-KEY': 'your_api_key',
'X-VALR-SIGNATURE': signature,
'X-VALR-TIMESTAMP': timestamp_str,
}
response = requests.request("GET", url, headers=headers, data=body)
dict = json.loads(response.text)
dict = pd.DataFrame.from_dict(dict)
print(dict)

How to access Parent imported functions from Module

I'm trying to create a test helper for testing an app that uses the Maru Framework. This is a simplified version of what I'm trying to achieve:
defmodule App.ExtendedMaru do
#moduledoc false
defmacro __using__(opts) do
quote do
use Maru.Test, unquote(opts)
end
end
def post_body(url, body) do
build_conn()
|> Plug.Conn.put_req_header("content-type", "application/json")
|> put_body_or_params(Poison.encode! body)
|> post(url)
end
end
The issues are with the build_conn/0 and post/2 functions. build_conn/0 is defined in Maru.Test and can thus be reached with import Maru.Test inside this module.
However, post/2 is a private function defined inside the __using__ macro for Maru.Test. So, it is present in the module that uses this one, but it's not available to post_body/2. I can't just use Maru.Test here as I'm required to pass the opts and haven't found a way to do so.
Is it possible to access the post/2 function that should be defined in the module that's including this one?
EDIT: How the code ended up:
defmodule Legacy.ExtendedMaru do
#moduledoc """
Adds a few extra function helpers for testing using the Maru framework. Use
it as you would Maru.Test, ie:
`use Legacy.ExtendedMaru, for: An.Api.Module`
"""
defmacro __using__(opts) do
quote do
use Maru.Test, unquote(opts)
unquote(add_post_body())
end
end
defp add_post_body() do
quote do
#doc """
Makes a POST request with the given body. Correctly encodes the body in the
requested format and sets Content-Type headers.
## Parameters
- url: The URL for the POST request
- body: The body to send on the request
- opts: For customizing function behaviour:
- format: What format to send the body in. Defaults to 'json'.
"""
#spec post_body(String.t, map(), keyword(String.t)) :: Plug.Conn.t
def post_body(url, body, opts \\ []) do
format = opts[:format] || "json"
build_conn()
|> add_content(body, format)
|> post(url)
end
def add_content(conn, body, "json") do
Plug.Conn.put_req_header(conn, "content-type", "application/json")
|> put_body_or_params(Poison.encode! body)
end
end
end
end
Your problem arises from two separate things:
The post_body method is private, so you can't call it outside of your Module.
The post_body method is not in the __using__ macro so it's not available to any of the other modules that use it.
There are two simple solutions to this:
Move the post_body method inside the __using__ macro. Once you do this, all modules that use it, will be able to call post_body except the original module.
(or) Make the post_body method public and call defdelegate on it inside __using__. This way you'll be able to call it in all modules including the original one.

Play Framework 2, Rest Services and gzip decompression

I'm facing what seems a charset issue of play when decompressing gzip content from rest services. When I try to run the code snippet below, an error is thrown, saying "Malformed JSON. Illegal character ((CTRL-CHAR, code 31))":
val url:String = "https://api.stackexchange.com/2.0/info?site=stackoverflow"
Async {
WS.url(url)
.withHeaders("Accept-Encoding" -> "gzip, deflate")
.get()
.map { response =>
Ok("Response: " + (response.json \ "items"))
}
}
At first I thought it would be a problem in StackExchange API itself, but I tried a similar service, which uses gzip compression as well, and the same error happens. It's hard to fix the code because I don't even know where is the "Illegal character". Is there something missing or it's actually a bug in play?
The clue I can provide is that the first byte of a gzip stream is 31 (0x1f). So you probably need to do something else to cause the gzip stream to be decompressed.
By the way, I recommend that you not accept deflate encoding, just gzip.
Here is how it can be done with Play 2.3
// set Http compression: https://www.playframework.com/documentation/2.3.x/ScalaWS
val clientConfig = new DefaultWSClientConfig()
val secureDefaults: AsyncHttpClientConfig = new NingAsyncHttpClientConfigBuilder(clientConfig).build()
val builder = new AsyncHttpClientConfig.Builder(secureDefaults)
builder.setCompressionEnabled(true)
val secureDefaultsWithSpecificOptions: AsyncHttpClientConfig = builder.build()
implicit val implicitClient = new NingWSClient(secureDefaultsWithSpecificOptions)
val response = WS.clientUrl("http://host/endpoint/item").withHeaders(("Accepts-encoding", "gzip")).get()