Authenticated api call to VALR - Python 3.8 - api

I'm trying to make an authenticated api call to VALR crypto exchange as first step towards automated trading. They provide most of the code so I thought it would be easy even as a non coding techie. The code below does actually create the correct HMAC SHA512 signature using the API Secret provided for testing but I have a problem in passing this result along to the next section of code to request balances (starting at line 17). If I cut and paste the result/displayed 'signature' and 'timestamp' (after running the code) back into the code it does in fact work. So what changes do I need to make the code automatically pick up the signature and timestamp. The user defined function appears to keep all parameters "secret" from the rest of the code, especially after using return.
import time
import hashlib
import hmac
def sign_request( api_key_secret,timestamp, verb,path,body=""):
payload = "{}{}{}{}".format(timestamp, verb.upper(), path, body)
message = bytearray(payload, 'utf-8')
signature = hmac.new(bytearray(api_key_secret, 'utf-8'), message, digestmod=hashlib.sha512).hexdigest()
print("Signature=",signature)
print ("Timestamp",timestamp)
return signature
sign_request( verb = "GET", timestamp = int(time.time()*1000),path="/v1/account/balances",api_key_secret="4961b74efac86b25cce8fbe4c9811c4c7a787b7a5996660afcc2e287ad864363" )
import requests
url = "https://api.valr.com/v1/account/balances"
payload = {}
headers = {
'X-VALR-API-KEY': '2589fb273e86aeee10bac1445232aa302feb37e27d32c1c599abc3757599139e',
'X-VALR-SIGNATURE': 'signature',
'X-VALR-TIMESTAMP': 'timestamp'
}
response = requests.request("GET", url, headers=headers, data = payload)
print(response.text.encode('utf8'))

Well after some hard thinking I decided to change to using global variables. The hmac still worked fine and gave me a signature. Then I removed the quotes around signature and timestamp and realised they were both integers. I was then able to convert that signature and timestamp to a string and everything started to work perfectly. Maybe someone else will make use of this. If you want to make a POST request remember to put single quotes around anything in the {body} statement to make it a string.

Here is the code that I am currently using for a GET request from VALR. It's been working fine for many months. You will need to change the path and the url to correspond to whatever you are trying to get, and obviously you will need to add your_api_key and your_api_secret.
If you need to send through other request parameters like transaction types etc. then you will ned to include them in the path and the url e.g. https://api.valr.com/v1/account/transactionhistory?skip=0&limit=100&transactionTypes=MARKET_BUY&currency=ZAR
def get_orders(): # to get open orders from valr
timestamp = int(time.time()*1000)
verb = "GET"
path = "/v1/orders/open"
body = ""
api_key_secret = 'your_api_secret'
payload = "{}{}{}".format(timestamp, verb.upper(), path)
message = bytearray(payload, 'utf-8')
signature = hmac.new(bytearray(api_key_secret, 'utf-8'), message, digestmod=hashlib.sha512).hexdigest()
timestamp_str = str(timestamp)
url = "https://api.valr.com/v1/orders/open"
headers = {
'Content-Type': 'application/json',
'X-VALR-API-KEY': 'your_api_key',
'X-VALR-SIGNATURE': signature,
'X-VALR-TIMESTAMP': timestamp_str,
}
response = requests.request("GET", url, headers=headers, data=body)
dict = json.loads(response.text)
dict = pd.DataFrame.from_dict(dict)
print(dict)

Related

Falcon - Difference in stream type between unittests and actual API on post

I'm trying to write unittests for my falcon api, and I encountered a really weird issue when I tried reading the body I added to the unittests.
This is my unittest:
class TestDetectionApi(DetectionApiSetUp):
def test_valid_detection(self):
headers = {"Content-Type": "application/x-www-form-urlencoded"}
body = {'test': 'test'}
detection_result = self.simulate_post('/environments/e6ce2a50-f68f-4a7a-8562-ca50822b805d/detectionEvaluations',
body=urlencode(body), headers=headers)
self.assertEqual(detection_result.json, None)
and this is the part in my API that reads the body:
def _get_request_body(request: falcon.Request) -> dict:
request_stream = request.stream.read()
request_body = json.loads(request_stream)
validate(request_body, REQUEST_VALIDATION_SCHEMA)
return request_body
Now for the weird part, my function for reading the body is working without any issue when I run the API, but when I run the unittests the stream type seems to be different which affect the reading of it.
The stream type when running the API is gunicorn.http.body.Body and using unittests: wsgiref.validate.InputWrapper.
So when reading the body from the api all I need to do it request.stream.read() but when using the unittests I need to do request.stream.input.read() which is pretty annoying since I need to change my original code to work with both cases and I don't want to do it.
Is there a way to fix this issue? Thanks!!
It seems like issue was with how I read it. instead of using stream I used bounded_stream which seemed to work, also I removed the headers and just decoded my body.
my unittest:
class TestDetectionApi(DetectionApiSetUp):
def test_valid_detection(self):
body = '''{'test': 'test'}'''
detection_result = self.simulate_post('/environments/e6ce2a50-f68f-4a7a-8562-ca50822b805d/detectionEvaluations',
body=body.encode(), headers=headers)
self.assertEqual(detection_result.json, None)
how I read it:
def _get_request_body(request: falcon.Request) -> dict:
request_stream = request.bounded_stream.read()
request_body = json.loads(request_stream)
validate(request_body, REQUEST_VALIDATION_SCHEMA)
return request_body

JXA for automation: url encoding for get request

I am trying to send a text via get request to my server, but I have some struggles with spaces/special characters.
How can I encode my text for the request?
(And how to decode it in php?)
Here is my code:
var app = Application.currentApplication();
app.includeStandardAdditions = true;
var text = app.displayDialog("enter your text:", { defaultAnswer: "" }).textReturned;
var result = JSON.parse(app.doShellScript('curl https://example.com?text=' + text));
result
Well, it looks like I found the answer myself:
encodeURIComponent(text)
seems to be working for me. PHP decoding not necessary.

How to pull data from CATSone API with Authentication Token?

I am completely new to coding. I am trying to build a dashboard in Klipfolio. I am using a CATSone API to pull data from CATSone to Klipfolio. However, I can only get 100 rows a time, which means I would have to pull data 2600 times.
I am now trying to build a script to get data from the API through Google Script Editor. However, since I have no experience in this, I am just trying stuff. I watched some videos, also from Ben Collins. The basis is simple, and I get what he is doing.
However, I have a problem with putting the API key.
var API_KEY = 'key'
function callCATSone(){
//Call the CATSone API for all candidate list
var response = UrlFetchApp.fetch("https://api.catsone.nl/v3/candidates");
Logger.log(response.getContentText());
// URL and params for the API
var url = 'https://api.catsone.nl/v3/candidates';
var params = {
'method': 'GET',
'muteHttpExceptions': true,
'headers': {
'Authorization': 'key ' + apikey
}
};
// call the API
var response = UrlFetchApp.fetch(url, params);
var data = response.getContentText();
var json = JSON.parse(data);
}
In the end, I would like to transfer all candidate list data to my sheets. Therefore, I call on the API with Authorization key. After that, I will manipulate the data, but that's for later. The first problem I now encounter, is this fail code:
'Verzoek voor https://api.catsone.nl/v3/candidates is mislukt. Foutcode: 401. Ingekorte serverreactie: {"message":"Invalid credentials."} (Gebruik de optie muteHttpExceptions om de volledige reactie te onderzoeken.) (regel 6, bestand 'Code')'.
I expect to get a list of all data from CATSone into my sheets.
Does anyone know how I can accomplish this?
Two changes should fix the credentials error:
Authorization header should be Authorization: 'Token ' + yourApiKey instead of 'key ', see the v3 API documentation https://docs.catsone.com/api/v3/#authentication.
API key in your case is stored in a global variable API_KEY, you should reference it exactly like that, not as an apikey (unless there is a typo in your sample or some missing code): Authorization : 'Token ' + API_KEY.
Btw, it should probably set either a Content-Type header or a contentType parameter for UrlFetchApp.fetch() method call to application/json as UrlFetchApp.fetch() request content type defaults to application/x-www-form-urlencoded.
If you plan to continue working with APIs, it would be beneficial to read this MDN article.

POSTMAN: Get Generated Request in test to compare to Response

I am using some of the auto generated parameters in my request body in a postman request(i.e: {{$guid}}).
I would like in my test to retrieve the request that was sent to the server to compare what this variable value was, and what the response parroted back to me me in my request.
for example, my request's body looks like this:
{
"Description": "testing this {{$guid}}"
}
and I would in the tests be able to do:
var req = JSON.parse(requestBody);
var resp = JSON.parse(responseBody);
test['description should match'] = req.Description === resp.Description;
is this doable?
This is possible.
But you have several small syntax errors.
To access the request body data use:
var req = JSON.parse(request.data);
I named the variable req to not be confused with the predefined request variable. You can log the result like this:
console.log(req.Description);
In the tests tab make sure you reference the correct variable tests with "s". Also you pass the test case name as a string e.g. "description should match".
var res = JSON.parse(responseBody);
console.log(res.Description);
tests["description should match"] = req.Description === res.Description;

Walmart Marketplace API Integration and Authentication

I am working on integrating my application Walmart Marketplace API using Ruby on Rails.
  1. if i try to generate Auth signature for multiple parameters, it does not generate it and returns exceptions. I am using a Jar file to generate Auth signature
    For e.g. -: https://marketplace.walmartapis.com/v3/orders?createdStartDate=2016-09-13&createdEndDate=2016-09-23 
Does anyone generate Auth Signature & timestamp for multiple parameter for Walmart Marketplace API
  2. Does Auth Signature & timestamp need to be generated for each API call for e.g . Pagination call Also?
Does Authentication need to do for each call?
Additional Comments
I know it is a month later and you already have your program figured out but in case you need some help with these parts or anyone else does, I thought I would include the following information I have on the Walmart API.
1.You might want to consider building a method in ruby since it'll be more interactive with the rest of your ruby program, it was kind of difficult but when I was doing it the most difficult part was wrapping the string in the with the SHA256 digest of string to sign. So I threw together a few methods and it works:
pem = make_pem('PRIVATE KEY', encodedKeyBytes)
digest = OpenSSL::Digest::SHA256.new
pkey = OpenSSL::PKey::RSA.new(pem)
signature = pkey.sign(digest, stringToSign)
def make_pem(tag, der)
box tag, Base64.strict_encode64(der).scan(/.{1,64}/)
end
def box(tag, lines)
lines.unshift "-----BEGIN #{tag}-----"
lines.push "-----END #{tag}-----"
lines.join("\n")
end
It's not perfect but ruby doesn't really have the functionality built in so you have to change it around to get it to work. If this still doesn't work feel free to contact me, but I started out using the jar they provide and I promise it is necessary when you are making thousands of different calls a day with different parameters and urls to be able to find the point of failure and if it isn't in ruby its going to be a lot harder to work with and fix.
2/3. You already answered that these need to be included in every call to the API and I don't really have anything else to add here except to not try to find a way around this, like submitting the same time stamp for a batch of calls. Even though it might work if the calls are made within a certain time window, Walmart uses the time stamp to determine which call came in last which is especially important for things like their price API. Again feel free to email me with any questions, I'll try to respond here too but I don't this website that often.
The variable names I am using these variable names just to reference the code provided in the walmart developer guide. I am just going to translate the java code there to ruby to show how I got the values for stringToSign and encodedKeyBytes.
# This is provided to you by walmart
consumerId = "b68d2a72...."
# Also provided by walmart
privateEncodedStr = "MIICeAIBADANBgkqhkiG9w0BAQEFAA......"
# Full path
baseUrl = "https://marketplace.walmartapis.com/v2/feeds"
# HTTP Method Verb
httpMethod = "GET"
timestamp = (Time.now.to_f * 1000).to_i.to_s
stringToSign = consumerId + "\n" + baseUrl + "\n" + httpMethod + "\n" + timestamp + "\n"
encodedKeyBytes = Base64.decode64(privateEncodedStr)
From there you just run it through the original code and then base64 encode the signature and remove white spaces and then you're good to make a request.
In Order to generate multiple parameter pass string as by escaping sting.
Auth Signature & timestamp need to be generated for each API call for e.g . Pagination call Also
if i try to generate Auth signature for multiple parameters, it does not generate it and returns exceptions. I am using a Jar file to generate Auth signature.
USE SHA class instead of jar file =>
It will generate signature for multiple parameters also.
import org.apache.commons.codec.binary.Base64;
import java.security.KeyFactory;
import java.security.PrivateKey;
import java.security.Signature;
import java.security.spec.PKCS8EncodedKeySpec;
public class SHA256WithRSAAlgo {
private static String consumerId = "b68d2a72...."; // Trimmed for security reason
private static String baseUrl = "https://marketplace.walmartapis.com/v2/feeds";
private static String privateEncodedStr = "MIICeAIBADANBgkqhkiG9w0BAQEFAA......"; //Trimmed for security reasons
public static void main(String[] args) {
String httpMethod = "GET";
String timestamp = String.valueOf(System.currentTimeMillis());
String stringToSign = consumerId + "\n" +
baseUrl + "\n" +
httpMethod + "\n" +
timestamp + "\n";
String signedString = SHA256WithRSAAlgo.signData(stringToSign, privateEncodedStr);
System.out.println("Signed String: " + signedString);
}
public static String signData(String stringToBeSigned, String encodedPrivateKey) {
String signatureString = null;
try {
byte[] encodedKeyBytes = Base64.decodeBase64(encodedPrivateKey);
PKCS8EncodedKeySpec privSpec = new PKCS8EncodedKeySpec(encodedKeyBytes);
KeyFactory kf = KeyFactory.getInstance("RSA");
PrivateKey myPrivateKey = kf.generatePrivate(privSpec);
Signature signature = Signature.getInstance("SHA256withRSA");
signature.initSign(myPrivateKey);
byte[] data = stringToBeSigned.getBytes("UTF-8");
signature.update(data);
byte[] signedBytes = signature.sign();
signatureString = Base64.encodeBase64String(signedBytes);
} catch (Exception e) {
e.printStackTrace();
}
return signatureString;
}
}
Does Auth Signature & timestamps need to be generated for each API call for e.g . Pagination call Also?
YES, for each and every call including pagination , you need to generate new Signature and Timestamps.
Does Authentication need to do for each call?
YES, Authentication need to do for each call.