Chaining REST calls in a pipeline while managing errors - error-handling

Coming from nodejs where I could chain asynchronous events using Promises and then operator I'm trying to explore how things are done in idiomatic F#.
The calls I'm trying to chain are HTTP rest calls on some entity from creation to update to uploading images to publishing.
Function composition says the output of one function should match the input of the second one to be composed and that common input and output in my case will be string, i.e. JSON serialized string as input and output of all of these functions.
I've learned that you can compose functions using >> operator. Ideally functions should not throw errors but things happen with IO, for instance in this case what if the id of the entity I'm trying to create exists etc.
The unknown and the question is what happens if an error occurs during the chained sequence, how the caller will know what went wrong along with description message? The operation could fail in the middle or towards the end or right in the beginning of the chain sequence.
What I'm expecting from these functions upon error to stop executing the chain and return the error message to the caller. Error message is also a JSON string so there's no incompatibility between inputs and outputs of a function so you know.
I looked at Choice too but not sure if that's the direction I should be going for.
The code is not necessarily complete and all I'm looking for a is a direction to research further to get answers and possibly improve this question. Here's some code to start with.
let create schema =
// POST request
"""{"id": 1, "title": "title 1"}""" // result output
let update schema =
// PUT request, update title
"""{"id": 1, "title": "title 2"}""" // output
let upload schema =
// PUT request, upload image and add thumbnail to json
"""{"id": 1, "title": "title 2", "thumbnail": "image.jpg"}"""
let publish schema =
// PUT request, publish the entity, add url for the entity
if response.StatusCode <> HttpStatusCode.OK then
"""{"code": "100", "message": "file size above limit"}"""
else
"""{"id": 1, "title": "title 2", "thumbnail": "image.jpg", "url": "http://example.com/1"}"""
let chain = create >> update >> upload >> publish
Edit - Attempt
Trying to parameterize the image thumbnail in the upload part
let create (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 1"}""" // result output
let update (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 2"}""" // output
let upload2 (img: string) (schema: string) : Result<string,string> =
printf "upload image %s\n" img
let statusCode = HttpStatusCode.OK
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg"}"""
| x -> Error (sprintf "%A happened" x)
let publish (schema: string) =
let statusCode = HttpStatusCode.InternalServerError
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg", "url": "http://example.com/1"}"""
| _ -> Error """{"code": "100", "message": "couldn't publish, file size above limit"}"""
let chain = create >> Result.bind update >> Result.bind (upload2 "image.jpg") >> Result.bind publish

A good general approach to this problem is wrapping your functions' return values in a Choice/Either-like type and using a higher-order function to bind them together such that a failure propagates/short-circuits with some meaningful data. F# has a Result type with a bind function that can be used like this:
type MyResult = Result<string,string>
let f1 x : MyResult = printfn "%s" x; Ok "yep"
let f2 x : MyResult = printfn "%s" x; Ok "yep!"
let f3 x : MyResult = printfn "%s" x; Error "nope :("
let fAll = f1 >> Result.bind f2 >> Result.bind f3
> fAll "howdy";;
howdy
yep
yep!
[<Struct>]
val it : Result<string,string> = Error "nope :("
The first two functions succeed, but the third fails and so you get an Error value back.
Also check out this article on Railway-oriented programming.
Update to be more specific to your example:
let create (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 1"}""" // result output
let update (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 2"}""" // output
let upload (schema: string) =
let statusCode = HttpStatusCode.OK
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg"}"""
| x -> Error (sprintf "%A happened" x)
let publish (schema: string) =
let statusCode = HttpStatusCode.InternalServerError
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg", "url": "http://example.com/1"}"""
| _ -> Error """{"code": "100", "message": "file size above limit"}"""
let chain =
create >> Result.bind update >> Result.bind upload >> Result.bind publish

Related

how to parse data from json file in flutter?

Error on line 60, column 2 of pubspec.yaml: Expected a key while parsing a block mapping.
â•·
60 │ assets:
│ ^
╵
Process finished with exit code 65
i need to show data from json file
you can user dart:convert API. you can use JsonCodec class to parse data from json file.
here is an example .
{
"name": "John Doe",
"age": 30,
"email": "johndoe#example.com"
}
String jsonData = await rootBundle.loadString('assets/data.json');
dynamic data = json.decode(jsonData);
String name = data['name'];
int age = data['age'];
String email = data['email'];

set pagesize limit for Getrequest using Java in Rally rest API

THe question i have is i am trying to get all the userstories related to a release, i created a query request for type release and got the ref to workproduct.
so i got something like this from the query response "https://rally1.rallydev.com/slm/webservice/v2.0/Release/12345678/WorkProducts"
Now I am use this in my GET REQUEST to get all the user stories related to the Release.
The response is giving only 20 user stories since the default page size is 20.
Is there a way to increase the page size for get request.
the code is something like this :
QueryRequest iterationRequest = new QueryRequest("release");
iterationRequest.setFetch(new Fetch("Name", "Workspace", "WorkProducts","Feature"));
iterationRequest.setWorkspace("/workspace/12345678");
iterationRequest.setProject("/project/4567890");
iterationRequest.setPageSize(200);
iterationRequest.setQueryFilter(new QueryFilter("Name", "=", Release));
QueryResponse iterationQueryResponse = restApi.query(iterationRequest);
JsonArray iterationQueryResults = iterationQueryResponse.getResults();
JsonElement iterationQueryElement = iterationQueryResults.get(0);
JsonObject iterationQueryObject = iterationQueryElement.getAsJsonObject();
JsonObject workprodobj = iterationQueryObject.get("WorkProducts").getAsJsonObject();
String workprodref = workprodobj.get("_ref").getAsString();
System.out.println("workprodref :" + workprodref);
GetRequest getRequest = new GetRequest(workprodref);
getRequest.setFetch(new Fetch("FormattedID"));
GetResponse getResponse = restApi.get(getRequest);
The response output is like this
{"QueryResult": {"_rallyAPIMajor": "2", "_rallyAPIMinor": "0", "Errors": [], "TotalResultCount": 120, "StartIndex": 1, "PageSize": 20,
"Results": [{"_rallyAPIMajor": "2","_rallyAPIMinor": "0","_ref": "https://rally1.rallydev.com/slm/webservice/v2.0/hierarchicalrequirement/123456789",
"_refObjectUUID": "xxxxxx",
"_objectVersion": "1",
"_refObjectName": "obj name removed masked",
"FormattedID": "US123456",
"DirectChildrenCount": 0,
} .....
Can we change the page size from 20 to 200 for a getrequest??
GetRequest extends Request which has method addParam(String, String) which can be used to pass get parameters into the encoded URL, such as setting page size.
getRequest.addParam("pagesize", "200")
ref: https://github.com/RallyTools/RallyRestToolkitForJava/blob/542afa16ea44c9c64cebb0299500dcbbb50b6d7d/src/main/java/com/rallydev/rest/request/Request.java#L54

Karate - how to access an array element by UUID during a 'retry until' statement

I have an endpoint which returns this JSON response:
{
"jobs": [
{
"name": "job1",
"id": "d6bd9aa1-0708-436a-81fd-cf22d5042689",
"status": "pending"
},
{
"name": "job2",
"id": "4fdaf09f-51de-4246-88fd-08d4daef6c3e",
"status": "pending"
}
]
I would like to repeatedly GET call this endpoint until the job I care about ("job2") has a "status" of "completed", but I'd like to check this by using a UUID stored in a variable from a previous call.
i.e. by doing something like this:
#NB: code for previous API call is executed
* def uuidVar = response.jobRef
#NB: uuidVar equates to '4fdaf09f-51de-4246-88fd-08d4daef6c3e' for this scenario
* configure retry = { count: 5, interval: 10000 }
Given path /blah
And retry until response.jobs[?(#.id==uuidVar)].status == 'completed'
When method GET
Could anyone suggest the correct syntax for the retry until?
I've tried referencing the fantastic Karate docs & examples (in particular, js-arrays.feature) and some questions on SO (including this one: Karate framework retry until not working as expected) but sadly I haven't been able to get this working.
I also tried using karate.match here as suggested in the link above, but no cigar.
Apologies in advance if I am missing something obvious.
First I recommend you read this answer on Stack Overflow, it is linked from the readme actually, and is intended to be the definitive reference. Let me know if it needs to be improved: https://stackoverflow.com/a/55823180/143475
Short answer, you can't use JsonPath in the retry until expression, it has to be pure JavaScript.
While you can use karate.jsonPath() to bridge the worlds of JsonPath and JS, JsonPath can get very hard to write and comprehend. Which is why I recommend using karate.filter() to do the same thing, but break down the steps into simple, readable chunks. Here is what you can try in a fresh Scenario:. Hint, this is a good way to troubleshoot your code without making any "real" requests.
* def getStatus = function(id){ var temp = karate.filter(response.jobs, function(x){ return x.id == id }); return temp[0].status }
* def response =
"""
{
"jobs": [
{
"name": "job1",
"id": "d6bd9aa1-0708-436a-81fd-cf22d5042689",
"status": "pending"
},
{
"name": "job2",
"id": "4fdaf09f-51de-4246-88fd-08d4daef6c3e",
"status": "pending"
}
]
}
"""
* def selected = '4fdaf09f-51de-4246-88fd-08d4daef6c3e'
* print getStatus(selected)
So if you have getStatus defined up-front, you can do this:
* retry until getStatus(selected) == 'completed'
Note you can use multiple lines for a JS function if you don't like squeezing it all into one line, or even read it from a file.

poloniex API request BUY SELL CANCEL cause error = "Invalid API key/secret pair.";

Hello To every one I am writing in swift 3 trading app. I have problem only with 3 commands buy sell and cancel , that ones cause error = "Invalid API key/secret pair."; others like returnOpenOrders , returnTradehistory returnBalances works fine and returns proper values.
That is may request function :
func getRawJSON(paramss:[String : Any]){
var paramss1:[String:Any] = [:]
let APIURL = "https://poloniex.com/tradingApi"
let timeNowInt = Int(NSDate().timeIntervalSince1970 ) * 10000000
var zdanie2:String! = ""
for (x,y) in paramss{
paramss1[x]=y
}
paramss1["nonce"]=timeNowInt
for (x,y) in paramss{
if (zdanie2 == "")
{zdanie2="\(x)=\(y)"
}
else
{
zdanie2=zdanie2+"&"+"\(x)=\(y)"
}
}
zdanie2=zdanie2+"&nonce=\(timeNowInt)"
let array: [UInt8] = Array(zdanie2.utf8)
let hmac: Array<UInt8> = try! HMAC(key: secretKey!.utf8.map({$0}), variant: .sha512).authenticate(array)
let hmacData = Data(bytes: hmac).toHexString()
let headers = ["Key": publicKey!,"Sign": hmacData] as [String : String]
request(APIURL,method: .post,parameters: paramss1,headers:headers).responseJSON {
response in
print(response)
print(response.request)
}
}
Here is my buy/sell function :
func buy(currencyPair:String,rate:Double,amount:Double){
return self.getRawJSON( paramss: ["command":"buy","currencyPair":currencyPair,"rate":rate ,"amount":amount])
}
func sell(currencyPair:String,rate:Double,amount:Double){
return self.getRawJSON( paramss: ["command":"sell","currencyPair":currencyPair,"rate": rate ,"amount":amount])
}
The output from headers parameters(var paramss1 ) is :
["amount": 2.0, "command": "sell", "nonce": 15308121310000000, "currencyPair": "BTC_XRP", "rate": 7.6000000000000004e-05]
The array for sign is :
amount=2.0&command=sell&currencyPair=BTC_XRP&rate=7.6e-05&nonce=15308121310000000
I really dont know what is wrong
Can you help with this problem ??
rate value is invalid.
rate and amount values must be like 1, 1.1, 1.00000001.
Invalid request:
amount=2.0&command=sell&currencyPair=BTC_XRP&rate=7.6e-05&nonce=15308121310000000
Valid request:
command=sell&amount=2.0&&currencyPair=BTC_XRP&rate=7.000006&nonce=15308121310000000

F# webpage Authentication

I've been trying to authenticate the bitstamp api however, I keep on getting the following error:
"{\"error\": \"Missing key, signature and nonce parameters\"}"
The code I have written to do this is below:
let nounce = System.DateTime.Today.Ticks
let hexdigest (bytes : byte[]) =
let sb = System.Text.StringBuilder()
bytes |> Array.iter (fun b -> b.ToString("X2") |> sb.Append |> ignore)
string sb
let signature =
use hmac = new HMACSHA256(System.Text.Encoding.ASCII.GetBytes(stampsecret))
hmac.ComputeHash(mes)
|> hexdigest
I am calling the website with the following:
let ordersBTCbuy()=
Http.Request("https://www.bitstamp.net/api/buy", meth="Post", query=["key", stampkey; "signature", signature.ToLower(); "nonce", string(nounce); "amount", "1"; "price", string(convertB)])
A reference to the API can be found here: https://www.bitstamp.net/api/
Update I've changed the web address to:
let ordersBTCbuy()=
Http.Request("https://www.bitstamp.net/api/buy/", meth="Post", query=["key", stampkey; "signature", signature.ToLower(); "nonce", string(nounce); "amount", "1"; "price", string(convertB)])
My new issue is now the signature my representation is 64 characters long and upper case however I still seem to have an error.
When creating a POST request. Your key-value pairs need to go in the body.
To do that using FSharp data do the following:
let postBody = FormValues([ "key", stampkey; "signature", signature.ToLower(); "nonce", string(nounce); "amount", "1"; "price", string(convertB)])
let ordersBTCbuy()=
Http.Request("https://www.bitstamp.net/api/buy", httpMethod="Post", body=postBody)