Lua in Redis from JSON - redis

I have a list of JSON strings stored in Redis that looks something like this:
[
{ "ID": 25, "DomainID": 23455, "Name": "Stuff", "Value": 23 },
{ "ID": 35, "DomainID": 23455, "Name": "Stuff", "Value": 10 }
]
The key would be something like "Event:23455".
Using a Lua script and ServiceStack.Redis how would I pull out an anonymous object containing only values where the value is less than 20?
So what I want to return would look like this:
[{ "ID": 35, "Value": 10}]
Thanks.
UPDATE 03/31/2013:
After trying what has been suggested I now have a new problem. A Lua script syntax error.
I'm getting a Lua syntax error about "expecting '=' near cjson". Here is the Lua script string (in C#) I am feeding to Redis:
string luaScript = "local tDecoded = cjson.decode(redis.call('GET', KEYS[1]));"
+ "local tFinal = {};"
+ "for iIndex, tValue in ipairs(tDecoded) do"
+ " if tonumber( tValue.Value ) < 20 then"
+ " table.insert(tFinal, { ID = tValue.ID, Value = tValue.Value});"
+ " end"
+ "end"
+ "return cjson.encode(tFinal);";
Are there any Lua or Redis Lua experts out there that can see what might be the problem? i.e. Does the Lua syntax look correct?
UPDATE 04/02/2013
The parsing error was resolved by adding \n newline characters to end of each line like so.
string luaScript = "local tDecoded = redis.call('GET', KEYS[1]);\n"
+ "local tFinal = {};\n"
+ "for iIndex, tValue in ipairs(tDecoded) do\n"
+ " if tonumber( tValue.Value ) < 20 then\n"
+ " table.insert(tFinal, { ID = tValue.ID, Value = tValue.Value});\n"
+ " else\n"
+ " table.insert(tFinal, { ID = 999, Value = 0});\n"
+ " end\n"
+ "end\n"
+ "return cjson.encode(tFinal);";
Unfortunately this works without errors but for some reason only returns "{}" or an empty list in the ServiceStack RedisClient. So I'm not there yet but I am one step closer.

You can use LuaJSON available on GitHub or you can also try this JSON to Lua table parser for the task. The usage will be something like this(for the GitHub link):
local json = require( "json" ) -- or JSON.lua
local tDecoded = json.decode(sJSON) -- sJSON is the original JSON string
local tFinal = {}
for iIndex, tValue in ipairs( tDecoded ) do
if tonumber( tValue.Value ) < 20 then
table.insert( tFinal, { ID = tValue.ID, Value = tValue.Value} )
end
end
print( json.encode(tFinal) )

Related

how to parse data from json file in flutter?

Error on line 60, column 2 of pubspec.yaml: Expected a key while parsing a block mapping.
â•·
60 │ assets:
│ ^
╵
Process finished with exit code 65
i need to show data from json file
you can user dart:convert API. you can use JsonCodec class to parse data from json file.
here is an example .
{
"name": "John Doe",
"age": 30,
"email": "johndoe#example.com"
}
String jsonData = await rootBundle.loadString('assets/data.json');
dynamic data = json.decode(jsonData);
String name = data['name'];
int age = data['age'];
String email = data['email'];

API call from google sheets is too long

I am trying to write a script in google sheets to update my 3commas bots. The API requires a number of mandatory fields are passed even when there's only 1 item that needs to be updated.
The code I have is below and it uses the values already read from the platform updating only the base_order_volume value. This works perfectly except for when the pairs value is long (more the 2k chars) and then I get an error from the UrlFetchApp call because the URL is too long.
var sheet = SpreadsheetApp.getActiveSheet();
var key = sheet.getRange('F4').getValue();
var secret = sheet.getRange('F5').getValue();
var baseUrl = "https://3commas.io";
var editBots = "/ver1/bots/"+bots[botCounter].id+"/update";
var patchEndPoint = "/public/api"+editBots+"?";
.
.
[loop around values in sheet]
.
.
var BaseOrder=Number(sheet.getRange(rowCounter,12).getValue().toFixed(2));
var botParams = {
"name": bots[botCounter].name,
"pairs": bots[botCounter].pairs,
"max_active_deals": bots[botCounter].max_active_deals,
"base_order_volume": BaseOrder,
"take_profit": Number(bots[botCounter].take_profit),
"safety_order_volume": bots[botCounter].safety_order_volume,
"martingale_volume_coefficient": bots[botCounter].martingale_volume_coefficient,
"martingale_step_coefficient": Number(bots[botCounter].martingale_step_coefficient),
"max_safety_orders": bots[botCounter].max_safety_orders,
"active_safety_orders_count": Number(bots[botCounter].active_safety_orders_count),
"safety_order_step_percentage": Number(bots[botCounter].safety_order_step_percentage),
"take_profit_type": bots[botCounter].take_profit_type,
"strategy_list": bots[botCounter].strategy_list,
"bot_id": bots[botCounter].id
};
var keys = Object.keys(botParams);
var totalParams = keys.reduce(function(q, e, i) {
q += e + "=" + encodeURIComponent(JSON.stringify(botParams[e])) + (i != keys.length - 1 ? "&" : "");
return q;
},endPoint);
var signature = Utilities.computeHmacSha256Signature(totalParams, secret);
signature = signature.map(function(e) {return ("0" + (e < 0 ? e + 256 : e).toString(16)).slice(-2)}).join("");
var headers = {
'APIKEY': key,
'Signature': signature,
};
try {
var params = {
'method': 'PATCH',
'headers': headers,
'muteHttpExceptions': true
};
var response = JSON.parse(UrlFetchApp.fetch(baseUrl + totalParams, params).getContentText());
I have tried to set the botParams as a payload in the params but when I do the signature is incorrect.
I anyone knows how to use sheets to make a call using extensive length of parameters I'd appreciate any help at all
Some sample data for the bots array would be
{
"name": "TestBot",
"base_order_volume": 0.001,
"take_profit": 1.5,
"safety_order_volume": 0.001,
"martingale_volume_coefficient": 2,
"martingale_step_coefficient": 1,
"max_safety_orders": 1,
"active_safety_orders_count": 1,
"safety_order_step_percentage": 2.5,
"take_profit_type": "total",
"stop_loss_percentage": 0,
"cooldown": 0,
"pairs": ["BTC_ADA","BTC_TRX"],
"trailing_enabled":"true",
"trailing_deviation":0.5,
"strategy_list": [{"strategy":"cqs_telegram"}]
}
Thanks in advance
I'd consider using a Cloud Function to either do the heavy lifting, or, if you're worried about costs, use it as a proxy. You can then call the cloud function from Google Sheets. Cloud Functions can be written in whatever language you're most comfortable with, including Node.
Check the GCP pricing calculator to see what the cost would be. For many cases it would be completely free.
This should give you a sense of how to use cloud functions for CSV creation:
https://codelabs.developers.google.com/codelabs/cloud-function2sheet#0
Here is a SO question with an answer that explains how to query cloud functions with authentication.

Arduino C language use toCharArray() not correct

I read data from sensor and show to serial monitor like this.
int humidity = dht.readHumidity();
int temperature = dht.readTemperature();
String place = "My Home";
if (online) {
if (!client.connected()) {
reconnect();
}
client.loop();
String data = "{\"data\": {\"humidity\":" + String(humidity) + ", \"temperature\":" + String(temperature) + ", \"place\": " + place + "}}";
data.toCharArray(msg, (data.length() + 1));
Serial.println(msg);
The output is
{"data": {"humidity":60, "temperature":27, "place": My Home}}
{"data": {"humidity":2147483647, "temperature":2147483647, "place": My Home}}
After I run code it have no error but the output is not correct and the program will stop after show this output.
If I comment at line data.toCharArray(msg, (data.length() + 1)); . Then, the program will show the correct output of temperature and humidity value. How to fix it?
Use the basic itoa function. It is included in stdlib.h
char buf[12]; // "-2147483648\0"
lcd.printIn(itoa(random(1024)-512, buf, 10));
Source : Arduino Playground

Chaining REST calls in a pipeline while managing errors

Coming from nodejs where I could chain asynchronous events using Promises and then operator I'm trying to explore how things are done in idiomatic F#.
The calls I'm trying to chain are HTTP rest calls on some entity from creation to update to uploading images to publishing.
Function composition says the output of one function should match the input of the second one to be composed and that common input and output in my case will be string, i.e. JSON serialized string as input and output of all of these functions.
I've learned that you can compose functions using >> operator. Ideally functions should not throw errors but things happen with IO, for instance in this case what if the id of the entity I'm trying to create exists etc.
The unknown and the question is what happens if an error occurs during the chained sequence, how the caller will know what went wrong along with description message? The operation could fail in the middle or towards the end or right in the beginning of the chain sequence.
What I'm expecting from these functions upon error to stop executing the chain and return the error message to the caller. Error message is also a JSON string so there's no incompatibility between inputs and outputs of a function so you know.
I looked at Choice too but not sure if that's the direction I should be going for.
The code is not necessarily complete and all I'm looking for a is a direction to research further to get answers and possibly improve this question. Here's some code to start with.
let create schema =
// POST request
"""{"id": 1, "title": "title 1"}""" // result output
let update schema =
// PUT request, update title
"""{"id": 1, "title": "title 2"}""" // output
let upload schema =
// PUT request, upload image and add thumbnail to json
"""{"id": 1, "title": "title 2", "thumbnail": "image.jpg"}"""
let publish schema =
// PUT request, publish the entity, add url for the entity
if response.StatusCode <> HttpStatusCode.OK then
"""{"code": "100", "message": "file size above limit"}"""
else
"""{"id": 1, "title": "title 2", "thumbnail": "image.jpg", "url": "http://example.com/1"}"""
let chain = create >> update >> upload >> publish
Edit - Attempt
Trying to parameterize the image thumbnail in the upload part
let create (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 1"}""" // result output
let update (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 2"}""" // output
let upload2 (img: string) (schema: string) : Result<string,string> =
printf "upload image %s\n" img
let statusCode = HttpStatusCode.OK
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg"}"""
| x -> Error (sprintf "%A happened" x)
let publish (schema: string) =
let statusCode = HttpStatusCode.InternalServerError
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg", "url": "http://example.com/1"}"""
| _ -> Error """{"code": "100", "message": "couldn't publish, file size above limit"}"""
let chain = create >> Result.bind update >> Result.bind (upload2 "image.jpg") >> Result.bind publish
A good general approach to this problem is wrapping your functions' return values in a Choice/Either-like type and using a higher-order function to bind them together such that a failure propagates/short-circuits with some meaningful data. F# has a Result type with a bind function that can be used like this:
type MyResult = Result<string,string>
let f1 x : MyResult = printfn "%s" x; Ok "yep"
let f2 x : MyResult = printfn "%s" x; Ok "yep!"
let f3 x : MyResult = printfn "%s" x; Error "nope :("
let fAll = f1 >> Result.bind f2 >> Result.bind f3
> fAll "howdy";;
howdy
yep
yep!
[<Struct>]
val it : Result<string,string> = Error "nope :("
The first two functions succeed, but the third fails and so you get an Error value back.
Also check out this article on Railway-oriented programming.
Update to be more specific to your example:
let create (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 1"}""" // result output
let update (schema: string) : Result<string,string> =
Ok """{"id": 1, "title": "title 2"}""" // output
let upload (schema: string) =
let statusCode = HttpStatusCode.OK
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg"}"""
| x -> Error (sprintf "%A happened" x)
let publish (schema: string) =
let statusCode = HttpStatusCode.InternalServerError
match statusCode with
| HttpStatusCode.OK -> Ok """{"id": 1, "title": "title 2", "thumbnail": "image.jpg", "url": "http://example.com/1"}"""
| _ -> Error """{"code": "100", "message": "file size above limit"}"""
let chain =
create >> Result.bind update >> Result.bind upload >> Result.bind publish

API Gateway exposing DynamoDB (Complex mapping template or Lambda Function needed?)

I am exposing my DDB via the AWS API Gateway. I have it working, but when I pass my "PUT" it's requiring me to include every field from my DDB in the body request. Some clients may only want to POST 5 fields of the 10 in my DDB. Would this be done as a POST or a PATCH? If so, what does my mapping template look like for these?
Second: I'd like to add an auto-generated order_number to each entry in the DDB. So now each PUT or POST will have a unique order_number. So my Primary Key will be a variable, and the secondary key would be a customer sku number, or vice versa.
Thanks for the help.
Body Template: I still need it to generate the "order_number"
automatically for each new entry.
#set($inputSep = $input.path('$'))
#set($thisLoop =
'{"CustomerNm":"1","ShipDate":"2","Shipping":"3","commentId":"4"}')
#set($myPlaces = $util.parseJson($thisLoop))
#set($allKeys = '')
#set($myVals = '"')
{
"TableName": "BRPI",
"Key": {
"CustomerID": {
"S": "$input.path('$.CustomerID')"
}
},
#foreach($type in $inputSep.keySet())
#if($type == "CustomerID")
#else
#set($thisPlace = $myPlaces.get($type))
#set($params = $inputSep.get($type))
#set($thisKey = "$type" + " = :val" + "$thisPlace")
#set($thisPath = "$params")
#if($foreach.hasNext)
#set($myComma = ",")
#set($myReturn = '"')
#else
#set($myComma = "")
#set($myReturn = "")
#end
#set($allKeys = "$allKeys" + "$thisKey" + "$myComma")
#set($myVals = "$myVals" + ":val" + "$thisPlace" + '": {"S": "' +
"$thisPath" + '"}' + "$myComma" + "$myReturn")
#end
#end
"UpdateExpression": "SET $allKeys",
"ExpressionAttributeValues": {
$myVals
},
"ReturnValues":"UPDATED_NEW"
}