I have cluster with several replicasets. I want to call some stored function on all nodes without calculate bucket_id, and after to map results. How should I do it?
You can use module cartridge.rpc function get_candidates for getting all nodes with some role, which you want to call and after to use module cartridge.pool function map_call for calling your function and mapping results. This function available from 1.2.0-17 version of cartridge. So your code could be like this:
local cartridge = require('cartridge')
local nodes = cartridge.rpc_get_candidates('my_role_name', { leaders_only = true, healthy_only = true })
local pool = require('cartridge.pool')
local results, err = pool.map_call('_G.my_function_name', { func_args }, { uri_list = nodes, timeout = 10 })
if (err ~= nil) then
#your error handling#
end
All function response will be saved to results variable and mapped for every URI. All errors will be saved to err variable as map with keys: line, class_name, err, file, suberrors, str
Another proposal.
If you use vshard and want to perform map-reduce over storages:
local replicaset, err = vshard.router.routeall()
for _, replica in pairs(replicaset) do
local _, err = replica:callrw('function', { args })
if err ~= nil then
return nil, err
end
end
Related
I would like to call an external graphql API (without authentication for the moment).
Here is my code :
local open_api = "https://graphqlzero.almansi.me/api"
local payload = '{"query": "query { post(id: 1) { id title body }}"}'
local headers = {
}
local function craftCall()
local response
local data
pcall(function ()
response = HttpService:PostAsync(open_api, payload, Enum.HttpContentType.ApplicationJson, false, headers)
data = HttpService:JSONDecode(response)
end)
if not data then return false end
print(data)
return false
end
if craftCall() then
print("Success")
else
print("Something went wrong")
end
I get always something went wrong. I need some help on what is going wrong... Specially I don't know if am I correctly formatting the Payload.
After your http call, you never return a success result. You've only outlined failure cases :
if not data then return false end
print(data)
return false
So your conditional, if craftCall() then always evaluates to false.
Why not make it return true or data after the print(data)? Then you'll know that it made it to the end of the call successfully.
local function craftCall()
local success, result = pcall(function()
local response = HttpService:PostAsync(open_api, payload, Enum.HttpContentType.ApplicationJson, false, headers)
return HttpService:JSONDecode(response)
end)
if not success then
warn("PostAsync failed with error : ", result)
return false
end
-- return the parsed data
return result
end
I am trying to retrieve Stripe datas and parse them into a CSV file.
Here is my code:
package main
import (
"github.com/stripe/stripe-go"
"github.com/stripe/stripe-go/invoice"
"fmt"
"os"
"encoding/csv"
)
func main() {
stripe.Key = "" // I can't share the API key
params := &stripe.InvoiceListParams{}
params.Filters.AddFilter("limit", "", "3")
params.Filters.AddFilter("status", "", "paid")
i := invoice.List(params)
// Create a CSV file
csvdatafile, err := os.Create("./mycsvfile.csv")
if err != nil {
fmt.Println(err)
}
defer csvdatafile.Close()
// Write Unmarshaled json data to CSV file
w := csv.NewWriter(csvdatafile)
//Column title
var header []string
header = append(header, "ID")
w.Write(header)
for i.Next() {
in := i.Invoice()
fmt.Printf(in.ID) // It is working
w.Write(in) // It is not working
}
w.Flush()
fmt.Println("Appending succed")
}
When I am running my program with go run *.go I obtain the following error:
./main.go:35:10: cannot use in (type *stripe.Invoice) as type []string in argument to w.Write
I think I am not far from the solution.
I just need to understand how to write correctly in the CSV file thank's to w.Write() command.
According to the doc, the Write function is:
func (w *Writer) Write(record []string) error
That is, it is expecting you to pass a slice of strings representing a line of CSV data with each string being a slice. So, if you have only one field, you have to pass a string slice of 1:
w.Write([]string{in.ID})
I'm developing a plugin to Kong API Gateway. I created a service pointing it to another service in the local network and basically every request to my service is redirected to the other one, so far so good.
What the plugin has to do is grab the field Authorization Bearer in the header, and pass to the upstream service as part of the URI. E.g.
Request is received on:
localhost/service
In its header, it have a Authorization Bearer that contains a JWT
The plugin has to receive it, take the JWT and parse it to URI to the upstream service:
productionServer/service/9a8udoadzlkndid813gru1gr <-JWT took from header
My attempt till now:
local singletons = require "kong.singletons"
local BasePlugin = require "kong.plugins.base_plugin"
local responses = require "kong.tools.responses"
local constants = require "kong.constants"
local multipart = require "multipart"
local cjson = require "cjson"
local url = require "socket.url"
local access = require "kong.plugins.ctk.access"
local CtkHandler = BasePlugin:extend()
CtkHandler.PRIORITY = 3505
CtkHandler.VERSION = "0.1.0"
file = io.open("/usr/local/kong/logs/ctk.lua", "a+")
io.input(file)
file:write("--- JUST EXTENDED THE BASE PLUGIN ---")
function CtkHandler:new()
CtkHandler.super.new(self, "ctk")
file = io.open("/usr/local/kong/logs/ctk.lua", "a+")
io.input(file)
file:write("--- INSTACIATED ITSELF ---")
end
function CtkHandler:access(conf)
CtkHandler.super.access(self)
file = io.open("/usr/local/kong/logs/ctk.lua", "a+")
io.input(file)
file:write("--- STARTED THE ACCESS PART ---")
do_authentication()
access.execute(conf)
end
file:close()
return CtkHandler
The idea, is that after every request, the access block at the end be executed, then, he will redirect to my access file
local singletons = require "kong.singletons"
local BasePlugin = require "kong.plugins.base_plugin"
local responses = require "kong.tools.responses"
local constants = require "kong.constants"
local multipart = require "multipart"
local cjson = require "cjson"
local url = require "socket.url"
local basic_serializer = require "kong.plugins.log-serializers.basic"
local string_format = string.format
local ngx_set_header = ngx.req.set_header
local get_method = ngx.req.get_method
local req_set_uri_args = ngx.req.set_uri_args
local req_get_uri_args = ngx.req.get_uri_args
local req_set_header = ngx.req.set_header
local req_get_headers = ngx.req.get_headers
local req_clear_header = ngx.req.clear_header
local req_set_method = ngx.req.set_method
local ngx_decode_args = ngx.decode_args
local ngx_re_gmatch = ngx.re.gmatch
local string_format = string.format
local cjson_encode = cjson.encode
local ipairs = ipairs
local request = ngx.request
local function retrieve_token(request, conf)
file = io.open("/usr/local/kong/logs/ctk.lua", "a+")
io.input(file)
file:write("--- RUNNING RETRIEVE TOKEN ---")
local uri_parameters = request.get_uri_args()
for _, v in ipairs(conf.uri_param_names) do
if uri_parameters[v] then
return uri_parameters[v]
end
end
local ngx_var = ngx.var
for _, v in ipairs(conf.cookie_names) do
local jwt_cookie = ngx_var["cookie_" .. v]
if jwt_cookie and jwt_cookie ~= "" then
return jwt_cookie
end
end
local authorization_header = request.get_headers()["authorization"]
if authorization_header then
local iterator, iter_err = ngx_re_gmatch(authorization_header, "\\s*[Bb]earer\\s+(.+)")
if not iterator then
return nil, iter_err
end
local m, err = iterator()
if err then
return nil, err
end
if m and #m > 0 then
return m[1]
end
end
end
local function do_authentication(conf)
file = io.open("/usr/local/kong/logs/ctk.lua", "a+")
io.input(file)
file:write("--- RUNNING DO_AUTHENTICATION ---")
local token, err = retrieve_token(ngx.req, conf)
if err then
return responses.send_HTTP_INTERNAL_SERVER_ERROR(err)
end
local ttype = type(token)
if ttype ~= "string" then
if ttype == "nil" then
return false, {status = 401}
elseif ttype == "table" then
return false, {status = 401, message = "Multiple tokens provided"}
else
return false, {status = 401, message = "Unrecognizable token"}
end
append_uri(token)
return true
end
end
local function append_uri(token)
file = io.open("/usr/local/kong/logs/ctk.lua", "a+")
io.input(file)
file:write("--- FUNCTION APPEND_URL ---")
local uri = ngx.get_uri_args
ngx.req.set_uri(ngx.unescape_uri("/" .. token))
end
In the Kong server, after installing the plugin above, I receive:
--- JUST EXTENDED THE BASE PLUGIN ------ INSTACIATED ITSELF ---
Which is the control inserted inside the code to trace it.
Any ideas?
Actually using io.write isn't recommended, so what i had to do was change it to:
ngx.log(ngx.WARN, "SOME MESSAGE")
After that, the block code access ran just fine.
There's a Kong plugin that can perform the OAuth 2.0 token validation, see: kong-oidc. You may want to deploy that.
Is there an atomic GET + EXPIRE command available for Redis? This would act as a sliding expiration value: attempt to get the value specified by the key, and then only if the key was found with this request, set the time to live for X seconds.
No, there isn't, but there's nothing preventing you from sending the two commands one after the other in a MULTI/EXEC block or using a Lua script. Using EXPIRE on a non-existent key does nothing.
Or, I use simple Lua script:
local val, err = redis.pcall('GET', KEYS[1])
if err then
return err
end
redis.call('EXPIRE', KEYS[1], ARGV[1])
return {val}
In Golang you can do:
import "github.com/go-redis/redis"
const lua = `
local val, err = redis.pcall('GET', KEYS[1])
if err then
return err
end
redis.call('EXPIRE', KEYS[1], ARGV[1])
return {val}
`
redisGetEx = redis.NewScript(lua)
result, err = redisGetEx.Run(redisClient, []string{"key"}, 1800).Result()
I have a file, display.lua in which I have code to load some resources.
----display.lua
Resources = {}
function Resources:new(rootdir)
local newObj = {image = {}, audio = {}, root = ""}
newObj.root = rootdir
return setmetatable(newObj, self)
end
function Resources:getSpriteSheet(name)
--- etc etc etc
end
and then I have a game variable I use to store gamestate, this is within another file game.lua.
---game.lua
require "display.lua"
function Game:new()
local newObj = {mode = "", map = {}, player = {}, resources = {}}
self.__index = self
return setmetatable(newObj, self)
end
function Game:init()
self.resources = Resources:new("/home/example/etc/game/")
local spriteSheet = self.resources:getSpriteSheet("spritesheet.png")
end
I have access to the resources code via use of require. My issue is that within Game:init() I can't access Resources:getSpriteSheet(), the lua interpreter complains of "attempt to call method (getSpriteSheet) a nil value"
I assume here I would have to export the methods in Resources but I don't know how I'd go about doing this, as I'm quite new to Lua.
I think you want return setmetatable(newObj, {__index = self}) instead of return setmetatable(newObj, self).
Also, require "display.lua" should probably be require "display" and game.lua should have Game = {} somewhere at the top. With these changes your example works for me.