I'm looking for a convenient way to dump and load variables in Julia, just like pickle does in Python.
Is there a package which does something like myVar = load(myPath) and dump(myVar, myPath) (or similarly f = open(myPath, "r"); myVar = load(f)) ?
HDF5.jl package have been split into HDF5 new package and JLD
see here:
https://github.com/JuliaLang/JLD.jl
Saving and loading julia variables while preserving native types is now possible using JLD
According doc:
JLD, for which files conventionally have the extension .jld, is a
widely-used format for data storage with the Julia programming
language. JLD is a specific "dialect" of HDF5, a cross-platform,
multi-language data storage format most frequently used for scientific
data. By comparison with "plain" HDF5, JLD files automatically add
attributes and naming conventions to preserve type information for
each object.
I think the HDF5 package has the functionality you want, it worked very good for me using some custom types and all:
see here:
https://github.com/JuliaLang/HDF5.jl
Related
Given an Octave dataframe object created as
c = cell(m,n);
%populate c...
pkg load dataframe
df = dataframe(c);
(see https://octave.sourceforge.io/dataframe/overview.html),
Is it possible to access the underlying cell array?
Is it there a conversion mechanism back to cell array?
Is it possible to save df to CSV?
Yes. A dataframe object, like any object, can be converted back into a struct.
Once you have the resulting struct, look for the fields x_name to get the column names, and x_data to get the data in the form of a cell array, i.e.
struct(df).x_data
As for conversion to csv, the dataframe package does not seem to provide any relevant methods as far as I can tell (in particular the package does not provide an overloaded #dataframe/csvwrite method). Therefore, I'd just extract the information as above, and go about writing it into a csv file from there.
If you're not dealing with strictly numerical data, you might want to have a look at the cell2csv / csv2cell methods from the io package (since the built-in csvwrite function is strictly for numerical data).
And if that doesn't do exactly what you want, I'd probably just go for creating a csv file manually via custom fprintf statements.
PS. You can generally see what methods a package provides via pkg describe -verbose dataframe, or the methods for a particular class via methods(dataframe) (or even methods(df)). Also, if you ever wanted to access the documentation for an overloaded method, e.g. say the summary method, then this is the syntax for doing so: help #dataframe/summary
I've been reading a lot of Julia documentation (version 0.4) and am still having problems with loading Julia files. This seems like it should be really easy. So, plainly and simply, how are we supposed to use Julia code from other files directly in our current code? And, as a related, helpful bonus, is there any history or language design decisions that, understood, would illuminate the situation?
P.S. I'm using 0.4.
If you want problem specifics, here are some things I'm dealing with:
First
Using the REPL, I want to use some functions I have written in a different file. Supposedly, I should be able to load said file like this:
julia> using Foobar
That just gives me ArgumentErrors no matter what I do. I've tried including it before trying to use it:
julia> include("Foobar.jl")
julia> using Foobar
I've also tried updating the load path before trying to use it:
julia> push!(LOAD_PATH, "/Users/me/julia")
julia> using Foobar
Second
When I try to fix the first problem by including the file before using it, I get an error for any line that has: using .... The message is that a module cannot be found in path. Or in other words, I'm trying to load a module in the current working directory that depends on another module in the current working directory. When I include the file I'm trying to load, it tries to find the dependency and cannot.
Third
I've tried relative paths. I.e. I'm in the same directory as the .jl file and do:
julia> using .Foobar
if you use include("/path/to/myscript.jl") then you should then have access to any functions, objects, etc. defined in the file you called with include(). No additional calls to using should be needed.
Here is an answer that gives more info on details for creating whole packages (rather than just individual scripts like in the example above), how to do them, and how the using terminology factors in with them: julia: create and use a local package without Internet . For instance, packages must be installed in a particular path relative to your other julia files, not just in the arbitrary working directory that your script is in.
See also here for a longer tutorial on packages.
It seems to work well enough here:
julia> push!(LOAD_PATH, "/Users/me/julia")
2-element Array{ByteString,1}:
"/Applications/Julia-0.4.5.app/Contents/Resources/julia/local/share/julia/site/v0.4"
"/Users/me/julia"
julia> readdir(LOAD_PATH[end])
1-element Array{ByteString,1}:
"MyModule.jl"
julia> using MyModule
julia> x
"Hi there"
where MyModule.jl contains:
module MyModule
export x
x = "Hi there"
end
The realm-browser is great to look at realm-files, but what about exporting data from the realm-browser? Is there a tool out there to convert realm-files into other formats?
For example, is there an easy way to get data converted from .realm into an excel spreadsheet?
The realm-browser will eventually be extended to both import and export in various formats. The realm bindings will also get export to json capabilities.
Until then you can just iterate through allObjects and print them to a file with the built-in description method or extract each property and dump it in another preferred format.
I have the following in my constants file:
typedef enum
{
AnimalTypeBear,
AnimalTypeCamel,
AnimalTypeCow,
AnimalTypeCount
}
AnimalType;
If I declare an AnimalType variable somewhere in my code like following and set it to AnimalTypeBear:
AnimalType animalType = 0;
Is there away to somehow derive the string "Bear" from that animalType variable or just in general to access the string of its corresponding constant type (in this case AnimalTypeBear).
Enums are constant expressions like #define. Enums at compile time will be "translated" into the code as constants (while #define will be evaluated before compilation). So basically it is not possible to reference the enum string in this way.
As suggested by others you can use a string array.
You cannot do this without code in (Objective-)C. If you want to be able to use actual enumeration literals as strings in your code, or during I/O, with language support then you need to use a language with enumeration type support such as Pascal or Ada.
If you are keen to have this and don't mind work as long as it is reusable then you need to learn about reading the symbol tables structures from a binary and make sure that the information is not stripped from your application. You'll see the debugger can show the correct literals, also if you use Xcode's "Product > Generate Output > Assembly File" menu item you'll see the literals are in there as strings. It will be a lot of work for you, but would be reusable once done.
After that give up and write some code - a simple static array of labels and an index operation. Yes, it's a maintenance headache if you ever change your enumeration.
Alternatively you can write some different code, say in Ruby... Xcode supports adding your own file "types" and running scripts to (pre-)process them. So you could define, say, a file type ".enum" and use a Ruby script to convert that into a C enumeration definition and code to provide the strings. Apple has examples of using Ruby to pre-process files in this way. Once you have your script Xcode will do the rest, on each compilation it will run your script to convert your ".enum" into ".m" (or ".c") and the compile the result. This approach is usually best though for files which contain only one thing, e.g. localised string file processing, you don't usually write your enum declarations in their own files.
I would like to save my objects as XML so that other applications can read from and write to the data files -- something that is very difficult with Matlab's binary mat files.
The underlying problem I'm running into is that Matlab's equivalent of reflection (which I've used to do similar things in .NET) is not very functional with respect to private properties. Matlab's struct(object) function offers a hack in terms of writing XML from an object because while I can't do
x = myInstance.myPrivateProperty;
...I can do
props = struct(myInstance);
x = props.myPrivateProperty;
So, I can create a pure (contains no objects) struct from any object using the code below, and then it's trivial to write an XML file using a pure struct.
But, is there any way to reverse the process? That is, to create an object instance using the data saved by the code below (that data contains a list of all non-Dependent, non-Constant, non-Transient properties of the class instance, and the name of the class)? I was thinking about having all my objects inherit from a class called XmlSerializable which would accept a struct as a single argument in the constructor and then assign all the values contained in the struct to the correspondingly-named properties. But, this doesn't work because if MyClass inherits XmlSerializable, the code in XmlSerializable isn't allowed to set the private properties of MyClass (related to How can I write generalized functions to manipulate private properties?). This would be no problem in .NET (See Is it possible to set private property via reflection?), but I'm having trouble figuring it out in Matlab.
This code creates a struct that contains all of the state information for the object(s) passed in, yet contains no object instances. The resulting struct can be trivially written to XML:
function s = toPureStruct(thing)
if isstruct(thing)
s = collapseObjects(thing);
s.classname = 'struct';
elseif isobject(thing)
s.classname = class(thing);
warning off MATLAB:structOnObject;
allprops = struct(thing);
warning on MATLAB:structOnObject
mc = metaclass(thing);
for i=1:length(mc.PropertyList)
p = mc.PropertyList(i);
if strcmp(p.Name, 'classname')
error('toStruct:PropertyNameCollision', 'Objects used in toStruct may not have a property named ''classname''');
end
if ~(p.Dependent || p.Constant || p.Transient)
if isobject(allprops.(p.Name))
s.(p.Name) = toPureStruct(allprops.(p.Name));
elseif isstruct(allprops.(p.Name))
s.(p.Name) = collapseObjects(allprops.(p.Name));
else
s.(p.Name) = allprops.(p.Name);
end
end
end
else
error(['Conversion to pure struct from ' class(thing) ' is not possible.']);
end
end
function s = collapseObjects(s)
fnames = fields(s);
for i=1:length(fnames)
f = s.(fnames{i});
if isobject(f)
s.(fnames{i}) = toPureStruct(f);
elseif isstruct(f)
s.(fnames{i}) = collapseObjects(f);
end
end
end
EDIT: One of the other "applications" I would like to read the saved files is a version control system (to track changes in parameters in configurations defined by Matlab objects), so any viable solution must be capable of producing human-intelligible text. The toPureStruct method above does this when the struct is converted to XML.
You might be able to sidestep this issue by using the new v7.3 MAT file format for your saved objects. Unlike the older MAT file formats, v7.3 is a variant of the HDF5, and there's HDF5 support and libraries in other languages. This could be a lot less work, and you'd probably get better performance too, since HDF5 is going to have a more efficient representation of numeric arrays than naive XML will.
It's not the default format; you can enable it using the -v7.3 switch to the save function.
To the best of my knowledge, what I want to do is impossible in Matlab 2011b. It might be possible, per #Andrew Janke's answer, to do something similar using Matlab's load command on binary HDF5 files that can be read and modified by other programs. But, this adds an enormous amount of complexity as Matlab's HDF5 representation of even the simplest class is extremely opaque. For instance, if I create a SimpleClass classdef in Matlab with two standard properties (prop1 and prop2), the HDF5 binary generated with the -v7.3 switch is 7k, and the expanded XML is 21k, and the text "prop1" and "prop2" do not appear anywhere. What I really want to create from that SimpleClass is this:
<root>
<classname>SimpleClass</classname>
<prop1>123</prop1>
<prop2>456</prop2>
</root>
I do not think it is possible to produce the above text from class properties in a generalized fashion in Matlab, even though it would be possible, for instance, in .NET or Java.