I am trying to use shadow variables in optapy but I am not sure I understand how I can update their values correctly. The documentation of OptaPlanner suggests that to update a shadow variable, OptaPlanner uses a VariableListener, but they seem not supported in optapy yet. Am I reading this wrong and I do not need a VariableListener?
If I use the example in the optapy documentation:
from optapy import planning_entity, planning_variable
from optapy.types import PlanningVariableGraphType
#planning_entity
class Customer:
#planning_variable(object, graph_type = PlanningVariableGraphType.CHAINED, ...)
def get_previous_standstill(self):
return self.previous_standstill
def set_previous_standstill(previous_standstill):
...
from optapy import planning_entity, inverse_relation_shadow_variable
#planning_entity
class Standstill:
#inverse_relation_shadow_variable(Customer, source_variable_name ="previous_standstill")
def get_next_customer(self):
return self.next_customer
def set_next_customer(Customer nextCustomer):
...
How is variable next_customer updated?
Custom shadow variables (which use custom VariableListeners) are currently not supported (tracking issue: https://github.com/optapy/optapy/issues/75), but builtin shadow variables (which use predefined VariableListeners) are. The builtin shadow variables are: #inverse_relation_shadow_variable, which updates when the source variable takes the object as a value; and #anchor_shadow_variable which updates when the start of a chain for the source chained variable changes.
In the above example, If I have a Standstill standstill, then whenever OptaPy updates a Customer customer via customer.set_previous_standstill(standstill), standstill.set_next_customer(customer) is called.
Related
I'm struggling to understand how exactly modules can be extended in Julia. Specifically, I'd like to create my own LinearAlgebra matrix whose parent class is AbstractMatrix{T} and implement its functionality similar to how the Diagonal or UpperTriangular matrices are implemented in the actual LA package. If I could literally add my matrix to the original package, then I would, but for now I am content creating my own MyLinearAlgebra package that simply imports the original and extends it. Here's what I've got so far in MyLinearAlgebra.jl:
module MyLinearAlgebra
import LinearAlgebra
import Base: getindex, setindex!, size
export
# Types
LocalMatrix,
SolutionVector,
# Functions
issymmetric,
isdiag
# Operators
# Constants
include("SolutionVector.jl")
include("LocalMatrix.jl")
end
Focusing solely on LocalMatrix.jl now, I have:
"""
struct LocalMatrix{T} <: AbstractMatrix{T}
Block diagonal structure for local matrix. `A[:,:,s,iK]` is a block matrix for
state s and element iK
"""
struct LocalMatrix{T} <: AbstractMatrix{T}
data::Array{T,4}
function LocalMatrix{T}(data) where {T}
new{T}(data)
end
end
[... implement size, getindex, setindex! ... all working perfectly]
"""
issymmetric(A::LocalMatrix)
Tests whether a LocalMatrix is symmetric
"""
function issymmetric(A::LocalMatrix)
println("my issymmetric")
all(LinearAlgebra.issymmetric, [#view A.data[:,:,i,j] for i=1:size(A.data,3), j=1:size(A.data,4)])
end
"""
isdiag(A::LocalMatrix)
Tests whether a LocalMatrix is diagonal
"""
function isdiag(A::LocalMatrix)
println("my isdiag")
all(LinearAlgebra.isdiag, [#view A.data[:,:,i,j] for i=1:size(A.data,3), j=1:size(A.data,4)])
end
When I try and run this however, I get
error in method definition: function LinearAlgebra.isdiag must be explicitly imported to be extended
OK not a problem, I can change the definition to function LinearAlgebra.isdiag() instead and it works. But if I also change the definition of the other function to function LinearAlgebra.issymmetric() and run a simple test I now get the error
ERROR: MethodError: no method matching issymmetric(::MyLinearAlgebra.LocalMatrix{Float64})
So I'm stumped. Obviously I have a workaround that lets me continue working for now, but I must be simply misunderstanding how Julia modules work because I can't seem to distinguish between the two functions. Why does one needs to be explicitly extended? Why can the other not? What even is the difference between them in this situation? What is the correct way here to extend a package's module? Thanks for any help.
You need to explicitly state that you are adding new methods to existing functions so it should be:
function LinearAlgebra.issymmetric(A::LocalMatrix)
...
end
function LinearAlgebra.isdiag(A::LocalMatrix)
...
end
The reason that you are getting the error most likely is because you forgot to import LinearAlgebra in the code that is testing your package.
Note that your constructor also should be corrected:
struct LocalMatrix{T} <: AbstractMatrix{T}
data::Array{T,4}
function LocalMatrix(data::Array{T,4}) where {T}
new{T}(data)
end
end
With the current constructor you need to write LocalMatrix{Float64}(some_arr) instead of simply LocalMatrix(some_arr). Even worse, if you provide your constructor with a 3-d array you will get a type conversion error, while when you used the syntax that I am proposing one gets no method matching LocalMatrix(::Array{Int64,3}) which is much more readable for users of your library.
There are two packages I want to use: CorpusLoaders.jl, and WordNet.jl
CorpusLoaders.SemCor exports sensekey(::SenseTaggedWord)
WordNet exports sensekey(::DB, ::Synset, ::Lemma)
I want to use both sensekey methods.
Eg
for some mixed list of items: mixedlist::Vector{Union{Tuple{SenseTaggedWord},Tuple{DB, Synset,Lemma}}.
Ie the items in the list are a mixture of 1-tuples of SenseTaggedWord, and3 tuples of DB, Synset, and Lemma.
for item in mixedlist
println(sensekey(item...)
end
should work.
This example is a little facetious, since why would I be mixing them like this.
But, hopefully it serves for illustrating the problem in the general case.
Trying to using CorpusLoaders.SemCor, WordNet to bring in both results in WARNING: both WordNet and Semcor export "sensekey"; uses of it in module Main must be qualified.
Manually importing both: import CorpusLoaders.SemCor.sensekey; import WordNet.sensekey results in WARNING: ignoring conflicting import of Semcor.sensekey into Main
What can be done? I want them both, and they don't really conflict, due to multiple-dispatch.
Given that CorpusLoaders.jl is a package I am writing I do have a few more options, since I could make my CorpusLoaders.jl depend on WordNet.jl.
If I did do than then I could say in CorpusLoaders.jl
import WordNet
function WordNet.sensekey(s::SenseTaggedWord)...
and that would make them both work.
But it would mean requiring WordNet as a dependency of CorpusLoaders.
And I want to know how to solve the problem for a consumer of the packages -- not as the creator of the packages.
tl;dr qualify the functions when using them in your script via their module namespace, i.e. CorpusLoader.sensekey() and WordNet.sensekey()
Explanation
My understanding of your question after the edits (thank you for clarifying) is that:
You have written a package called CorpusLoaders.jl, which exports the function sensekey(::SenseTaggedWord)
There is an external package called WordNet.jl, which exports the function sensekey(::DB, ::Synset, ::Lemma)
You have a script that makes use of both modules.
and you are worried that using the modules or "importing" the functions directly could potentially create ambiguity and / or errors in your script, asking
how can I write my CorpusLoaders package to prevent potential clashes with other packages, and
how can I write my script to clearly disambiguate between the two functions while still allowing their use?
I think this stems from a slight confusion how using and import are different from each other, and how modules create a namespace. This is very nicely explained in the docs here.
In essence, the answers are:
You should not worry about exporting things from your module that will clash with other modules. This is what modules are for: you're creating a namespace, which will "qualify" all exported variables, e.g. CorpusLoaders.sensekey(::SenseTaggedWord).
When you type using CorpusLoaders, what you're saying to julia is "import the module itself, and all the exported variables stripped from their namespace qualifier, and bring them into Main". Note that this means you now have access to sensekey as a function directly from Main without a namespace qualifier, and as CorpusLoaders.sensekey(), since you've also imported the module as a variable you can use.
If you then try using the module WordNet as well, julia very reasonably issues a warning, which essentially says:
"You've imported two functions that have the same name. I can't just strip their namespace off because that could create problems in some scenarios (even though in your case it wouldn't because they have different signatures, but I couldn't possibly know this in general). If you want to use either of these functions, please do so using their appropriate namespace qualifier".
So, the solution for 2. is:
you either do
using CorpusLoaders;
using WordNet;
, disregarding the warning, to import all other exported variables as usual in your Main namespace, and access those particular functions directly via their modules as CorpusLoaders.sensekey() and WordNet.sensekey() each time you need to use them in your script, or
you keep both modules clearly disambiguated at all times by doing
import CorpusLoaders;
import WordNet;
and qualify all variables appropriately, or
in this particular case where the function signatures don't clash, if you'd really like to be able to use the function without a namespace qualifier, relying on multiple dispatch instead, you can do something like what FengYang suggested:
import CorpusLoaders;
import WordNet;
sensekey(a::SenseTaggedWord) = CorpusLoader.sensekey(a);
sensekey(a::DB, b::Synset, c::Lemma) = WordNet.sensekey(a, b, c);
which is essentially a new function, defined on module Main, acting as a wrapper for the two namespace-qualified functions.
In the end, it all comes down to using using vs import and namespaces appropriately for your particular code. :)
As an addendum, code can get very unwieldy with long namespace qualifiers like CorpusLoader and WordNet. julia doesn't have something like python's import numpy as np, but at the same time modules become simple variables on your workspace, so it's trivial to create an alias for them. So you can do:
import CorpusLoaders; const cl = CorpusLoaders;
import Wordnet; const wn = WordNet;
# ... code using both cl.sensekey() and wn.sensekey()
In this case, the functions do not conflict, but in general that is impossible to guarantee. It could be the case that a package loaded later will add methods to one of the functions that will conflict. So to be able to use the sensekey for both packages requires some additional guarantees and restrictions.
One way to do this is to ignore both package's sensekey, and instead provide your own, dispatching to the correct package:
sensekey(x) = CorpusLoaders.sensekey(x)
sensekey(x, y, z) = WordNet.sensekey(x,y,z)
I implemented what #Fengyang Wang said,
as a function:
function importfrom(moduleinstance::Module, functionname::Symbol, argtypes::Tuple)
meths = methods(moduleinstance.(functionname), argtypes)
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol)
meths = methods(moduleinstance.(functionname))
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol, meths::Base.MethodList)
for mt in meths
paramnames = collect(mt.lambda_template.slotnames[2:end])
paramtypes = collect(mt.sig.parameters[2:end])
paramsig = ((n,t)->Expr(:(::),n,t)).(paramnames, paramtypes)
funcdec = Expr(:(=),
Expr(:call, functionname, paramsig...),
Expr(:call, :($moduleinstance.$functionname), paramnames...)
)
current_module().eval(funcdec) #Runs at global scope, from calling module
end
end
Call with:
using WordNet
using CorpusLoaders.Semcor
importfrom(CorpusLoaders.Semcor, :sensekey)
importfrom(WordNet, :sensekey)
methods(sensekey)
2 methods for generic function sensekey:
sensekey(db::WordNet.DB, ss::WordNet.Synset, lem::WordNet.Lemma)
sensekey(saword::CorpusLoaders.Semcor.SenseAnnotatedWord
If you wanted to get really flash you could reexport the DocString too.
I would like to learn the best way to modify TensorFlow built-in operator kernels.
For example, I want to modify the value of static const double A in tensorflow/core/kernels/resize_bicubic_op.cc. I have come up with two possible ways:
Modify it directly and recompile the whole TensorFlow library. The problems of this solution are: A. This influences all the functions which use bicubic interpolation. B. This requires me to recompile the entire library and does not work when installing from a binary.
Define it as a custom op. The problem is that in the source code, there is no REGISTER_OP() inside. I don't know how to write the REGISTER_OP() for this bicubic function and whether other modification needs to be made.
Are there other better ways?
Thanks.
The best way to approach this problem is to build a custom op. See this tutorial for more details about how to add custom ops in general. The REGISTER_OP call for the tf.image.resize_bicubic() op is in tensorflow/core/ops/image_ops.cc.
Another alternative is to re-use the same op registration, and register a new kernel with the alternative implementation. This would enable you to use the (experimental) Graph.kernel_label_map() API to select an alternative implementation for the "ResizeBicubic" op. For example, you could do the following in your Python program:
_ = tf.load_op_library(...) # Load the .so containing your implementation.
with tf.get_default_graph().kernel_label_map({"ResizeBicubic": "my_impl"}):
images = tf.image.resize_bicubic(...) # Will use your implementation.
...and add a kernel registration that specifies the label "my_impl" with your C++ code:
template <typename Device, typename T>
class MyResizeBicubicOp<Device, T> : public OpKernel {
// Custom implementation goes here...
}
#define REGISTER_KERNEL(T) \
REGISTER_KERNEL_BUILDER(Name("ResizeBicubic") \
.Device(DEVICE_CPU) \
.Label("my_impl") \
.TypeConstraint<T>("T") \
.HostMemory("size"), \
MyResizeBicubicOp<CPUDevice, T>);
TF_CALL_REAL_NUMBER_TYPES(REGISTER_KERNEL);
I try to incorporate a self-designed optimization algorithm PSGLD into TensorFlow. And that algorithm is similar to the concept of RMSProp. So I didn't create a new Op, but complement PSGLD following RMSProp. My procedure of incorporating is as follows:
In Python side, create a psgld.py under the folder of tensorflow\python\training,which represents the Python wrapper. And in psgld.py, define the class of PSGLDOptimizer.
class PSGLDOptimizer(optimizer.Optimizer)
Then, in tensorflow\python\training\training_ops.py, define the shape function of _ApplyPSGLDShape and _SparseApplyPSGLD, for dense and sparse circumstances respectively.
For C++ side, in tensorflow\core\ops\training_ops.cc, define the input, output and attribute of ApplyPSGLD Op:
REGISTER_OP("ApplyPSGLD")
.Input("var: Ref(T)")
.Input("ms: Ref(T)")
.Input("mom: Ref(T)")
.Input("lr: T")
.Input("decay: T")
.Input("epsilon: T")
.Input("grad: T")
.Output("out: Ref(T)")
.Attr("T: numbertype")
.Attr("use_locking: bool = false")
Meanwhile, also define ApplyPSGLD in the header file of tensorflow\core\kernels\training_ops.h
template <typename Device, typename T>
struct ApplyPSGLD {
...
};
To realize the computation of our algorithm on C++ side, complement corresponding code in the kernel of tensorflow\core\kernels\training_ops.cc.
After all, when I run tensorflow/models/image/mnist/convolutional.py, and the optimizer is adjusted,
optimizer = tf.train.PSGLDOptimizer(learning_rate).minimize(loss, global_step=batch)
an AttributeError happens:
AttributeError: 'module' object has no attribute 'PSGLDOptimizer'
And the environment is TF-0.9, cudnn5. So I ask if someone can give me any advice on this issue or the whole procedure of adding an optimizer.
(I'm assuming that you've rebuilt TensorFlow from source, as Olivier suggested in his comment, and you are trying to construct your optimizer as optimizer = tf.train.PSGLDOptimizer(...).)
To add a symbol to the tf.train namespace, you have to do the following:
Add an explicit import to the file tensorflow/python/training/training.py. In that file, you can see imports for, e.g., the tf.train.RMSPropOptimizer class.
Either:
Add documentation for your new class, and add ##PSGLDOptimizer to the module docstring. The corresponding line for tf.train.RMSPropOptimizer is here. This marks the class as a publicly documented API symbol.
Add an exception to the whitelist of symbols that are added to __all__ in that file. For example, this line whitelists the symbol tf.train.LooperThread.
For most TensorFlow modules*, the rule is that a symbol can appear in __all__ if it is either (i) publicly documented, or (ii) explicitly whitelisted. If neither condition holds, it will not be accessible through a tf.* name. This is intended to keep the API surface small, and avoid exposing private implementation details that might change between versions.
* Note however that this is a work in progress. At present, a method is considered to be stable only if it is documented in the public API docs.
I want to programmatically retrieve the value of the word wrap setting for GEdit3 from inside a Python plugin.
The GtkSettings class provides a method to set a string property, but how does one retrieve the value of a string property? I see no "getter" method.
I have also consulted pydoc gi.repository.Gtk.Settings - the methods listed there are the same as the online docs.
I am able to retrieve the property value of interest with the gsettings CLI utility. The command gsettings get org.gnome.gedit.preferences.editor wrap-mode
yields the value 'word'. I was hoping not to have to use subprocess.Popen() just to retrieve this property, however.
This will work
from gi.repository import Gio
a = Gio.Settings('org.gnome.gedit.preferences.editor')
a.get_string('wrap-mode')
Since you're using automatic generated bindings, C code samples will work just fine for you, it is just about changing the syntax.