I I have a very simple question. Is it possible to load modules on demand in Julia. That is, can the modules be loaded when they are actually needed instead of being loaded at "parse-time" at the top level.
The use case scenario I have in mind is that I have some set of code that is able to do some plotting using PyPlot, but code is far from always executed.
At the moment this means that I have at top level a statement like using PyPlot, which takes quite a load of time to load.
(Yes i know: One should not restart Julia to often, bla bla bla... but nevertheless this is a point of annoyance)
Is there a way to ensure that PyPlot is only loaded is if is actually needed?
The simplest idea would have been to include the using PyPlot inside the function that actually do the plotting
function my_plot()
using PyPlot
plot(1:10,1:10)
end
but this results in a syntax error:
ERROR: syntax: "using" expression not at top level
So, is there another way to achieve this?
The "using" statement runs when the line of code is encountered, and does not have to be at the top of the file. It does need to be in global scope, which means that the variables in the module loaded with "using" will be available to all functions in your program after the "using" statement is executed, not just a single function as might happen in the local scope of a function.
If you call the using statement as an expression within a Julia eval statement, all code executed within the "eval" statement in Julia is automatically done so in global scope, even if the eval is syntactically called within a function's local scope. So if you use the macro #eval
function my_plot()
#eval using PyPlot # or without the macro, as eval(:(using PyPlot))
plot(1:10,1:10)
end
this acts as if the using PyPlot is done outside a function, and so avoids the syntax error.
Related
I'm writing a Julia macro that takes an expression and serializes it, so that I can run it somewhere else. The macro therefore takes the expression and replaces all symbols with variables. The expression is then serialized and evaluated somewhere else.
My problem is related to evaluating variables that are not in the global scope. I.e. the following works fine, as a is defined in the global scope:
macro myprintf(ex)
print(eval(ex))
end
# works
a = 2
#myprintf a
This throws an error as the macro doesn't see a, which is defined in the local scope of the loop (run in a new session):
macro myprintf(ex)
print(eval(ex))
end
# UndefVarError: a not defined
for j=1:3
a = 2
#myprintf a
end
Is there any way I can access a inside the macro if it is defined in a local scope such as a loop? I'm aware that I'm not necessarily using macros as intended as I am calling eval on the expression inside the macro definition. The overall idea is that I want to serialize the expression that's passed to the macro and evaluate it somewhere else later (e.g. in a different Julia session).
eval only works in global scope. But then I don't see how much use there is in replacing the variables by evaluated literals, resulting in an expression of literals.
Anyway, a different approach to the original problem could be mimick what R does: constructing thunks out of an expression plus its environment at the place where it was called. To recreate that in Julia, you'd have to
Figure out the free variables in the expression
Create an environment data structure of their values
Wrap the original expression in a closure such that it takes the local variables from the environment
Put everything into a new object
Example:
for j=1:3
a = 2
#saveexpr a + 2
end
should expand to something like
for j=1:3
a = 2
SerializedExpr(
(; a),
function (env)
let (a,) = env
a + 2
end
end,
:(a + 2))
end
If saving the closure is not feasible in your use case, I guess you have to either implement your own evaluator, or use something like JuliaInterpreter.jl.
I have a pde to solve. For optimisation, i am using forall loops and inside the loop, the variables change using a user defined function in the following way.
forall(i=2:n-1,j=2:n-1, w(i,j).gt.wmax/1000)
k3(i,j)=w(i,j)+h*k(x(i),t)
end forall
Here, the k(x,t) is an externally defined function I defined earlier.
The error comes:
Reference to impure function âkâ at (1) inside a FORALL block
I am using gfortran. What is the solution if I need a user defined function inside a forall loop? Is it possible at all inside a forall or I need to do some other thing, that also would optimise? If some other thing is possible, kindly explain that too.
The problem is that you are referencing an impure function called k inside the FORALL block. To get this to work when you write your function you must make it a pure one, and have an interface in scope at the point you call it in the loop - pure is an assertion that the function will not (amongst other things) change its arguments, which, if this were to occur, could make parallel processing of the Forall construct give incorrect answers. If you had given a complete, minimal program showing your problem I would have shown you the changes you need to make, but as you haven't, well I can't.
But really this is by the by. DON'T use FORALL. Almost certainly your program won't run any faster than using a simple do loop, and quite possibly slower. Forall seemed like a good idea at the time, but for a variety of reasons it hasn't really worked out - I note in the latest edition of "Modern Fortran Explained" by Metcalf, Reid and Cohen, the classic book on Fortran, they mark it as obsolescent. Instead I would look into the more modern Do Concurrent, or, probably best, learn how to parallelise your loop with OpenMP.
i'm in the process of making an interpreter, and i started with the ast classes.
The code is object oriented in c++.
Each action is a "statement", a block is a statement itself and it contains a list of statements.
Blocks also define different scopes, keeping record of what variables have been allocated within that scope and deleting them at the end of it.
The execution consists in the main block calling "execute" for every statement it contains, where each statement can consist in another block, a simple instruction or a function call.
With this structure i could think of how to implement most constructs, (while, if-else) but i don't understand how could i make a goto. Despite the fact gotos are almost not used anyway, i'd need the goto as a starting base to implement break and continue for loops.
Does anyone have a conceptual suggestion? (conceptually speaking, not actual code needed).
NOTE: the code is not executed line by line while being parsed, it's first entirely parsed into an ast, and then executed.
If you're gotoing to line numbers, just add an attribute to your AST node class indicating on which line it's declared in the source. When executing a goto node, find the highest node in the tree with that line number, and link that as your goto target.
For labels, you have a few options. You could turn the labels into an AST node, itself, or add an attribute to the node class which is only set if the previous statement is a label. When processing your goto, do the same thing as you'd do with line numbers. Find the top-most node that has the label attribute set.
This assumes that your targets appear in the source before the goto. Otherwise, you'll have to do a fixup pass to resolve the invocation targets, much like function calls.
There are probably better ways; this is just what comes to mind.
Essentially, your goto is a jmp, and you need a way to resolve the target address. I'd just parse the source, then do a second pass to fixup any gotos, which would have a pointer to the first AST node after the label.
There are two packages I want to use: CorpusLoaders.jl, and WordNet.jl
CorpusLoaders.SemCor exports sensekey(::SenseTaggedWord)
WordNet exports sensekey(::DB, ::Synset, ::Lemma)
I want to use both sensekey methods.
Eg
for some mixed list of items: mixedlist::Vector{Union{Tuple{SenseTaggedWord},Tuple{DB, Synset,Lemma}}.
Ie the items in the list are a mixture of 1-tuples of SenseTaggedWord, and3 tuples of DB, Synset, and Lemma.
for item in mixedlist
println(sensekey(item...)
end
should work.
This example is a little facetious, since why would I be mixing them like this.
But, hopefully it serves for illustrating the problem in the general case.
Trying to using CorpusLoaders.SemCor, WordNet to bring in both results in WARNING: both WordNet and Semcor export "sensekey"; uses of it in module Main must be qualified.
Manually importing both: import CorpusLoaders.SemCor.sensekey; import WordNet.sensekey results in WARNING: ignoring conflicting import of Semcor.sensekey into Main
What can be done? I want them both, and they don't really conflict, due to multiple-dispatch.
Given that CorpusLoaders.jl is a package I am writing I do have a few more options, since I could make my CorpusLoaders.jl depend on WordNet.jl.
If I did do than then I could say in CorpusLoaders.jl
import WordNet
function WordNet.sensekey(s::SenseTaggedWord)...
and that would make them both work.
But it would mean requiring WordNet as a dependency of CorpusLoaders.
And I want to know how to solve the problem for a consumer of the packages -- not as the creator of the packages.
tl;dr qualify the functions when using them in your script via their module namespace, i.e. CorpusLoader.sensekey() and WordNet.sensekey()
Explanation
My understanding of your question after the edits (thank you for clarifying) is that:
You have written a package called CorpusLoaders.jl, which exports the function sensekey(::SenseTaggedWord)
There is an external package called WordNet.jl, which exports the function sensekey(::DB, ::Synset, ::Lemma)
You have a script that makes use of both modules.
and you are worried that using the modules or "importing" the functions directly could potentially create ambiguity and / or errors in your script, asking
how can I write my CorpusLoaders package to prevent potential clashes with other packages, and
how can I write my script to clearly disambiguate between the two functions while still allowing their use?
I think this stems from a slight confusion how using and import are different from each other, and how modules create a namespace. This is very nicely explained in the docs here.
In essence, the answers are:
You should not worry about exporting things from your module that will clash with other modules. This is what modules are for: you're creating a namespace, which will "qualify" all exported variables, e.g. CorpusLoaders.sensekey(::SenseTaggedWord).
When you type using CorpusLoaders, what you're saying to julia is "import the module itself, and all the exported variables stripped from their namespace qualifier, and bring them into Main". Note that this means you now have access to sensekey as a function directly from Main without a namespace qualifier, and as CorpusLoaders.sensekey(), since you've also imported the module as a variable you can use.
If you then try using the module WordNet as well, julia very reasonably issues a warning, which essentially says:
"You've imported two functions that have the same name. I can't just strip their namespace off because that could create problems in some scenarios (even though in your case it wouldn't because they have different signatures, but I couldn't possibly know this in general). If you want to use either of these functions, please do so using their appropriate namespace qualifier".
So, the solution for 2. is:
you either do
using CorpusLoaders;
using WordNet;
, disregarding the warning, to import all other exported variables as usual in your Main namespace, and access those particular functions directly via their modules as CorpusLoaders.sensekey() and WordNet.sensekey() each time you need to use them in your script, or
you keep both modules clearly disambiguated at all times by doing
import CorpusLoaders;
import WordNet;
and qualify all variables appropriately, or
in this particular case where the function signatures don't clash, if you'd really like to be able to use the function without a namespace qualifier, relying on multiple dispatch instead, you can do something like what FengYang suggested:
import CorpusLoaders;
import WordNet;
sensekey(a::SenseTaggedWord) = CorpusLoader.sensekey(a);
sensekey(a::DB, b::Synset, c::Lemma) = WordNet.sensekey(a, b, c);
which is essentially a new function, defined on module Main, acting as a wrapper for the two namespace-qualified functions.
In the end, it all comes down to using using vs import and namespaces appropriately for your particular code. :)
As an addendum, code can get very unwieldy with long namespace qualifiers like CorpusLoader and WordNet. julia doesn't have something like python's import numpy as np, but at the same time modules become simple variables on your workspace, so it's trivial to create an alias for them. So you can do:
import CorpusLoaders; const cl = CorpusLoaders;
import Wordnet; const wn = WordNet;
# ... code using both cl.sensekey() and wn.sensekey()
In this case, the functions do not conflict, but in general that is impossible to guarantee. It could be the case that a package loaded later will add methods to one of the functions that will conflict. So to be able to use the sensekey for both packages requires some additional guarantees and restrictions.
One way to do this is to ignore both package's sensekey, and instead provide your own, dispatching to the correct package:
sensekey(x) = CorpusLoaders.sensekey(x)
sensekey(x, y, z) = WordNet.sensekey(x,y,z)
I implemented what #Fengyang Wang said,
as a function:
function importfrom(moduleinstance::Module, functionname::Symbol, argtypes::Tuple)
meths = methods(moduleinstance.(functionname), argtypes)
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol)
meths = methods(moduleinstance.(functionname))
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol, meths::Base.MethodList)
for mt in meths
paramnames = collect(mt.lambda_template.slotnames[2:end])
paramtypes = collect(mt.sig.parameters[2:end])
paramsig = ((n,t)->Expr(:(::),n,t)).(paramnames, paramtypes)
funcdec = Expr(:(=),
Expr(:call, functionname, paramsig...),
Expr(:call, :($moduleinstance.$functionname), paramnames...)
)
current_module().eval(funcdec) #Runs at global scope, from calling module
end
end
Call with:
using WordNet
using CorpusLoaders.Semcor
importfrom(CorpusLoaders.Semcor, :sensekey)
importfrom(WordNet, :sensekey)
methods(sensekey)
2 methods for generic function sensekey:
sensekey(db::WordNet.DB, ss::WordNet.Synset, lem::WordNet.Lemma)
sensekey(saword::CorpusLoaders.Semcor.SenseAnnotatedWord
If you wanted to get really flash you could reexport the DocString too.
While modifying an existing program's CASE statement, I had to add a second block where some logic is repeated to set NetWeaver portal settings. This is done by setting values in a local variable, then assigning that variable to a Changing parameter. I copied over the code and did a Pretty Print, expecting to compiler to complain about the unknown variable. To my surprise however, this code actually compiles just fine:
CASE i_actionid.
WHEN 'DOMIGO'.
DATA: ls_portal_actions TYPE powl_follow_up_sty.
CLEAR ls_portal_actions.
ls_portal_actions-bo_system = 'SAP_ECC_Common'.
" [...]
c_portal_actions = ls_portal_actions.
WHEN 'EBELN'.
ls_portal_actions-bo_system = 'SAP_ECC_Common'.
" [...]
C_PORTAL_ACTIONS = ls_portal_actions.
ENDCASE.
As I have seen in every other programming language, the DATA: declaration in the first WHEN statement should be encapsulated and available only inside that switch block. Does SAP ignore this encapsulation to make that value available in the entire CASE statement? Is this documented anywhere?
Note that this code compiles just fine and double-clicking the local variable in the second switch takes me to the data declaration in the first. I have however not been able to test that this code executes properly as our testing environment is down.
In short you cannot do this. You will have the following scopes in an abap program within which to declare variables (from local to global):
Form routine: all variables between FORM and ENDFORM
Method: all variables between METHOD and ENDMETHOD
Class - all variables between CLASS and ENDCLASS but only in the CLASS DEFINITION section
Function module: all variables between FUNCTION and ENDFUNCTION
Program/global - anything not in one of the above is global in the current program including variables in PBO and PAI modules
Having the ability to define variables locally in a for loop or if is really useful but unfortunately not possible in ABAP. The closest you will come to publicly available documentation on this is on help.sap.com: Local Data in the Subroutine
As for the compile process do not assume that ABAP will optimize out any variables you do not use it won't, use the code inspector to find and remove them yourself. Since ABAP works the way it does I personally define all my variables at the start of a modularization unit and not inline with other code and have gone so far as to modify the pretty printer to move any inline definitions to the top of the current scope.
Your assumption that a CASE statement defines its own scope of variables in ABAP is simply wrong (and would be wrong for a number of other programming languages as well). It's a bad idea to litter your code with variable declarations because that makes it awfully hard to read and to maintain, but it is possible. The DATA statements - as well as many other declarative statements - are only evaluated at compile time and are completely ignored at runtime. You can find more information about the scopes in the online documentation.
The inline variable declarations are now possible with the newest version of SAP Netweaver. Here is the link to the documentation DATA - inline declaration. Here are also some guidelines of a good and bad usage of this new feature
Here is a quote from this site:
A declaration expression with the declaration operator DATA declares a variable var used as an operand in the current writer position. The declared variable is visible statically in the program from DATA(var) and is valid in the current context. The declaration is made when the program is compiled, regardless of whether the statement is actually executed.
Personally have not had time to check it out yet, because of lack of access to such system.