I've been reading a lot of Julia documentation (version 0.4) and am still having problems with loading Julia files. This seems like it should be really easy. So, plainly and simply, how are we supposed to use Julia code from other files directly in our current code? And, as a related, helpful bonus, is there any history or language design decisions that, understood, would illuminate the situation?
P.S. I'm using 0.4.
If you want problem specifics, here are some things I'm dealing with:
First
Using the REPL, I want to use some functions I have written in a different file. Supposedly, I should be able to load said file like this:
julia> using Foobar
That just gives me ArgumentErrors no matter what I do. I've tried including it before trying to use it:
julia> include("Foobar.jl")
julia> using Foobar
I've also tried updating the load path before trying to use it:
julia> push!(LOAD_PATH, "/Users/me/julia")
julia> using Foobar
Second
When I try to fix the first problem by including the file before using it, I get an error for any line that has: using .... The message is that a module cannot be found in path. Or in other words, I'm trying to load a module in the current working directory that depends on another module in the current working directory. When I include the file I'm trying to load, it tries to find the dependency and cannot.
Third
I've tried relative paths. I.e. I'm in the same directory as the .jl file and do:
julia> using .Foobar
if you use include("/path/to/myscript.jl") then you should then have access to any functions, objects, etc. defined in the file you called with include(). No additional calls to using should be needed.
Here is an answer that gives more info on details for creating whole packages (rather than just individual scripts like in the example above), how to do them, and how the using terminology factors in with them: julia: create and use a local package without Internet . For instance, packages must be installed in a particular path relative to your other julia files, not just in the arbitrary working directory that your script is in.
See also here for a longer tutorial on packages.
It seems to work well enough here:
julia> push!(LOAD_PATH, "/Users/me/julia")
2-element Array{ByteString,1}:
"/Applications/Julia-0.4.5.app/Contents/Resources/julia/local/share/julia/site/v0.4"
"/Users/me/julia"
julia> readdir(LOAD_PATH[end])
1-element Array{ByteString,1}:
"MyModule.jl"
julia> using MyModule
julia> x
"Hi there"
where MyModule.jl contains:
module MyModule
export x
x = "Hi there"
end
Related
I have different set of math expressions that must be evaluated at run-time. Currently the task is done by replacing symbols with equivalent values and eval the result. (could be done by any existing symbolic packages)
Now, refer to the definition of modules in Julia-lang:
Modules in Julia are separate variable workspaces, i.e. they introduce
a new global scope .... Modules allow you to create top-level definitions (aka
global variables) without worrying about name conflicts when your code
is used together with somebody else’s.
And with the power of Julia to do meta-things,
I'm wondering if it is possible to create anonymous modules at run-time m=Module(), and use them as a scope to evaluate expressions eval(m, :(a+b)).
But I simply can't find a way to load variable into the run-time modules.
Although I could get result with:
julia> ex=:(module mo; a=1; b=4; end)
julia> eval(ex)
julia> eval(mo,:(a+b))
I prefer more functional way, using anonymous modules.
Any Help.
This works:
julia> m=Module()
anonymous
julia> eval(m, :(a=5))
5
julia> m.a
5
julia> eval(m, :(a))
5
julia> eval(m, :(2a))
10
There are two packages I want to use: CorpusLoaders.jl, and WordNet.jl
CorpusLoaders.SemCor exports sensekey(::SenseTaggedWord)
WordNet exports sensekey(::DB, ::Synset, ::Lemma)
I want to use both sensekey methods.
Eg
for some mixed list of items: mixedlist::Vector{Union{Tuple{SenseTaggedWord},Tuple{DB, Synset,Lemma}}.
Ie the items in the list are a mixture of 1-tuples of SenseTaggedWord, and3 tuples of DB, Synset, and Lemma.
for item in mixedlist
println(sensekey(item...)
end
should work.
This example is a little facetious, since why would I be mixing them like this.
But, hopefully it serves for illustrating the problem in the general case.
Trying to using CorpusLoaders.SemCor, WordNet to bring in both results in WARNING: both WordNet and Semcor export "sensekey"; uses of it in module Main must be qualified.
Manually importing both: import CorpusLoaders.SemCor.sensekey; import WordNet.sensekey results in WARNING: ignoring conflicting import of Semcor.sensekey into Main
What can be done? I want them both, and they don't really conflict, due to multiple-dispatch.
Given that CorpusLoaders.jl is a package I am writing I do have a few more options, since I could make my CorpusLoaders.jl depend on WordNet.jl.
If I did do than then I could say in CorpusLoaders.jl
import WordNet
function WordNet.sensekey(s::SenseTaggedWord)...
and that would make them both work.
But it would mean requiring WordNet as a dependency of CorpusLoaders.
And I want to know how to solve the problem for a consumer of the packages -- not as the creator of the packages.
tl;dr qualify the functions when using them in your script via their module namespace, i.e. CorpusLoader.sensekey() and WordNet.sensekey()
Explanation
My understanding of your question after the edits (thank you for clarifying) is that:
You have written a package called CorpusLoaders.jl, which exports the function sensekey(::SenseTaggedWord)
There is an external package called WordNet.jl, which exports the function sensekey(::DB, ::Synset, ::Lemma)
You have a script that makes use of both modules.
and you are worried that using the modules or "importing" the functions directly could potentially create ambiguity and / or errors in your script, asking
how can I write my CorpusLoaders package to prevent potential clashes with other packages, and
how can I write my script to clearly disambiguate between the two functions while still allowing their use?
I think this stems from a slight confusion how using and import are different from each other, and how modules create a namespace. This is very nicely explained in the docs here.
In essence, the answers are:
You should not worry about exporting things from your module that will clash with other modules. This is what modules are for: you're creating a namespace, which will "qualify" all exported variables, e.g. CorpusLoaders.sensekey(::SenseTaggedWord).
When you type using CorpusLoaders, what you're saying to julia is "import the module itself, and all the exported variables stripped from their namespace qualifier, and bring them into Main". Note that this means you now have access to sensekey as a function directly from Main without a namespace qualifier, and as CorpusLoaders.sensekey(), since you've also imported the module as a variable you can use.
If you then try using the module WordNet as well, julia very reasonably issues a warning, which essentially says:
"You've imported two functions that have the same name. I can't just strip their namespace off because that could create problems in some scenarios (even though in your case it wouldn't because they have different signatures, but I couldn't possibly know this in general). If you want to use either of these functions, please do so using their appropriate namespace qualifier".
So, the solution for 2. is:
you either do
using CorpusLoaders;
using WordNet;
, disregarding the warning, to import all other exported variables as usual in your Main namespace, and access those particular functions directly via their modules as CorpusLoaders.sensekey() and WordNet.sensekey() each time you need to use them in your script, or
you keep both modules clearly disambiguated at all times by doing
import CorpusLoaders;
import WordNet;
and qualify all variables appropriately, or
in this particular case where the function signatures don't clash, if you'd really like to be able to use the function without a namespace qualifier, relying on multiple dispatch instead, you can do something like what FengYang suggested:
import CorpusLoaders;
import WordNet;
sensekey(a::SenseTaggedWord) = CorpusLoader.sensekey(a);
sensekey(a::DB, b::Synset, c::Lemma) = WordNet.sensekey(a, b, c);
which is essentially a new function, defined on module Main, acting as a wrapper for the two namespace-qualified functions.
In the end, it all comes down to using using vs import and namespaces appropriately for your particular code. :)
As an addendum, code can get very unwieldy with long namespace qualifiers like CorpusLoader and WordNet. julia doesn't have something like python's import numpy as np, but at the same time modules become simple variables on your workspace, so it's trivial to create an alias for them. So you can do:
import CorpusLoaders; const cl = CorpusLoaders;
import Wordnet; const wn = WordNet;
# ... code using both cl.sensekey() and wn.sensekey()
In this case, the functions do not conflict, but in general that is impossible to guarantee. It could be the case that a package loaded later will add methods to one of the functions that will conflict. So to be able to use the sensekey for both packages requires some additional guarantees and restrictions.
One way to do this is to ignore both package's sensekey, and instead provide your own, dispatching to the correct package:
sensekey(x) = CorpusLoaders.sensekey(x)
sensekey(x, y, z) = WordNet.sensekey(x,y,z)
I implemented what #Fengyang Wang said,
as a function:
function importfrom(moduleinstance::Module, functionname::Symbol, argtypes::Tuple)
meths = methods(moduleinstance.(functionname), argtypes)
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol)
meths = methods(moduleinstance.(functionname))
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol, meths::Base.MethodList)
for mt in meths
paramnames = collect(mt.lambda_template.slotnames[2:end])
paramtypes = collect(mt.sig.parameters[2:end])
paramsig = ((n,t)->Expr(:(::),n,t)).(paramnames, paramtypes)
funcdec = Expr(:(=),
Expr(:call, functionname, paramsig...),
Expr(:call, :($moduleinstance.$functionname), paramnames...)
)
current_module().eval(funcdec) #Runs at global scope, from calling module
end
end
Call with:
using WordNet
using CorpusLoaders.Semcor
importfrom(CorpusLoaders.Semcor, :sensekey)
importfrom(WordNet, :sensekey)
methods(sensekey)
2 methods for generic function sensekey:
sensekey(db::WordNet.DB, ss::WordNet.Synset, lem::WordNet.Lemma)
sensekey(saword::CorpusLoaders.Semcor.SenseAnnotatedWord
If you wanted to get really flash you could reexport the DocString too.
This is a fairly niche problem, but I'm currently trying to write a conventions-based settings storage library with golang. It would be a great API boon if I could programmatically determine the running package name that wants to store something (eg "github.net/author/projectname/pkg") calling my library function.
With Python a similar thing could be achieved with the inspect module, or even with __main__.__file__ and a look at the file system.
You can get similar information if you use the following functions:
runtime.Caller
runtime.FuncForPC
The code may look like this:
pc, file, line, ok := runtime.Caller(1)
if !ok { /*failed*/ }
println(pc, file, line, ok)
f := runtime.FuncForPC(pc)
if f == nil { /*failed*/ }
println(f.Name())
If I put the above code (with the 1st line changed into runtime.Caller(0)) into a (randomly chosen) Go library which I have installed in GOROOT, it prints:
134626026 /tmp/go-build223663414/github.com/mattn/go-gtk/gtk/_obj/gtk.cgo1.go -4585 true
github.com/mattn/go-gtk/gtk.Init
Or it prints:
134515752 /home/user/go/src/github.com/mattn/go-gtk/example/event/event.go 12 true
main.main
The filename on the 1st line, and the 2nd line, seem to contain the information you are looking for.
There are two problems:
It may give incorrect result if functions are automatically inlined by the compiler
For any function F defined in package main, the function name is just main.F. For example, if runtime.Caller(0) is called from main(), the function name is main.main even if the main() function is defined in a Go file found in GOROOT/src/github.com/mattn/go-gtk/.... In this case, the output from runtime.Caller is more useful than the output from runtime.FuncForPC.
I'm learning SciLab and I need to figure out the equivalent from MATLAB for running user-defined functions.
I'm used to MATLAB, where when you type foo(27), it looks for the foo.m script in the current directory and then the MATLAB path, and if it finds one, it calls that function with argument 27.
What's the equivalent to SciLab? It doesn't seem to want to look in the current directory for the appropriate .sci file.
In Scilab, you need to explicitly load the script that contains the function. Assuming you've changed your directory to the directory where the function file is loaded, which can be done in Scilab using the menu buttons or the following command:
cd("path/to/working/directory")
Now you load the function file. Assuming the function foo is stored in a file called foo.sci, you need to load this script using the following command:
exec("foo.sci")
Now you should be able to use your function as you would be able in MATLAB.
foo(27)
Let's say you have a Fortran 90 module containing lots of variables, functions and subroutines. In your USE statement, which convention do you follow:
explicitly declare which variables/functions/subroutines you're using with the , only : syntax, such as USE [module_name], only : variable1, variable2, ...?
Insert a blanket USE [module_name]?
On the one hand, the only clause makes the code a bit more verbose. However, it forces you to repeat yourself in the code and if your module contains lots of variables/functions/subroutines, things begin to look unruly.
Here's an example:
module constants
implicit none
real, parameter :: PI=3.14
real, parameter :: E=2.71828183
integer, parameter :: answer=42
real, parameter :: earthRadiusMeters=6.38e6
end module constants
program test
! Option #1: blanket "use constants"
! use constants
! Option #2: Specify EACH variable you wish to use.
use constants, only : PI,E,answer,earthRadiusMeters
implicit none
write(6,*) "Hello world. Here are some constants:"
write(6,*) PI, &
E, &
answer, &
earthRadiusInMeters
end program test
Update
Hopefully someone says something like "Fortran? Just recode it in C#!" so I can down vote you.
Update
I like Tim Whitcomb's answer, which compares Fortran's USE modulename with Python's from modulename import *. A topic which has been on Stack Overflow before:
‘import module’ or ‘from module import’
In an answer, Mark Roddy mentioned:
don't use 'from module import *'. For
any reasonable large set of code, if
you 'import *' your will likely be
cementing it into the module, unable
to be removed. This is because it is
difficult to determine what items used
in the code are coming from 'module',
making it east to get to the point
where you think you don't use the
import anymore but its extremely
difficult to be sure.
What are good rules of thumb for python imports?
dbr's answer contains
don't do from x import * - it makes
your code very hard to understand, as
you cannot easily see where a method
came from (from x import *; from y
import *; my_func() - where is my_func
defined?)
So, I'm leaning towards a consensus of explicitly stating all the items I'm using in a module via
USE modulename, only : var1, var2, ...
And as Stefano Borini mentions,
[if] you have a module so large that you
feel compelled to add ONLY, it means
that your module is too big. Split it.
I used to just do use modulename - then, as my application grew, I found it more and more difficult to find the source to functions (without turning to grep) - some of the other code floating around the office still uses a one-subroutine-per-file, which has its own set of problems, but it makes it much easier to use a text editor to move through the code and quickly track down what you need.
After experiencing this, I've become a convert to using use...only whenever possible. I've also started picking up Python, and view it the same way as from modulename import *. There's a lot of great things that modules give you, but I prefer to keep my global namespace tightly controlled.
It's a matter of balance.
If you use only a few stuff from the module, it makes sense if you add ONLY, to clearly specify what you are using.
If you use a lot of stuff from the module, specifying ONLY will be followed by a lot of stuff, so it makes less sense. You are basically cherry-picking what you use, but the true fact is that you are dependent on that module as a whole.
However, in the end the best philosophy is this one: if you are concerned about namespace pollution, and you have a module so large that you feel compelled to add ONLY, it means that your module is too big. Split it.
Update: Fortran? just recode it in python ;)
Not exactly answering the question here, just throwing in another solution that I have found useful in some circumstances, if for whatever reason you don't want to split your module and start to get namespace clashes. You can use derived types to store several namespaces in one module.
If there is some logical grouping of the variables, you can create your own derived type for each group, store an instance of this type in the module and then you can import just the group that you happen to need.
Small example: We have a lot of data some of which is user input and some that is the result of miscellaneous initializations.
module basicdata
implicit none
! First the data types...
type input_data
integer :: a, b
end type input_data
type init_data
integer :: b, c
end type init_data
! ... then declare the data
type(input_data) :: input
type(init_data) :: init
end module basicdata
Now if a subroutine only uses data from init, you import just that:
subroutine doesstuff
use basicdata, only : init
...
q = init%b
end subroutine doesstuff
This is definitely not a universally applicable solution, you get some extra verbosity from the derived type syntax and then it will of course barely help if your module is not the basicdata sort above, but instead more of a allthestuffivebeenmeaningtosortoutvariety. Anyway, I have had some luck in getting code that fits easier into the brain this way.
The main advantage of USE, ONLY for me is that it avoids polluting my global namespace with stuff I don't need.
Agreed with most answers previously given, use ..., only: ... is the way to go, use types when it makes sense, apply python thinking as much as possible. Another suggestion is to use appropriate naming conventions in your imported module, along with private / public statements.
For instance, the netcdf library uses nf90_<some name>, which limits the namespace pollution on the importer side.
use netcdf ! imported names are prefixed with "nf90_"
nf90_open(...)
nf90_create(...)
nf90_get_var(...)
nf90_close(...)
similarly, the ncio wrapper to this library uses nc_<some name> (nc_read, nc_write...).
Importantly, with such designs where use: ..., only: ... is made less relevant, you'd better control the namespace of the imported module by setting appropriate private / public attributes in the header, so that a quick look at it will be sufficient for readers to assess which level of "pollution" they are facing. This is basically the same as use ..., only: ..., but on the imported module side - thus to be written only once, not at each import).
One more thing: as far as object-orientation and python are concerned, a difference in my view is that fortran does not really encourage type-bound procedures, in part because it is a relatively new standard (e.g. not compatible with a number of tools, and less rationally, it is just unusual) and because it breaks handy behavior such as procedure-free derived type copy (type(mytype) :: t1, t2 and t2 = t1). That means you often have to import the type and all would-be type-bound procedures, instead of just the class. This alone makes fortran code more verbose compared to python, and practical solutions like a prefix naming convention may come in handy.
IMO, the bottom line is: choose your coding style for people who will read it (this includes your later self), as taught by python. The best is the more verbose use ..., only: ... at each import, but in some cases a simple naming convention will do it (if you are disciplined enough...).
Yes, please use use module, only: .... For large code bases with multiple programmers, it makes the code easier to follow by everyone (or just use grep).
Please do not use include, use a smaller module for that instead. Include is a text insert of source code which is not checked by the compiler at the same level as use module, see: FORTRAN: Difference between INCLUDE and modules. Include generally makes it harder for both humans and computer to use the code which means it should not be used. Ex. from mpi-forum: "The use of the mpif.h include file is strongly discouraged and may be deprecated in a future version of MPI." (http://mpi-forum.org/docs/mpi-3.1/mpi31-report/node411.htm).
I know I'm a little late to the party, but if you're only after a set of constants and not necessarily computed values, you could do like C and create an include file:
inside a file,
e.g., constants.for
real, parameter :: pi = 3.14
real, parameter :: g = 6.67384e-11
...
program main
use module1, only : func1, subroutine1, func2
implicit none
include 'constants.for'
...
end program main
Edited to remove "real(4)" as some think it is bad practice.