I have 4 files all in the same directory: main.rakumod, infix_ops.rakumod, prefix_ops.rakumod and script.raku:
main module has a class definition (class A)
*_ops modules have some operator routine definitions to write, e.g., $a1 + $a2 in an overloaded way.
script.raku tries to instantaniate A object(s) and use those user-defined operators.
Why 3 files not 1? Since class definition might be long and separating overloaded operator definitions in files seemed like a good idea for writing tidier code (easier to manage).
e.g.,
# main.rakumod
class A {
has $.x is rw;
}
# prefix_ops.rakumod
use lib ".";
use main;
multi prefix:<++>(A:D $obj) {
++$obj.x;
$obj;
}
and similar routines in infix_ops.rakumod. Now, in script.raku, my aim is to import main module only and see the overloaded operators also available:
# script.raku
use lib ".";
use main;
my $a = A.new(x => -1);
++$a;
but it naturally doesn't see ++ multi for A objects because main.rakumod doesn't know the *_ops.rakumod files as it stands. Is there a way I can achieve this? If I use prefix_ops in main.rakumod, it says 'use lib' may not be pre-compiled perhaps because of circular dependentness
it says 'use lib' may not be pre-compiled
The word "may" is ambiguous. Actually it cannot be precompiled.
The message would be better if it said something to the effect of "Don't put use lib in a module."
This has now been fixed per #codesections++'s comment below.
perhaps because of circular dependentness
No. use lib can only be used by the main program file, the one directly run by Rakudo.
Is there a way I can achieve this?
Here's one way.
We introduce a new file that's used by the other packages to eliminate the circularity. So now we have four files (I've rationalized the naming to stick to A or variants of it for the packages that contribute to the type A):
A-sawn.rakumod that's a role or class or similar:
unit role A-sawn;
Other packages that are to be separated out into their own files use the new "sawn" package and does or is it as appropriate:
use A-sawn;
unit class A-Ops does A-sawn;
multi prefix:<++>(A-sawn:D $obj) is export { ++($obj.x) }
multi postfix:<++>(A-sawn:D $obj) is export { ($obj.x)++ }
The A.rakumod file for the A type does the same thing. It also uses whatever other packages are to be pulled into the same A namespace; this will import symbols from it according to Raku's standard importing rules. And then relevant symbols are explicitly exported:
use A-sawn;
use A-Ops;
sub EXPORT { Map.new: OUTER:: .grep: /'fix:<'/ }
unit class A does A-sawn;
has $.x is rw;
Finally, with this setup in place, the main program can just use A;:
use lib '.';
use A;
my $a = A.new(x => -1);
say $a++; # A.new(x => -1)
say ++$a; # A.new(x => 1)
say ++$a; # A.new(x => 2)
The two main things here are:
Introducing an (empty) A-sawn package
This type eliminates circularity using the technique shown in #codesection's answer to Best Way to Resolve Circular Module Loading.
Raku culture has a fun generic term/meme for techniques that cut through circular problems: "circular saws". So I've used a -sawn suffix of the "sawn" typename as a convention when using this technique.[1]
Importing symbols into a package and then re-exporting them
This is done via sub EXPORT { Map.new: ... }.[2] See the doc for sub EXPORT.
The Map must contain a list of symbols (Pairs). For this case I've grepped through keys from the OUTER:: pseudopackage that refers to the symbol table of the lexical scope immediately outside the sub EXPORT the OUTER:: appears in. This is of course the lexical scope into which some symbols (for operators) have just been imported by the use Ops; statement. I then grep that symbol table for keys containing fix:<; this will catch all symbol keys with that string in their name (so infix:<..., prefix:<... etc.). Alter this code as needed to suit your needs.[3]
Footnotes
[1] As things stands this technique means coming up with a new name that's different from the one used by the consumer of the new type, one that won't conflict with any other packages. This suggests a suffix. I think -sawn is a reasonable choice for an unusual and distinctive and mnemonic suffix. That said, I imagine someone will eventually package this process up into a new language construct that does the work behind the scenes, generating the name and automating away the manual changes one has to make to packages with the shown technique.
[2] A critically important point is that, if a sub EXPORT is to do what you want, it must be placed outside the package definition to which it applies. And that in turn means it must be before a unit package declaration. And that in turn means any use statement relied on by that sub EXPORT must appear within the same or outer lexical scope. (This is explained in the doc but I think it bears summarizing here to try head off much head scratching because there's no error message if it's in the wrong place.)
[3] As with the circularity saw aspect discussed in footnote 1, I imagine someone will also eventually package up this import-and-export mechanism into a new construct, or, perhaps even better, an enhancement of Raku's built in use statement.
Hi #hanselmann here is how I would write this (in 3 files / same dir):
Define my class(es):
# MyClass.rakumod
unit module MyClass;
class A is export {
has $.x is rw;
}
Define my operators:
# Prefix_Ops.rakumod
unit module Prefix_Ops;
use MyClass;
multi prefix:<++>(A:D $obj) is export {
++$obj.x;
$obj;
}
Run my code:
# script.raku
use lib ".";
use MyClass;
use Prefix_Ops;
my $a = A.new(x => -1);
++$a;
say $a.x; #0
Taking my cue from the Module docs there are a couple of things I am doing different:
Avoiding the use of main (or Main, or MAIN) --- I am wary that MAIN is a reserved name and just want to keep clear of engaging any of that (cool) machinery
Bringing in the unit module declaration at the top of each 'rakumod' file ... it may be possible to use bare files in Raku ... but I have never tried this and would say that it is not obvious from the docs that it is even possible, or supported
Now since I wanted this to work first time you will note that I use the same file name and module name ... again it may be possible to do that differently (multiple modules in one file and so on) ... but I have not tried that either
Using the 'is export' trait where I want my script to be able to use these definitions ... as you will know from close study of the docs ;-) is that each module has it's own namespace (the "stash") and we need export to shove the exported definitions into the namespace of the script
As #raiph mentions you only need the script to define the module library location
Since you want your prefix multi to "know" about class A then you also need to use MyClass in the Prefix_Ops module
Anyway, all-in-all, I think that the raku module system exemplifies the unique combination of "easy things easy and hard thinks doable" ... all I had to do with your code (which was very close) was tweak a few filenames and sprinkle in some concise concepts like 'unit module' and 'is export' and it really does not look much different since raku keeps all the import/export machinery under the surface like the swan gliding over the river...
There is a lot of pyboard module which can be use by micropython. Currently I just know that these module's real implementation is done in C. My question is:
How is the relationship mapped between the Python module and C implementation?
Such as we can use import pyb, where is the pyb Python file?
Such as we can use from pyb import LED and call the intensity function, where is the Python LED class definition? Where is the definition of its intensity function?
Easiest way to find this out is clone the source code and then start looking around using whatever text/file search tool you prefer. Search for files/text 'pyb' and/or 'LED'. Then you'll find for instance modpyb.c which defines the pyb module (in C, not in Python). There you can see the module's global dictionary has an entry
{ MP_ROM_QSTR(MP_QSTR_LED), MP_ROM_PTR(&pyb_led_type) }
which is MicroPython lingo to say 'there's a thing with the name LED and it is of type pyb_led_type. The latter being the the C code for the LED class and found in led.c, including the led_obj_intensity function.
There are two packages I want to use: CorpusLoaders.jl, and WordNet.jl
CorpusLoaders.SemCor exports sensekey(::SenseTaggedWord)
WordNet exports sensekey(::DB, ::Synset, ::Lemma)
I want to use both sensekey methods.
Eg
for some mixed list of items: mixedlist::Vector{Union{Tuple{SenseTaggedWord},Tuple{DB, Synset,Lemma}}.
Ie the items in the list are a mixture of 1-tuples of SenseTaggedWord, and3 tuples of DB, Synset, and Lemma.
for item in mixedlist
println(sensekey(item...)
end
should work.
This example is a little facetious, since why would I be mixing them like this.
But, hopefully it serves for illustrating the problem in the general case.
Trying to using CorpusLoaders.SemCor, WordNet to bring in both results in WARNING: both WordNet and Semcor export "sensekey"; uses of it in module Main must be qualified.
Manually importing both: import CorpusLoaders.SemCor.sensekey; import WordNet.sensekey results in WARNING: ignoring conflicting import of Semcor.sensekey into Main
What can be done? I want them both, and they don't really conflict, due to multiple-dispatch.
Given that CorpusLoaders.jl is a package I am writing I do have a few more options, since I could make my CorpusLoaders.jl depend on WordNet.jl.
If I did do than then I could say in CorpusLoaders.jl
import WordNet
function WordNet.sensekey(s::SenseTaggedWord)...
and that would make them both work.
But it would mean requiring WordNet as a dependency of CorpusLoaders.
And I want to know how to solve the problem for a consumer of the packages -- not as the creator of the packages.
tl;dr qualify the functions when using them in your script via their module namespace, i.e. CorpusLoader.sensekey() and WordNet.sensekey()
Explanation
My understanding of your question after the edits (thank you for clarifying) is that:
You have written a package called CorpusLoaders.jl, which exports the function sensekey(::SenseTaggedWord)
There is an external package called WordNet.jl, which exports the function sensekey(::DB, ::Synset, ::Lemma)
You have a script that makes use of both modules.
and you are worried that using the modules or "importing" the functions directly could potentially create ambiguity and / or errors in your script, asking
how can I write my CorpusLoaders package to prevent potential clashes with other packages, and
how can I write my script to clearly disambiguate between the two functions while still allowing their use?
I think this stems from a slight confusion how using and import are different from each other, and how modules create a namespace. This is very nicely explained in the docs here.
In essence, the answers are:
You should not worry about exporting things from your module that will clash with other modules. This is what modules are for: you're creating a namespace, which will "qualify" all exported variables, e.g. CorpusLoaders.sensekey(::SenseTaggedWord).
When you type using CorpusLoaders, what you're saying to julia is "import the module itself, and all the exported variables stripped from their namespace qualifier, and bring them into Main". Note that this means you now have access to sensekey as a function directly from Main without a namespace qualifier, and as CorpusLoaders.sensekey(), since you've also imported the module as a variable you can use.
If you then try using the module WordNet as well, julia very reasonably issues a warning, which essentially says:
"You've imported two functions that have the same name. I can't just strip their namespace off because that could create problems in some scenarios (even though in your case it wouldn't because they have different signatures, but I couldn't possibly know this in general). If you want to use either of these functions, please do so using their appropriate namespace qualifier".
So, the solution for 2. is:
you either do
using CorpusLoaders;
using WordNet;
, disregarding the warning, to import all other exported variables as usual in your Main namespace, and access those particular functions directly via their modules as CorpusLoaders.sensekey() and WordNet.sensekey() each time you need to use them in your script, or
you keep both modules clearly disambiguated at all times by doing
import CorpusLoaders;
import WordNet;
and qualify all variables appropriately, or
in this particular case where the function signatures don't clash, if you'd really like to be able to use the function without a namespace qualifier, relying on multiple dispatch instead, you can do something like what FengYang suggested:
import CorpusLoaders;
import WordNet;
sensekey(a::SenseTaggedWord) = CorpusLoader.sensekey(a);
sensekey(a::DB, b::Synset, c::Lemma) = WordNet.sensekey(a, b, c);
which is essentially a new function, defined on module Main, acting as a wrapper for the two namespace-qualified functions.
In the end, it all comes down to using using vs import and namespaces appropriately for your particular code. :)
As an addendum, code can get very unwieldy with long namespace qualifiers like CorpusLoader and WordNet. julia doesn't have something like python's import numpy as np, but at the same time modules become simple variables on your workspace, so it's trivial to create an alias for them. So you can do:
import CorpusLoaders; const cl = CorpusLoaders;
import Wordnet; const wn = WordNet;
# ... code using both cl.sensekey() and wn.sensekey()
In this case, the functions do not conflict, but in general that is impossible to guarantee. It could be the case that a package loaded later will add methods to one of the functions that will conflict. So to be able to use the sensekey for both packages requires some additional guarantees and restrictions.
One way to do this is to ignore both package's sensekey, and instead provide your own, dispatching to the correct package:
sensekey(x) = CorpusLoaders.sensekey(x)
sensekey(x, y, z) = WordNet.sensekey(x,y,z)
I implemented what #Fengyang Wang said,
as a function:
function importfrom(moduleinstance::Module, functionname::Symbol, argtypes::Tuple)
meths = methods(moduleinstance.(functionname), argtypes)
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol)
meths = methods(moduleinstance.(functionname))
importfrom(moduleinstance, functionname, meths)
end
function importfrom(moduleinstance::Module, functionname::Symbol, meths::Base.MethodList)
for mt in meths
paramnames = collect(mt.lambda_template.slotnames[2:end])
paramtypes = collect(mt.sig.parameters[2:end])
paramsig = ((n,t)->Expr(:(::),n,t)).(paramnames, paramtypes)
funcdec = Expr(:(=),
Expr(:call, functionname, paramsig...),
Expr(:call, :($moduleinstance.$functionname), paramnames...)
)
current_module().eval(funcdec) #Runs at global scope, from calling module
end
end
Call with:
using WordNet
using CorpusLoaders.Semcor
importfrom(CorpusLoaders.Semcor, :sensekey)
importfrom(WordNet, :sensekey)
methods(sensekey)
2 methods for generic function sensekey:
sensekey(db::WordNet.DB, ss::WordNet.Synset, lem::WordNet.Lemma)
sensekey(saword::CorpusLoaders.Semcor.SenseAnnotatedWord
If you wanted to get really flash you could reexport the DocString too.
I'm unable to import and use a module I've written, not sure where I went wrong.
My starting directory structure is:
/src/main.rs <- contains fn main()
/Cargo.toml
I wanted to refactor some code out of one big main file I had, so I ran
cargo new mnemonic
leaving me with:
/src/main.rs <- contains fn main()
/Cargo.toml
/mnemonic/src/lib.rs
/mnemonic/Cargo.toml
in the /mnemonic/src/lib.rs that was created I put:
pub mod mnemonic;
then put my code in /mnemonic/src/mnemonic.rs (has 2 public functions I want to use from main). I ran cargo build from mnemonic/ and back in my main src, I tried
extern crate mnemonic;
and
use mnemonic;
I also tried use mnemonic::mnemonic;
In all instances, it was unable to find the crate or gave an unresolved import error.
I've also tried putting my code containing the 2 pub functions in /src/mnemonic.rs
and in /src/main.rs putting:
use mnemonic;
I also tried just putting my code in mnemonic/src/lib.rs and trying extern crate mnemonic; use mnemonic; from src/main.rs
Am I supposed to edit Cargo.toml in any of the instances you proposed?
Updated Answer
Yes, you need to tell Cargo where to find your crate. In your binaries Cargo.toml, add something like:
[dependencies.mnemonic]
path = "./mnemonic"
Original Answer
When you create a new crate, it automatically has one layer of namespacing: the name of the crate itself. In addition, you don't need to extern use a crate if it is your current crate!
Using crate mnemonic from other crates
This is probably what you want to do: Create a new crate called mnemonic, then you will have a src/lib.rs. Put your code in there, and you can then use it from other crates with
extern crate mnemonic;
use mnemonic::ItemInLibDotRs; // Make sure this item is marked as `pub`!
Using a nested module mnemonic from the crate mnemonic
Say you have some implementation detail you want to hide from the end user. We will put it in a module within the crate, and use it only from within the same crate.
Put your code in src/mnemonic.rs. In src/lib.rs, you will reference that module and items:
use mnemonic::ItemInMnemonic; // Make sure this is `pub`!
mod mnemonic;
Using a nested module mnemonic from the crate mnemonic in other crates
Export the module from your crate, and then reference it from other crates:
// src/lib.rs
pub mod mnemonic; // Note the module is now public
// In the crate that consumes mnemonic
extern crate mnemonic;
use mnemonic::mnemonic::ItemInMnemonic;
This last option is kind of ugly though, you probably don't want to make your users type mnemonic::mnemonic, which is why I suggest just putting it in src/lib.rs.
All examples that I found create a lib.rs and then inside that file create mod foo;, mod bar; for each file foo.rs, bar.rs and so on.
Basically what I want is to to split my crate into multiple files but without having to introduce lots of modules for every file. Is this not possible with Rust?
In order to include another source file, it has to be its own module.
However, this does not mean that you must expose this fact in your API. For example, let's say you want to put some code in a file called hidden.rs:
pub fn inside_hidden() {}
Now in your lib.rs file, you could expose this module:
#![crate_type = "lib"]
pub mod hidden;
But if your library doesn't require sub-modules, it's probably better to keep the hidden module a secret, and instead expose its contents as members of your top-level module:
#![crate_type = "lib"]
pub use hidden::inside_hidden;
mod hidden;
This way, users won't know you have a hidden sub-module. They will see inside_hidden as if it were a member of your top-level module.