I understand that the module! type provides a better structure for protected namespaces than object! or the 'use function. How are words bound within the module—I notice some errors related to unbound words:
REBOL [Type: 'module] set 'foo "Bar"
Also, how does Rebol distinguish between a word local to the module ('foo) and that of a system function ('set)?
Minor update, shortly after:
I see there's a switch that changes the method of binding:
REBOL [Type: 'module Options: [isolate]] set 'foo "Bar"
What does this do differently? What gotchas are there in using this method by default?
OK, this is going to be a little tricky.
In Rebol 3 there are no such things as system words, there are just words. Some words have been added to the runtime library lib, and set is one of those words, which happens to have a function assigned to it. Modules import words from lib, though what "import" means depends on the module options. That might be more tricky than you were expecting, so let me explain.
Regular Modules
For starters, I'll go over what importing means for "regular" modules, ones that don't have any options specified. Let's start with your first module:
REBOL [Type: 'module] set 'foo "Bar"
First of all, you have a wrong assumption here: The word foo is not local to the module, it's just the same as set. If you want to define foo as a local word you have to use the same method as you do with objects, use the word as a set-word at the top level, like this:
REBOL [Type: 'module] foo: "Bar"
The only difference between foo and set is that you hadn't exported or added the word foo to lib yet. When you reference words in a module that you haven't declared as local words, it has to get their values and/or bindings from somewhere. For regular modules, it binds the code to lib first, then overrides that by binding the code again to the module's local context. Any words defined in the local context will be bound to it. Any words not defined in the local context will retain their old bindings, in this case to lib. That is what "importing" means for regular modules.
In your first example, assuming that you haven't done so yourself, the word foo was not added to the runtime library ahead of time. That means that foo wasn't bound to lib, and since it wasn't declared as a local word it wasn't bound to the local context either. So as a result, foo wasn't bound to anything at all. In your code that was an error, but in other code it might not be.
Isolated Modules
There is an "isolate" option that changes the way that modules import stuff, making it an "isolated" module. Let's use your second example here:
REBOL [Type: 'module Options: [isolate]] set 'foo "Bar"
When an isolated module is made, every word in the module, even in nested code, is collected into the module's local context. In this case, it means that set and foo are local words. The initial values of those words are set to whatever values they have in lib at the time the module is created. That is, if the words are defined in lib at all. If the words don't have values in lib, they won't initially have values in the module either.
It is important to note that this import of values is a one-time thing. After that initial import, any changes to these words made outside the module don't affect the words in the module. That is why we say the module is "isolated". In the case of your code example, it means that someone could change lib/set and it wouldn't affect your code.
But there's another important module type you missed...
Scripts
In Rebol 3, scripts are another kind of module. Here's your code as a script:
REBOL [] set 'foo "Bar"
Or if you like, since script headers are optional in Rebol 3:
set 'foo "Bar"
Scripts also import their words from lib, and they import them into an isolated context, but with a twist: All scripts share the same isolated context, known as the "user" context. This means that when you change the value of a word in a script, the next script to use that word will see the change when it starts. So if after running the above script, you try to run this one:
print foo
Then it will print "Bar", rather than have foo be undefined, even though foo is still not defined in lib. You might find it interesting to know that if you are using Rebol 3 interactively, entering commands into the console and getting results, that every command line you enter is a separate script. So if your session looks like this:
>> x: 1
== 1
>> print x
1
The x: 1 and print x lines are separate scripts, the second taking advantage of the changes made to the user context by the first.
The user context is actually supposed to be task-local, but for the moment let's ignore that.
Why the difference?
Here is where we get back to the "system function" thing, and that Rebol doesn't have them. The set function is just like any other function. It might be implemented differently, but it's still a normal value assigned to a normal word. An application will have to manage a lot of these words, so that's why we have modules and the runtime library.
In an application there will be stuff that needs to change, and other stuff that needs to not change, and which stuff is which depends on the application. You will want to group your stuff, to keep things organized or for access control. There will be globally defined stuff, and locally defined stuff, and you will want to have an organized way to get the global stuff to the local places, and vice-versa, and resolve any conflicts when more than one thing wants to define stuff with the same name.
In Rebol 3, we use modules to group stuff, for convenience and access control. We use the runtime library lib as a place to collect the exports of the modules, and resolve conflicts, in order to control what gets imported to the local places like other modules and the user context(s). If you need to override some stuff, you do this by changing the runtime library, and if necessary propagating your changes out to the user context(s). You can even upgrade modules at runtime, and have the new version of the module override the words exported by the old version.
For regular modules, when things are overridden or upgraded, your module will benefit from such changes. Assuming those changes are a benefit, this can be a good thing. A regular module cooperates with other regular modules and scripts to make a shared environment to work in.
However, sometimes you need to stay separate from these kinds of changes. Perhaps you need a particular version of some function and don't want to be upgraded. Perhaps your module will be loaded in a less trustworthy environment and you don't want your code hacked. Perhaps you just need things to be more predictable. In cases like this, you may want to isolate your module from these kinds of external changes.
The downside to being isolated is that, if there are changes to the runtime library that you might want, you're not going to get them. If your module is somehow accessible (such as by having been imported with a name), someone might be able to propagate those changes to you, but if you're not accessible then you're out of luck. Hopefully you've thought to monitor lib for changes you want, or reference the stuff through lib directly.
Still, we've missed another important issue...
Exporting
The other part of managing the runtime library and all of these local contexts is exporting. You have to get your stuff out there somehow. And the most important factor is something that you wouldn't suspect: whether or not your module has a name.
Names are optional for Rebol 3's modules. At first this might seem like just a way to make it simpler to write modules (and in Carl's original proposal, that is exactly why). However, it turns out that there is a lot of stuff that you can do when you have a name that you can't when you don't, simply because of what a name is: a way to refer to something. If you don't have a name, you don't have a way to refer to something.
It might seem like a trivial thing, but here are some things that a name lets you do:
You can tell whether a module is loaded.
You can make sure a module is only loaded once.
You can tell whether an older version of a module was there earlier, and maybe upgrade it.
You can get access to a module that was loaded earlier.
When Carl decided to make names optional, he gave us a situation where it would be possible to make modules for which you couldn't do any of those things. Given that module exports were intended to be collected and organized in the runtime library, we had a situation where you could have effects on the library that you couldn't easily detect, and modules that got reloaded every time they were imported.
So for safety we decided to cut out the runtime library completely and just export words from these unnamed modules directly to the local (module or user) contexts that were importing them. This makes these modules effectively private, as if they are owned by the target contexts. We took a potentially awkward situation and made it a feature.
It was such a feature that we decided to support it explicitly with a private option. Making this an explicit option helps us deal with the last problem not having a name caused us: making private modules not have to reload over and over again. If you give a module a name, its exports can still be private, but it only needs one copy of what it's exporting.
However, named or not, private or not, that is 3 export types.
Regular Named Modules
Let's take this module:
REBOL [type: module name: foo] export bar: 1
Importing this adds a module to the loaded modules list, with the default version of 0.0.0, and exports one word bar to the runtime library. "Exporting" in this case means adding a word bar to the runtime library if it isn't there, and setting that word lib/bar to the value that the word foo/bar has after foo has finished executing (if it isn't set already).
It is worth noting that this automatic exporting happens only once, when the body of foo is finished executing. If you make a change to foo/bar after that, that doesn't affect lib/bar. If you want to change lib/bar too, you have to do it manually.
It is also worth noting that if lib/bar already exists before foo is imported, you won't have another word added. And if lib/bar is already set to a value (not unset), importing foo won't overwrite the existing value. First come, first served. If you want to override an existing value of lib/bar, you'll have to do so manually. This is how we use lib to manage overrides.
The main advantages that the runtime library gives us is that we can manage all of our exported words in one place, resolving conflicts and overrides. However, another advantage is that most modules and scripts don't actually have to say what they are importing. As long as the runtime library is filled in properly ahead of time with all the words you need, your script or module that you load later will be fine. This makes it easy to put a bunch of import statements and any overrides in your startup code which sets up everything the rest of your code will need. This is intended to make it easier to organize and write your application code.
Named Private Modules
In some cases, you don't want to export your stuff to the main runtime library. Stuff in lib gets imported into everything, so you should only export stuff to lib that you want to make generally available. Sometimes you want to make modules that only export stuff for the contexts that want it. Sometimes you have some related modules, a general facility and a utility module or so. If this is the case, you might want to make a private module.
Let's take this module:
REBOL [type: module name: foo options: [private]] export bar: 1
Importing this module doesn't affect lib. Instead, its exports are collected into a private runtime library that is local to the module or user context that is importing this module, along with those of any other private modules that the target is importing, then imported to the target from there. The private runtime library is used for the same conflict resolution that lib is used for. The main runtime library lib takes precedence over the private lib, so don't count on the private lib overriding global things.
This kind of thing is useful for making utility modules, advanced APIs, or other such tricks. It is also useful for making strong-modular code which requires explicit imports, if that is what you're into.
It's worth noting that if your module doesn't actually export anything, there is no difference between a named private module or a named public module, so it's basically treated as public. All that matters is that it has a name. Which brings us to...
Unnamed Modules
As explained above, if your module doesn't have a name then it pretty much has to be treated as private. More than private though, since you can't tell if it's loaded, you can't upgrade it or even keep from reloading it. But what if that's what you want?
In some cases, you really want your code run for effect. In these cases having your code rerun every time is what you want to do. Maybe it's a script that you're running with do but structuring as a module to avoid leaking words. Maybe you're making a mixin, some utility functions that have some local state that needs initializing. It could be just about anything.
I frequently make my %rebol.r file an unnamed module because I want to have more control over what it exports and how. Plus, since it's done for effect and doesn't need to be reloaded or upgraded there's no point in giving it a name.
No need for a code example, your earlier ones will act this way.
I hope this gives you enough of an overview of the design of R3's module system.
Related
How do I structure Raku code so that certain symbols are public within the the library I am writing, but not public to users of the library? (I'm saying "library" to avoid the terms "distribution" and "module", which the docs sometimes use in overlapping ways. But if there's a more precise term that I should be using, please let me know.)
I understand how to control privacy within a single file. For example, I might have a file Foo.rakumod with the following contents:
unit module Foo;
sub private($priv) { #`[do internal stuff] }
our sub public($input) is export { #`[ code that calls &private ] }
With this setup, &public is part of my library's public API, but &private isn't – I can call it within Foo, but my users cannot.
How do I maintain this separation if &private gets large enough that I want to split it off into its own file? If I move &private into Bar.rakumod, then I will need to give it our (i.e., package) scope and export it from the Bar module in order to be able to use it from Foo. But doing so in the same way I exported &public from Foo would result in users of my library being able to use Foo and call &private – exactly the outcome I am trying to avoid. How do maintain &private's privacy?
(I looked into enforcing privacy by listing Foo as a module that my distribution provides in my META6.json file. But from the documentation, my understanding is that provides controls what modules package managers like zef install by default but do not actually control the privacy of the code. Is that correct?)
[EDIT: The first few responses I've gotten make me wonder whether I am running into something of an XY problem. I thought I was asking about something in the "easy things should be easy" category. I'm coming at the issue of enforcing API boundaries from a Rust background, where the common practice is to make modules public within a crate (or just to their parent module) – so that was the X I asked about. But if there's a better/different way to enforce API boundaries in Raku, I'd also be interested in that solution (since that's the Y I really care about)]
I will need to give it our (i.e., package) scope and export it from the Bar module
The first step is not necessary. The export mechanism works just as well on lexically scoped subs too, and means they are only available to modules that import them. Since there is no implicit re-export, the module user would have to explicitly use the module containing the implementation details to have them in reach. (As an aside, personally, I pretty much never use our scope for subs in my modules, and rely entirely on exporting. However, I see why one might decide to make them available under a fully qualified name too.)
It's also possible to use export tags for the internal things (is export(:INTERNAL), and then use My::Module::Internals :INTERNAL) to provide an even stronger hint to the module user that they're voiding the warranty. At the end of the day, no matter what the language offers, somebody sufficiently determined to re-use internals will find a way (even if it's copy-paste from your module). Raku is, generally, designed with more of a focus on making it easy for folks to do the right thing than to make it impossible to "wrong" things if they really want to, because sometimes that wrong thing is still less wrong than the alternatives.
Off the bat, there's very little you can't do, as long as you're in control of the meta-object protocol. Anything that's syntactically possible, you could in principle do it using a specific kind of method, or class, declared using that. For instance, you could have a private-class which would be visible only to members of the same namespace (to the level that you would design). There's Metamodel::Trusting which defines, for a particular entity, who it does trust (please bear in mind that this is part of the implementation, not spec, and then subject to change).
A less scalable way would be to use trusts. The new, private modules would need to be classes and issue a trusts X for every class that would access it. That could include classes belonging to the same distribution... or not, that's up to you to decide. It's that Metamodel class above who supplies this trait, so using it directly might give you a greater level of control (with a lower level of programming)
There is no way to enforce this 100%, as others have said. Raku simply provides the user with too much flexibility for you to be able to perfectly hide implementation details externally while still sharing them between files internally.
However, you can get pretty close with a structure like the following:
# in Foo.rakumod
use Bar;
unit module Foo;
sub public($input) is export { #`[ code that calls &private ] }
# In Bar.rakumod
unit module Bar;
sub private($priv) is export is implementation-detail {
unless callframe(1).code.?package.^name eq 'Foo' {
die '&private is a private function. Please use the public API in Foo.' }
#`[do internal stuff]
}
This function will work normally when called from a function declared in the mainline of Foo, but will throw an exception if called from elsewhere. (Of course, the user can catch the exception; if you want to prevent that, you could exit instead – but then a determined user could overwrite the &*EXIT handler! As I said, Raku gives users a lot of flexibility).
Unfortunately, the code above has a runtime cost and is fairly verbose. And, if you want to call &private from more locations, it would get even more verbose. So it is likely better to keep private functions in the same file the majority of the time – but this option exists for when the need arises.
I prefer working with files that are less than 1000 lines long, so am thinking of breaking up some Erlang modules into more bite-sized pieces.
Is there a way of doing this without expanding the public API of my library?
What I mean is, any time there is a module, any user can do module:func_exported_from_the_module. The only way to really have something be private that I know of is to not export it from any module (and even then holes can be poked).
So if there is technically no way to accomplish what I'm looking for, is there a convention?
For example, there are no private methods in Python classes, but the convention is to use a leading _ in _my_private_method to mark it as private.
I accept that the answer may be, "no, you must have 4K LOC files."
The closest thing to a convention is to use edoc tags, like #private and #hidden.
From the docs:
#hidden
Marks the function so that it will not appear in the
documentation (even if "private" documentation is generated). Useful
for debug/test functions, etc. The content can be used as a comment;
it is ignored by EDoc.
#private
Marks the function as private (i.e., not part of the public
interface), so that it will not appear in the normal documentation.
(If "private" documentation is generated, the function will be
included.) Only useful for exported functions, e.g. entry points for
spawn. (Non-exported functions are always "private".) The content can
be used as a comment; it is ignored by EDoc.
Please note that this answer started as a comment to #legoscia's answer
Different visibilities for different methods is not currently supported.
The current convention, if you want to call it that way, is to have one (or several) 'facade' my_lib.erl module(s) that export the public API of your library/application. Calling any internal module of the library is playing with fire and should be avoided (call them at your own risk).
There are some very nice features in the BEAM VM that rely on being able to call exported functions from any module, such as
Callbacks (funs/anonymous funs), MFA, erlang:apply/3: The calling code does not need to know anything about the library, just that it's something that needs to be called
Behaviours such as gen_server need the previous point to work
Hot reloading: You can upgrade the bytecode of any module without stopping the VM. The code server inside the VM maintains at most two versions of the bytecode for any module, redirecting external calls (those with the Module:) to the most recent version and the internal calls to the current version. That's why you may see some ?MODULE: calls in long-running servers, to be able to upgrade the code
You'd be able to argue that these points'd be available with more fine-grained BEAM-oriented visibility levels, true. But I don't think it would solve anything that's not solved with the facade modules, and it'd complicate other parts of the VM/code a great deal.
Bonus
Something similar applies to records and opaque types, records only exist at compile time, and opaque types only at dialyzer time. Nothing stops you from accessing their internals anywhere, but you'll only find problems if you go that way:
You insert a new field in the record, suddenly, all your {record_name,...} = break
You use a library that returns an opaque_adt(), you know that it's a list and use like so. The library is upgraded to include the size of the list, so now opaque_adt() is a tuple() and chaos ensues
Only those functions that are specified in the -export attribute are visible to other modules i.e "public" functions. All other functions are private. If you have specified -compile(export_all) only then all functions in module are visible outside. It is not recommended to use -compile(export_all).
I don't know of any existing convention for Erlang, but why not adopt the Python convention? Let's say that "library-private" functions are prefixed with an underscore. You'll need to quote function names with single quotes for that to work:
-module(bar).
-export(['_my_private_function'/0]).
'_my_private_function'() ->
foo.
Then you can call it as:
> bar:'_my_private_function'().
foo
To me, that communicates clearly that I shouldn't be calling that function unless I know what I'm doing. (and probably not even then)
Do I really have to use the ClassImp macro to benefit the automatic dictionary and streamer generation in ROOT? Some online tutorials and examples mention it but I noticed that simply adding the ClassDef(MyClass, <ver>) macro to MyClass.h and processing it with rootcint/rootcling already generates most of such code.
I did look at Rtypes.h where these macros are defined but to follow preprocessor macros calling each other is not easy and so, it would be nice if experts could confirm the role of ClassImp. I am specifically interested in recent versions of ROOT >= 5.34
Here is the answer I got on roottalk mailing list confirming that the usage of ClassImp is essentially outdated.
ClassImp is used to register in the TClass the name of the source file
for the class. This was used in particular by THtml (which has now
been deprecated in favor of Doxygen). So unless you code/framework
needs to know the name of the source files, it is no longer necessary
to have ClassImp.
ClassDef is necessary for class inheriting from TObject (or from any
classes that has a ClassDef). In the other cases, it provide
accelerator that makes the I/O slightly faster (and thus is
technically not compulsory in this case). It also assign a version
number to the schema layout which simplifies writing schema evolution
rules (on the other hand, there is other alternative to assign a
version number to the schema layout).
What exactly are you trying to do? The ClassImp and ClassDef macros add members to the class that provide Run-Time Type Information and allow the class to be written to root files. If you are not interested in that, then don't bother with these macros.
I never use them.
The project I'm working on, being rather large, consists of one master module, which I'd like to be the API interface, with a number of sub-modules defined within it. This is being done as follows:
<Library.ts>
module Library { }
<Core/Core.ts>
module Library.Core {}
Often the submodules will span a number of files. The problem I'm having is in such situations, one file cannot use non-exported properties defined within the same sub-module, but in another file.
Is there any way I can use these properties, or failing that, any way I can prevent the entirety of a sub-module's exports being exposed within its parent module?
Is there any way I can use these properties, or failing that, any way I can prevent the entirety of a sub-module's exports being exposed within its parent module?
No. You need to export from module Foo for it to be available to module Foo in another file. The same applies to submodules
My XCode project has grown somewhat, and I know that there are class files in there which are no longer being used. Is there an easy way to find all of these and remove them?
If the class files just sit in your project without being part of a target, just click on the project itself in the tree view, so you see all files in the table. Make sure you see the "Target" column in the table view, iterate through your targets and find the files that don't have a check anywhere -> they are no longer compiled.
But if you still compile the classes and they are no longer used, that case is a bit more difficult. Check out this project
http://www.karppinen.fi/analysistool/#dependency-graphs
You could create a dependency graph and try to find orphaned classes that way.
Edit: Link went dead, but there still seem to be projects of Objective-C dependency graphs around, for example https://github.com/nst/objc_dep
if they are C or C++ symbols, then you can just let the linker do the work for you.
if you're looking to remove objc symbols, then try to refactor the class name (e.g. to rename the class), and preview the dependencies that it turns up. if you reference classes/selectors/etc. by strings then... it may not be so effective. unfortunately, you often have to also test manually, to verify that removing a class does not break anything. remember that resources (like xibs) may reference/load objc classes as well.
This is a tricky question due to how dynamic objective-c is as you can never guarantee that a class is not going to be used.
Consider if you generate a class name and a selector at run time and then look up that class, instantiate that class and then call a method on that newly created object using that newly created selector. No where in your code do you explicitly name and instantiate that object but you are able to use it anyways. You could get that class name and selector name from anywhere outside of your code, even from some data from a server some where. How would you ever know which class is not going to be used? Because of this there are no tools that are able to perform what you are requesting.
Searching the project with the class name might be an option, thought it may not be the best solution. Specially it might be time consuming when you have many classes.