In Node.js, I could get an array of the objects in foo with
Object.keys(require("foo"));
Is there any way I could do the same thing in Rust?
mod foo;
getobjs(foo);
No, there is no way to do this. This level of introspection of compile-time information simply doesn't exist at runtime. The concept of a module doesn't even exist.
If you are interested in compile-time information, you can do such things as build and view the docs (cargo doc --open) to see all the public items of the entire crate. You can probably also view the crate's documentation online before you use it.
There are also tools like the Rust Language Server which provide this type of information (and more) to editors and IDEs.
Related
How do I structure Raku code so that certain symbols are public within the the library I am writing, but not public to users of the library? (I'm saying "library" to avoid the terms "distribution" and "module", which the docs sometimes use in overlapping ways. But if there's a more precise term that I should be using, please let me know.)
I understand how to control privacy within a single file. For example, I might have a file Foo.rakumod with the following contents:
unit module Foo;
sub private($priv) { #`[do internal stuff] }
our sub public($input) is export { #`[ code that calls &private ] }
With this setup, &public is part of my library's public API, but &private isn't – I can call it within Foo, but my users cannot.
How do I maintain this separation if &private gets large enough that I want to split it off into its own file? If I move &private into Bar.rakumod, then I will need to give it our (i.e., package) scope and export it from the Bar module in order to be able to use it from Foo. But doing so in the same way I exported &public from Foo would result in users of my library being able to use Foo and call &private – exactly the outcome I am trying to avoid. How do maintain &private's privacy?
(I looked into enforcing privacy by listing Foo as a module that my distribution provides in my META6.json file. But from the documentation, my understanding is that provides controls what modules package managers like zef install by default but do not actually control the privacy of the code. Is that correct?)
[EDIT: The first few responses I've gotten make me wonder whether I am running into something of an XY problem. I thought I was asking about something in the "easy things should be easy" category. I'm coming at the issue of enforcing API boundaries from a Rust background, where the common practice is to make modules public within a crate (or just to their parent module) – so that was the X I asked about. But if there's a better/different way to enforce API boundaries in Raku, I'd also be interested in that solution (since that's the Y I really care about)]
I will need to give it our (i.e., package) scope and export it from the Bar module
The first step is not necessary. The export mechanism works just as well on lexically scoped subs too, and means they are only available to modules that import them. Since there is no implicit re-export, the module user would have to explicitly use the module containing the implementation details to have them in reach. (As an aside, personally, I pretty much never use our scope for subs in my modules, and rely entirely on exporting. However, I see why one might decide to make them available under a fully qualified name too.)
It's also possible to use export tags for the internal things (is export(:INTERNAL), and then use My::Module::Internals :INTERNAL) to provide an even stronger hint to the module user that they're voiding the warranty. At the end of the day, no matter what the language offers, somebody sufficiently determined to re-use internals will find a way (even if it's copy-paste from your module). Raku is, generally, designed with more of a focus on making it easy for folks to do the right thing than to make it impossible to "wrong" things if they really want to, because sometimes that wrong thing is still less wrong than the alternatives.
Off the bat, there's very little you can't do, as long as you're in control of the meta-object protocol. Anything that's syntactically possible, you could in principle do it using a specific kind of method, or class, declared using that. For instance, you could have a private-class which would be visible only to members of the same namespace (to the level that you would design). There's Metamodel::Trusting which defines, for a particular entity, who it does trust (please bear in mind that this is part of the implementation, not spec, and then subject to change).
A less scalable way would be to use trusts. The new, private modules would need to be classes and issue a trusts X for every class that would access it. That could include classes belonging to the same distribution... or not, that's up to you to decide. It's that Metamodel class above who supplies this trait, so using it directly might give you a greater level of control (with a lower level of programming)
There is no way to enforce this 100%, as others have said. Raku simply provides the user with too much flexibility for you to be able to perfectly hide implementation details externally while still sharing them between files internally.
However, you can get pretty close with a structure like the following:
# in Foo.rakumod
use Bar;
unit module Foo;
sub public($input) is export { #`[ code that calls &private ] }
# In Bar.rakumod
unit module Bar;
sub private($priv) is export is implementation-detail {
unless callframe(1).code.?package.^name eq 'Foo' {
die '&private is a private function. Please use the public API in Foo.' }
#`[do internal stuff]
}
This function will work normally when called from a function declared in the mainline of Foo, but will throw an exception if called from elsewhere. (Of course, the user can catch the exception; if you want to prevent that, you could exit instead – but then a determined user could overwrite the &*EXIT handler! As I said, Raku gives users a lot of flexibility).
Unfortunately, the code above has a runtime cost and is fairly verbose. And, if you want to call &private from more locations, it would get even more verbose. So it is likely better to keep private functions in the same file the majority of the time – but this option exists for when the need arises.
I prefer working with files that are less than 1000 lines long, so am thinking of breaking up some Erlang modules into more bite-sized pieces.
Is there a way of doing this without expanding the public API of my library?
What I mean is, any time there is a module, any user can do module:func_exported_from_the_module. The only way to really have something be private that I know of is to not export it from any module (and even then holes can be poked).
So if there is technically no way to accomplish what I'm looking for, is there a convention?
For example, there are no private methods in Python classes, but the convention is to use a leading _ in _my_private_method to mark it as private.
I accept that the answer may be, "no, you must have 4K LOC files."
The closest thing to a convention is to use edoc tags, like #private and #hidden.
From the docs:
#hidden
Marks the function so that it will not appear in the
documentation (even if "private" documentation is generated). Useful
for debug/test functions, etc. The content can be used as a comment;
it is ignored by EDoc.
#private
Marks the function as private (i.e., not part of the public
interface), so that it will not appear in the normal documentation.
(If "private" documentation is generated, the function will be
included.) Only useful for exported functions, e.g. entry points for
spawn. (Non-exported functions are always "private".) The content can
be used as a comment; it is ignored by EDoc.
Please note that this answer started as a comment to #legoscia's answer
Different visibilities for different methods is not currently supported.
The current convention, if you want to call it that way, is to have one (or several) 'facade' my_lib.erl module(s) that export the public API of your library/application. Calling any internal module of the library is playing with fire and should be avoided (call them at your own risk).
There are some very nice features in the BEAM VM that rely on being able to call exported functions from any module, such as
Callbacks (funs/anonymous funs), MFA, erlang:apply/3: The calling code does not need to know anything about the library, just that it's something that needs to be called
Behaviours such as gen_server need the previous point to work
Hot reloading: You can upgrade the bytecode of any module without stopping the VM. The code server inside the VM maintains at most two versions of the bytecode for any module, redirecting external calls (those with the Module:) to the most recent version and the internal calls to the current version. That's why you may see some ?MODULE: calls in long-running servers, to be able to upgrade the code
You'd be able to argue that these points'd be available with more fine-grained BEAM-oriented visibility levels, true. But I don't think it would solve anything that's not solved with the facade modules, and it'd complicate other parts of the VM/code a great deal.
Bonus
Something similar applies to records and opaque types, records only exist at compile time, and opaque types only at dialyzer time. Nothing stops you from accessing their internals anywhere, but you'll only find problems if you go that way:
You insert a new field in the record, suddenly, all your {record_name,...} = break
You use a library that returns an opaque_adt(), you know that it's a list and use like so. The library is upgraded to include the size of the list, so now opaque_adt() is a tuple() and chaos ensues
Only those functions that are specified in the -export attribute are visible to other modules i.e "public" functions. All other functions are private. If you have specified -compile(export_all) only then all functions in module are visible outside. It is not recommended to use -compile(export_all).
I don't know of any existing convention for Erlang, but why not adopt the Python convention? Let's say that "library-private" functions are prefixed with an underscore. You'll need to quote function names with single quotes for that to work:
-module(bar).
-export(['_my_private_function'/0]).
'_my_private_function'() ->
foo.
Then you can call it as:
> bar:'_my_private_function'().
foo
To me, that communicates clearly that I shouldn't be calling that function unless I know what I'm doing. (and probably not even then)
I'm trying to learn PyQt5 and I am finding it very difficult since I can't just guess what methods are available. I've just spent an entire week trying to find a method to simulate a button push. I eventually found the solution ( QPushButton.animateClick() ) only after stumbling across an example someone left out there (how did this person know this?). It's very difficult to develop without some reference to what's available for tools!
Riverbank has a version of what I'm looking for but it is not complete making it virtually useless.
pyqt5 being a qt5 binding has almost all the functionalities (there are minimal known incompatibilities) so the qt5 documentation: https://doc.qt.io/ is valid for pyqt5 except for small exceptions.
Although the target of the documentation is c++ the description of the classes and methods are generic, so they also validly apply for pyqt5, on the other hand many of the examples are written in c++ but the translation to python in many cases is trivial .
So to avoid doing a double task it seems that Riverbank Computing Limited only documents the exceptions indicated in the pyqt5 docs: https://www.riverbankcomputing.com/static/Docs/PyQt5/
The next part of my answer will propose tips to handle the Qt documentation.
The Qt documentation also has an easy to understand structure, for example let's analyze the QPushButton class (https://doc.qt.io/qt-5/qpushbutton.html):
At the top there is a table:
This table indicates how to include the class in a C++ project, how to add it to qmake, from which class it inherits, and which classes inherit from it. From the above, relevant information for PyQt5 can be extracted, such as to which sub-module the class belongs to: In this case we use QT += widgets that inform us that it belongs to the QtWidgets sub-module, in general if Qt += submodulefoo belongs to QtSubModuleFoo (camelcase)
If you want to know all the methods of the QPushButton class, you must use the "List of all members, including inherited members" link below the table, in this case the link will be https://doc.qt.io/qt-5/qpushbutton-members.html where is the complete list of all class methods, enumerations, etc.
Other tips to understand the conversion between Qt/C++ and PyQt5/Python are:
Some methods use pointers to receive information such as:
void QLayout::getContentsMargins(int *left, int *top, int *right, int *bottom) const
bool QProcess::startDetached(qint64 *pid = nullptr), etc
those transformed to PyQt5 as:
lay = QtWidgets.QXLayout()
left, top, right, bottom = lay.getContentsMargins()
process = QProcess()
# ...
ok, pid = process.startDetached()
Some methods collide with reserved words such as exec , raise, print, etc so to avoid incompatibilities, the underscore is added at the end: exec_, raise_, print_, etc
In Qt, the Q_SLOT and Q_SIGNAL that are translated into python are used through the #pyqtSlot and #pyqtSignal decorators.
In conclusion, my recommendation is that you use the Qt and PyQt5 documentation at the same time to know all the functionalities, in addition there are many Q&A in SO about translations from one language to another so you could learn from there.
The Qt documentation can also be consulted using the Qt Assistant tool.
The main PyQt5 documentation is on the official website:
https://www.riverbankcomputing.com/static/Docs/PyQt5/
But it's still incomplete, and most parts refer to the official Qt documentation:
https://doc.qt.io/qt-5/
While that's C++ oriented, consider that almost every module, class and function behave exactly in the same way as it does in python, so it's usually better to use that.
Consider that:
in the function lists you'll always see the returned type on the left of each function;
"void" means that the function returns None;
when overriding some existing method (expecially private and virtual), you always have to return the expected types listed for that function;
function arguments are usually written in the form [const] Type argName=default: you can usually ignore the "const" part (it's a C++ term), while the argName for keyword arguments might be different in PyQt;
some functions could have different names, since they're reserved on python (print, raise, etc); in those cases, an underscore is always appended;
some positional or keyword arguments might be different, or the return type signature might; that's because in C++ you can use a pointer to a variable as an argument, and the function will change that variable using the pointer (this is an oversimplification);
all "properties" are not python properties, and they are only accessible through their parenthesis functions, such as self.width() an self.setWidth();
some methods have different overrides, in some cases python has special cases with different arguments that are not available in C++, and viceversa; also, some methods don't exist at all in one case or the other;
My suggestion is to always use the official documentation, it's only a matter of habit to get used to the C++ references (and you'll see that it is educational too); whenever some doubt raises, check the PyQt documentation to see what's different and use the help command in the python shell.
I'm aware this might be a broad question (there's no specific code for you to look at), but I'm hoping I'd get some insights as to what to do, or how to approach the problem.
To keep things simple, suppose the compiler that I'm writing performs these three steps:
parse (and bind all variables)
typecheck
codegen
Also the language that I'm building the compiler for wants to support late-analysis/late-binding (ie., it has a function that takes a String, which is to be compiled and executed as a piece of source-code during runtime).
Now during parse-phase, I have a piece of context that I need to keep around till run-time for the sole benefit of the aforementioned function (because it needs to parse and typecheck its argument in that context).
So the question, how should I do this? What do other compilers do?
Should I just serialise the context object to disk (codegen for it) and resurrect it during run-time or something?
Thanks
Yes, you'll need to emit the type information (or other context, you weren't very specific) in your object/executable files, so that your eval can read it at runtime. You might look at Java's .class file format for inspiration; Java doesn't have eval as such, but you can dynamically spin new bytecode at runtime that must be linked in a type-safe manner. David Conrad's comment is spot-on: this information can also be used to implement reflection, if your language has such a feature.
That's as much as I can help you without more specifics.
This guy came up with a pretty neat tool to generate a class dependency graph - however, it relies on parsing your source code and looking for #import directives.
http://seriot.ch/blog.php?article=20110124
https://github.com/nst/objc_dep/blob/master/objc-dep.py
This is neat, but I have a number of problems with this. Not least of which is it doesn't take into account imports of imports nor prefix headers nor whether-or-not the class(es) in the file referenced by the import are actually being used.
I'd like to do something more akin to class-dump and examine the Objective-C metadata stored in the Mach-O file to generate an in-memory representation of the class dependencies.
I'd rather not do this from scratch, so I'm wondering:
Has it already been done?
Is there an open-source library which would provide me with the foundational tools I need to extract this information (a library which examines the Mach-O file and creates a façade of the Objective-C information contained within - such that I could iterate over all of the classes, their methods, properties, ivars, etc and scan for references to other classes) I figure class-dump's source would be a good place to start.
If you have experience in this sort of thing, is what I'm trying to accomplish feasible?
What roadblocks will I need to overcome?
Has it already been done?
Not that I know of.
Is there an open-source library which would provide me with the
foundational tools I need to extract this information?
At the core of class-dump is libMachObjC which does exatly what you want, i.e. parse all classes/methods/ivars and more. The API is very clean, it should be very easy to use.
If you have experience in this sort of thing, is what I'm trying to
accomplish feasible?
Unfortunately, no because some classes don't declare the real class but use id instead. For example, here is the information that can be extracted from a class-dump of UIKit:
#interface UITableView : UIScrollView <NSCoding>
{
int _style;
id <UITableViewDataSource> _dataSource;
id _rowData;
...
The _rowData ivar type information is id but if you check at runtime you will see that _rowData is an instance of the UITableViewRowData class. This information is not present in the Mach-O binary so you have no way to find the relation between UITableView and UITableViewRowData. The same applies for method parameters.
Here's a solution that relies on information in mach.o files, and generates graph dependency based on that information: https://github.com/PaulTaykalo/objc-dependency-visualizer
Has it already been done?
yes - but i can't recommend a good public implementation
Is there an open-source library which would provide me with the foundational tools I need to extract this information (a library which examines the Mach-O file and creates a façade of the Objective-C information contained within - such that I could iterate over all of the classes, their methods, properties, ivars, etc and scan for references to other classes) I figure class-dump's source would be a good place to start.
most use cases would benefit by using the objc runtime facilities objc/... rather than examining the binary.
If you have experience in this sort of thing, is what I'm trying to accomplish feasible?
yes. i've done something similar using the objc runtime.
What roadblocks will I need to overcome?
that depends largely on the level of detail you want... implementation time if you find no such implementation, but i figure you will find a few options if you google the more esoteric functions in the objc runtime; perhaps you would find one in an (open) language binding or bridge?
if you do end up writing one yourself, you can get registered objc classes using objc_getClassList, then access the properties/information you want from there.