When creating a library from multiple modules, I cannot find a nice way to do proper information hiding to the user of the library (external interface) while being able to access everything I need on the internal interface.
To be more specific, I have two modules (Files a.ml[i] and b.ml[i]). In A, I define some type t, thats internals I don't want to hide from the user (external interface).
module A : sig
type t
end
module A = struct
type t = float
end
In module B, I then want to use the secret type of A.t.
module B : sig
create_a : float -> A.t
end
module B = struct
create_a x = x
end
This of course does not compile, because the compilation unit of B does not know the type of A.t.
Solutions I know, but don't like:
Move the function create_a to module A
Copy the definition of A.t to B and cheat the type checker with some external cheat : `a -> `b = "%identity"
Is there some other way to know the type of A.t in B without leaking this information to the library's interface?
As always an extra layer of indirection can solve this problem. Define a module Lib that will specify an external interface, e.g.,
module Lib : sig
module A : sig
type t
(* public interface *)
end
module B : sig
type t
(* public interface *)
end = struct
module A = A
module B = B
end
If you don't want to repeat yourself and write module signatures twice, then you can define them once in a module sigs.ml:
module Sigs = struct
module type A = sig
type t
(* public interface *)
end
(* alternatively, you can move it into sigs_priv.ml *)
module type A_private = sig
include A
val create_a : float -> t
end
...
end
Finally, make sure that you're not installing interfaces (the .cmi files),
during your installation step, so that users can't bypass your abstraction. If your're using oasis, then it is simple: just make all your modules internal, except the module Lib, i.e., specify them with InternalModules field.
Related
In OCaml, to bring another module in scope you can use open. But what about code like this:
module A = struct
include B.C
module D = B.E
end
Does this create an entirely new module called A that has nothing to do with the modules created by B? Or are the types in B equivalent to this new structure and can a type in A.t can be used interchangeably with a type in B.C.t for example?
Especially, comparing to Rust I believe this is very different from writing something like
pub mod a {
pub use b::c::*;
pub use b::e as d;
}
Yes, module A = struct include B.C end creates an entirely new module and exports all definitions from B.C. All abstract types and data types that are imported from B.C are explicitly related to that module.
In other words, suppose you have
module Inner = struct
type imp = Foo
type t = int
end
so when we import Inner we can access the Inner definitions,
module A = struct
include Inner
let x : imp = Foo
let 1 : t = 1
end
and the Foo constructor in A belongs to the same type as the Foo constructor in the Inner module so that the following typechecks,
A.x = Inner.Foo
In other words, include is not a mere copy-paste, but something like this,
module A = struct
(* include Inner expands to *)
type imp = Inner.imp = Foo
type t = Inner.t = int
end
This operation of preserving type equalities is formally called strengthening and always applied when OCaml infers module type. In other words, the type system never forgets the type sharing constraints and the only way to remove them is to explicitly specify the module type that doesn't expose the sharing constraints (or use the module type of construct, see below).
For example, if we will define a module type
module type S = sig
type imp = Foo
type t = int
end
then
module A = struct
include (Inner : S)
end
will generate a new type foo, so A.Foo = Inner.Foo will no longer type check. The same could be achieved with the module type of construct that explicitly disables module type strengthening,
module A = struct
include (Inner : module type of Inner)
end
will again produce A.Foo that is distinct from Inner.Foo. Note that type t will be still compatible in all implementation as it is a manifest type and A.t is equal to Inner.t not via a sharing constraint but since both are equal to int.
Now, you might probably have the question, what is the difference between,
module A = Inner
and
module A = struct include Inner end
The answer is simple. Semantically they are equivalent. Moreover, the former is not a module alias as you might think. Both are module definitions. And both will define a new module A with exactly the same module type.
A module alias is a feature that exists on the (module) type level, i.e., in the signatures, e.g.,
module Main : sig
module A = Inner (* now this is the module alias *)
end = struct
module A = Inner
end
So what the module alias is saying, on the module level, is that A is not only has the same type as Inner but it is exactly the Inner module. Which opens to the compiler and toolchain a few opportunities. For example, the compiler may eschew module copying as well as enable module packing.
But all this has nothing to do with the observed semantics and especially with the typing. If we will forget about the explicit equality (that is again used mostly for more optimal module packing, e.g., in dune) then the following definition of the module A
module Main = struct
module A = Inner
end
is exactly the same as the above that was using the module aliasing. Anything that was typed with the previous definition will be typed with the new definition (modulo module type aliases). It is as strong. And the following is as strong,
module Main = struct
module A = struct include Inner end
end
and even the following,
module Main : sig
module A : sig
type imp = Impl.imp = Foo
type t = Impl.t = int
end
end = struct
module A = Impl
end
With the Signature/Functor pattern, I refer to the style of Map.S / Map.Make in the OCaml standard library. This pattern is highly successful when you want to parameterize a large piece of code over some type without making it fully polymorphic. Basically, you introduce a parameterized module by providing a signature (usually called S) and a constructor (Make).
However, when you take a closer look, there is a lot of redundancy in the declaration:
First, both the signature and the functor have to be announced in the .mli file
Second, the signature has to be repeated completely in the .ml file (is there actually any legal way to differ from the .mli file here?)
Finally, the functor itself has to repeat all definitions again to actually implement the module type
Summa summarum, I get 3 definition sites for non-abstract types (e.g. when I want to allow pattern matching). This is completely ridiculous, and thus I assume there is some way around. So my question is two-fold:
Is there a way to repeat a module type from an .mli file in an .ml file, without having to write it manually? E.g. something like ppx_import for module signatures?
Is there a way to include a module type in a module inside an .ml file? E.g. when the module type has only one abstract type definition, define that type and just copy the non-abstract ones?
You can already use ppx_import for module signatures. You can even use it in a .ml to query the corresponding .mli.
If a module is composed only of module signatures, you can define the .mli alone, without any .ml. This way you can define a module, let's say Foo_sigs, containing the signature and use it everywhere else.
Repeating type and module type definitions can be avoided to move them to external .ml file. Let's see the following example:
module M : sig
(* m.mli *)
module type S = sig
type t
val x : t
end
module type Result = sig
type t
val xs : t list
end
module Make(A : S) : Result with type t = A.t
end = struct
(* m.ml *)
module type S = sig
type t
val x : t
end
module type Result = sig
type t
val xs : t list
end
module Make(A : S) = struct
type t = A.t
let xs = [A.x;A.x]
end
end
Instead of writing two files m.mli and m.ml, I used a module M with an explicit signature: this is equivalent to have the two files and you can try it on OCaml toplevel by copy-and-paste.
In M, things are duped in sig .. end and struct .. end. This is cumbersome if module types become bigger.
You can share these dupes by moving them to another .ml file. For example, like the following n_intf.ml:
module N_intf = struct
(* n_intf.ml *)
module type S = sig
type t
val x : t
end
module type Result = sig
type t
val xs : t list
end
end
module N : sig
(* n.mli *)
open N_intf
module Make(A : S) : Result with type t = A.t
end = struct
(* n.ml *)
open N_intf
module Make(A : S) = struct
type t = A.t
let xs = [A.x;A.x]
end
end
You can also use *_intf.mli instead of *_intf.ml, but I recommend using *_intf.ml, since:
Module packing does not take mli only modules into account therefore you have to copy *_intf.cmi at installation.
Code generation from type definitions such as ppx_deriving needs things defined in .ml. In this example, it is no the case since there is no type definition.
In that specific case, you can just skip the .mli part:
Your abstraction is specified by the .ml
Reading it makes it quite clear (as people know the pattern from the stdlib)
Everything that you'd put in the .mli is already in the .ml
If you work in a group that's requiring you to actually give a mli, just generate it automatically by using the ocamlc -i trick.
ocamlc -i m.ml >m.mli # automatically generate mli from ml
I know it doesn't exactly answer your question, but hey, it solves your problem.
I know that always putting a mli is considered to be best practice, but it's not mandatory, and that may be for some very good reasons.
As for your second question, I'm not sure I understood it well but I think this answers it:
module type ToCopy = sig type t val f : t -> unit end
module type Copy1 = sig include ToCopy with type t = int end
module type Copy2 = ToCopy with type t = int;;
Adding to camlspoter's answer and since the question mentions pattern matching, maybe you want to "re-export" the signatures and types with constructors declared in N_intf so they are accessible through N instead. In that case, you can replace the open's with include and module type of, i.e.:
module N_intf = struct
type t = One | Two
(* n_intf.ml *)
module type S = sig
type t
val x : t
end
module type Result = sig
type t
val xs : t list
end
end
module N : sig
(* n.mli *)
include module type of N_intf
module Make(A : S) : Result with type t = A.t
end = struct
(* n.ml *)
include N_intf
module Make(A : S) = struct
type t = A.t
let xs = [A.x;A.x]
end
end
Then you'll get the following signatures:
module N_intf :
sig
type t = One | Two
module type S = sig type t val x : t end
module type Result = sig type t val xs : t list end
end
module N :
sig
type t = One | Two
module type S = sig type t val x : t end
module type Result = sig type t val xs : t list end
module Make : functor (A : S) -> sig type t = A.t val xs : t list end
end
Now the constructors One and Two can be qualified by N instead of N_intf, so you can ignore N_intf in the rest of the program.
I am reading through OCaml lead designer's 1994 paper on modules, types, and separate compilation. (kindly pointed to me by Norman Ramsey in another question ). I understand that the paper discusses the origins of OCaml's present module type / signature system. It it, the author proposes opaque interpretation of type declarations in signatures (to allow separate compilation) together with manifest type declarations (for expressiveness). Attempting to put together some examples of my own to demonstrate the kind of problems the OCaml module signature notation is trying to tackle I wrote the following code in two files:
In file ordering.ml (or .mli — I've tried both) (file A):
module type ORDERING = sig
type t
val isLess : t -> t -> bool
end
and in file useOrdering.ml (file B):
open Ordering
module StringOrdering : ORDERING
let main () =
Printf.printf "%b" StringOrdering.isLess "a" "b"
main ()
The idea being to expect the compiler to complain (when compiling the second file) that not enough type information is available on module StringOrdering to typecheck the StringOrdering.isLess application (and thus motivate the need for the with type syntax).
However, although file A compiles as expected, file B causes the 3.11.2 ocamlc to complain for a syntax error. I understood that signatures were meant to allow someone to write code based on the module signature, without access to the implementation (the module structure).
I confess that I am not sure about the syntax: module A : B which I encountered in this rather old paper on separate compilation but it makes me wonder whether such or similar syntax exists (without involving functors) to allow someone to write code based only on the module type, with the actual module structure provided at linking time, similar to how one can use *.h and *.c files in C/C++. Without such an ability it would seem to be that module types / signatures are basically for sealing / hiding the internals of modules or more explicit type checking / annotations but not for separate / independent compilation.
Actually, looking at the OCaml manual section on modules and separate compilation it seems that my analogy with C compilation units is broken because the OCaml manual defines the OCaml compilation unit to be the A.ml and A.mli duo, whereas in C/C++ the .h files are pasted to the compilation unit of any importing .c file.
The right way to do such a thing is to do the following:
In ordering.mli write:
(* This define the signature *)
module type ORDERING = sig
type t
val isLess : t -> t -> bool
end
(* This define a module having ORDERING as signature *)
module StringOrdering : ORDERING
Compile the file: ocamlc -c ordering.mli
In another file, refer to the compiled signature:
open Ordering
let main () =
Printf.printf "%b" (StringOrdering.isLess "a" "b")
let () = main ()
When you compile the file, you get the expected type error (ie. string is not compatible with Ordering.StringOrdering.t). If you want to remove the type error, you should add the with type t = string constraint to the definition of StringOrdering in ordering.mli.
So answer to you second question: yes, in bytecode mode the compiler just needs to know about the interfaces your are depending on, and you can choose which implementation to use at link time. By default, that's not true for native code compilation (because of inter-module optimizations) but you can disable it.
You are probably just confused by the relation between explicit module and signature definitions, and the implicit definition of modules through .ml/.mli files.
Basically, if you have a file a.ml and use it inside some other file, then it is as if you had written
module A =
struct
(* content of file a.ml *)
end
If you also have a.mli, then it is as if you had written
module A :
sig
(* content of file a.mli *)
end =
struct
(* content of file a.ml *)
end
Note that this only defines a module named A, not a module type. A's signature cannot be given a name through this mechanism.
Another file using A can be compiled against a.mli alone, without providing a.ml at all. However, you want to make sure that all type information is made transparent where needed. For example, suppose you are to define a map over integers:
(* intMap.mli *)
type key = int
type 'a map
val empty : 'a map
val add : key -> 'a -> 'a map -> 'a map
val lookup : key -> 'a map -> 'a option
...
Here, key is made transparent, because any client code (of the module IntMap that this signature describes) needs to know what it is to be able to add something to the map. The map type itself, however, can (and should) be kept abstract, because a client shouldn't mess with its implementation details.
The relation to C header files is that those basically only allow transparent types. In Ocaml, you have the choice.
module StringOrdering : ORDERING is a module declaration. You can use this in a signature, to say that the signature contains a module field called StringOrdering and having the signature ORDERING. It doesn't make sense in a module.
You need to define a module somewhere that implements the operations you need. The module definition can be something like
module StringOrderingImplementation = struct
type t = string
let isLess x y = x <= y
end
If you want to hide the definition of the type t, you need to make a different module where the definition is abstract. The operation to make a new module out of an old one is called sealing, and is expressed through the : operator.
module StringOrderingAbstract = (StringOrdering : ORDERING)
Then StringOrderingImplementation.isLess "a" "b" is well-typed, whereas StringOrderingAbstract.isLess "a" "b" cannot be typed since StringOrderingAbstract.t is an abstract type, which is not compatible with string or any other preexisting type. In fact, it's impossible to build a value of type StringOrderingAbstract.t, since the module does not include any constructor.
When you have a compilation unit foo.ml, it is a module Foo, and the signature of this module is given by the interface file foo.mli. That is, the files foo.ml and foo.mli are equivalent to the module definition
module Foo = (struct (*…contents of foo.ml…*) end :
sig (*…contents of foo.mli…*) end)
When compiling a module that uses Foo, the compiler only looks at foo.mli (or rather the result of its compilation: foo.cmi), not at foo.ml¹. This is how interfaces and separate compilation fit together. C needs #include <foo.h> because it lacks any form of namespace; in OCaml, Foo.bar automatically refers to a bar defined in the compilation unit foo if there is no other module called Foo in scope.
¹ Actually, the native code compiler looks at the implementation of Foo to perform optimizations (inlining). The type checker never looks at anything but what is in the interface.
I have a decomposition where module A defines a structure type, and exports a field of this type which is defined as a value in module B:
a.ml:
type t = {
x : int
}
let b = B.a
b.ml:
open A (* to avoid fully qualifying fields of a *)
let a : t = {
x = 1;
}
Circular dependence is avoided, since B only depends on type declarations (not values) in A.
a.mli:
type t = {
x : int
}
val b : t
As far as I know, this should be kosher. But the compiler errors out with this:
File "a.ml", line 1, characters 0-1:
Error: The implementation a.ml does not match the interface a.cmi:
Values do not match: val b : A.t is not included in val b : t
This is all particularly obtuse, of course, because it is unclear which val b is interpreted as having type t and which has type A.t (and to which A--the interface definition or the module definition--this refers).
I'm assuming there is some arcane rule (along the lines of the "structure fields must be referenced by fully module-qualified name when the module is not opened" semantics which bite every OCaml neophyte at some point), but I am so far at a loss.
Modules in the microscope are more subtle than it appears
(If your eyes glaze over at some point, skip to the second section.)
Let's see what would happen if you put everything in the same file. This should be possible since separate computation units do not increase the power of the type system. (Note: use separate directories for this and for any test with files a.* and b.*, otherwise the compiler will see the compilation units A and B which may be confusing.)
module A = (struct
type t = { x : int }
let b = B.a
end : sig
type t = { x : int }
val b : t
end)
module B = (struct
let a : A.t = { A.x = 1 }
end : sig
val a : A.t
end)
Oh, well, this can't work. It's obvious that B is not defined here. We need to be more precise about the dependency chain: define the interface of A first, then the interface of B, then the implementations of B and A.
module type Asig = sig
type t = { x : int }
type u = int
val b : t
end
module B = (struct
let a : Asig.t = { Asig.x = 1 }
end : sig
val a : Asig.t
end)
module A = (struct
type t = { x : int }
let b = B.a
end : Asig)
Well, no.
File "d.ml", line 7, characters 12-18:
Error: Unbound type constructor Asig.t
You see, Asig is a signature. A signature is a specification of a module, and no more; there is no calculus of signatures in Ocaml. You cannot refer to fields of a signature. You can only refer to fields of a module. When you write A.t, this refers to the type field named t of the module A.
In Ocaml, it is fairly rare for this subtlety to arise. But you tried poking at a corner of the language, and this is what's lurking there.
So what's going on then when there are two compilation units? A closer model is to see A as a functor which takes a module B as an argument. The required signature for B is the one described in the interface file b.mli. Similarly, B is a function which takes a module A whose signature is given in a.mli as an argument. Oh, wait, it's a bit more involved: A appears in the signature of B, so the interface of B is really defining a functor that takes an A and produces a B, so to speak.
module type Asig = sig
type t = { x : int }
type u = int
val b : t
end
module type Bsig = functor(A : Asig) -> sig
val a : A.t
end
module B = (functor(A : Asig) -> (struct
let a : A.t = { A.x = 1 }
end) : Bsig)
module A = functor(B : Bsig) -> (struct
type t = { x : int }
let b = B.a
end : Asig)
And here, when defining A, we run into a problem: we don't have an A yet, to pass as an argument to B. (Barring recursive modules, of course, but here we're trying to see why we can't get by without them.)
Defining a generative type is a side effect
The fundamental sticking point is that type t = {x : int} is a generative type definition. If this fragment appears twice in a program, two different types are defined. (Ocaml takes steps and forbids you to define two types with the same name in the same module, except at the toplevel.)
In fact, as we've seen above, type t = {x : int} in a module implementation is a generative type definition. It means “define a new type, called d, which is a record type with the fields …”. That same syntax can appear in a module interface, but there it has a different meaning: there, it means “the module defines a type t which is a record type …”.
Since defining a generative type twice creates two distinct types, the particular generative type that is defined by A cannot be fully described by the specification of the module A (its signature). Hence any part of the program that uses this generative type is really using the implementation of A and not just its specification.
When you get down to it, defining a generative type it is a form of side effect. This side effect happens at compile time or at program initialization time (the distinction between these two only appears when you start looking at functors, which I shall not do here.) So it is important to keep track of when this side effect happens: it happens when the module A is defined (compiled or loaded).
So, to express this more concretely: the type definition type t = {x : int} in the module A is compiled into “let t be type #1729, a fresh type which is a record type with a field …”. (A fresh type means one that is different from any type that has ever been defined before.). The definition of B defines a to have the type #1729.
Since the module B depends on the module A, A must be loaded before B. But the implementation of A clearly uses the implementation of B. The two are mutually recursive. Ocaml's error message is a little confusing, but you are indeed outstepping the bounds of the language.
(and to which A--the interface definition or the module definition--this refers).
A refers to the whole module A. With the normal build procedure it would refer to the implementation in a.ml contrained by signature in a.mli. But if you are playing tricks moving cmi's around and such - you are on your own :)
As far as I know, this should be kosher.
I personally qualify this issue as circular dependency and would stay strongly against structuring the code in such a way. IMHO it causes more problems and head-scratching, than solving real issues. E.g. moving shared type definitions to type.ml and be done with it is what comes first to mind. What is your original problem that leads to such structuring?
module <name> =
struct
..
end;;
module type <name> =
struct (* should have been sig *)
..
end;;
The first declares a module and the second declares a module type (aka a signature). A module type contains type and val declarations, whereas a module can contain definitions (e.g., let bindings). You can use a signature to restrict the type of a module, much as you might for a function. For example,
module type T = sig
val f : int -> int
end
module M : T = struct
let f x = x + 1
let g x = 2 * x
end
Now, we have
# M.f 0 ;;
- : int = 1
# M.g 0 ;;
Error: Unbound value M.g
M.g is unbound because it's hidden by the signature T.
Another common way to use module types is as arguments to and return values of functors. For example, the Map.Make functor in the standard library takes a module with signature Map.OrderedType and creates a module with signature Map.S
P.S. Note that there's a mistake in the question. A module type is declared using
module type <name> = sig
...
end
A structure (written struct … end) is a bunch of definitions. Any object in the language can be defined in a module: core values (let x = 2 + 2), types (type t = int), modules (module Empty = struct end), signatures (module type EMPTY = sig end), etc. Modules are a generalization of structures: a structure is a module, and so is a functor (think of it as a function that takes a module as argument and returns a new module). Modules are like core values, but live one level above: a module can contain anything, whereas a core value can only contain other core values¹.
A signature (written sig … end) is a bunch of specifications (some languages use the term declaration). Any object in the language can be specified in a module: core values (val x : int), types (type t = int), modules (module Empty : sig end), signatures (module type EMPTY = sig end), etc. Module types generalize signatures: a module type specifies a module, and a module type that happens to specify a structure is called a signature. Module types are to modules what ordinary types are to core values.
Compilation units (.ml files) are structures. Interfaces (.mli files) are signatures.
So module Foo = struct … end defines a module called Foo, which happens to be a structure. This is analogous to let foo = (1, "a") which defines a value called foo which happens to be a pair. And module type FOO = sig … end (note: sig, not struct) defines a module type called FOO, which happens to be a signature. This is analogous to type foo = int * string which defines a type called foo which happens to be a product type.
¹
This is in fact no longer true since OCaml 3.12 introduced first-class modules, but it's close enough for an introductory presentation.
module type describe a module. It is the same as the difference between .ml and .mli