I'm particularly interested to understand the K prelude (how it is structured, why its content is like that, how "kompile" calculates dependencies, etc).
The main question is: what is the criterion for a hooked symbol from the K prelude to be copied into the generated Kore file?
Here some examples of potential problems:
The symbol andBool is copied with its associated rewrite rules, which does not seem to be the case for the symbol in_keys, which is simply copied without its rewrite rules.
Other symbols seem to be useless (for the IMP semantic) but exist, with or without its rewrite rules, in the generated Kore file, such as countAllOccurrences, findChar, signExtendBitRangeInt or Float2String.
It seems that SortId is generated by the line syntax Id [token]. However, the lines "syntax Bool ::= "true" [token] and syntax Bool ::= "false" [token] do not generate true and false symbols.
(Moreover, is it a choice that true and false are values and not constructors?)
The sort named SortId is not generated for the following example, whereas some generated hooked symbols depend on this sort. This problem does not exist with the IMP semantic.
module MAX-OW-SYNTAX
imports INT
imports BOOL
syntax Exp ::= Int | "(" Exp ")" [bracket]
| "max" Exp Exp
endmodule
module MAX-OW
imports MAX-OW-SYNTAX
syntax KResult ::= Int
rule max X Y => Y requires X <Int Y
rule max X _ => X [owise]
endmodule
Is it correct that the K prelude is implemented in each language of each backend, and that an implementation in the Kore language is available in the K prelude?
Do you have the necessary interface to implement for a new backend? (For instance, Bag is obsolete, but not Set, List and Map, but I don't know the list of set operators, map operators, etc. that the new backend must provide.)
Is there a reason why andThenBool and andBool have the same semantics once implemented in the Kore syntax (Booleans module)?
Where are the rewrite rules defined for ==Bool, used in the definition of =/=Bool (Booleans module)?
The best reference point for the K internals is the User Manual, along with the K source for the prelude. To respond to your specific questions as best as I can:
in_keys only has simplification rules that apply on symbolic backends. These will not apply on concrete backends, and so those backends use the hooked implementation MAP.in_keys. Some functions (such as andBool) can be implemented both in K and as an efficient backend hook. For example, on the K LLVM backend, andBool is implemented by code generation. If a backend didn't support that hook, the (relatively) inefficient K rewriting implementation would be used.
The Id sort is built in for convenience. It represents program identifiers.
You haven't imported DOMAINS in this example. Doing so will pull in the Id sort and related rewrites.
Very roughly, and largely for internal purposes. Do you have a hypothetical K backend in mind, or is there a way in which the LLVM / Haskell backends provided by K are inadequate for your specific use case?
andThenBool is required to short-circuit its arguments; andBool is permitted to short-circuit, but may evaluate both arguments strictly. An implementation that makes both perform short-circuiting is valid.
==Bool is implemented only in terms of a hook. In domains.md, you can see the hook(BOOL.eq) attribute that indicates how ==Bool is implemented.
Do let us know if you have further questions, or would like help implementing a specific semantics in K.
Related
In the SWI Prolog manual, I found the following remark:
For example, assume an application that can reason about multiple worlds. It is attractive to store the data of a particular world in a module, so we extract information from a world simply by invoking goals in this world.
This is actually a very good description of what I'm trying to achieve. However I ran into a problem. While I do want to model many different worlds, there are also things that I want to share across all of them. So my idea is to have an allworlds module for things that are true in every world, and one module for every world that I want to reason about, and the latter imports from the former. So I'd do something like this in the REPL:
allworlds:asserta(grandparent(X, Z) :- (parent(X, Y), parent(Y, Z))).
allworlds:dynamic(parent/2).
add_import_module(greece, allworlds, start).
greece:asserta(parent(kronos, zeus)).
greece:asserta(parent(zeus, ares)).
Now I'd like to query greece:grandparent(kronos, X) and get X = ares, but all I get is false. When allworlds:grandparent calls parent, it doesn't call greece:parent like I want it to, but allworlds:parent. My research seems to indicate that I need to make the grandparent predicate module-transparent. But calling allworlds:module_transparent(grandparent/2). didn't fix the issue, and it's also deprecated. This is where I'm stuck. How can I get this working? Is meta_predicate/1 part of the solution? Unfortunately I can't make heads or tails of its documentation.
Prolog modules don't provide a good solution for the "many worlds" design pattern. Notably, making the predicates meta-predicates (or module transparent or multifile) would be a problematic hack. But this pattern is trivial with Logtalk, which is a language extends Prolog and can use most Prolog systems as a backend compiler. A minimal (but not unique) solution for your problem is:
:- object(allworlds).
:- public(grandparent/2).
grandparent(X, Z) :-
::parent(X, Y),
::parent(Y, Z).
:- public(parent/2).
:- end_object.
:- object(greece,
extends(allworlds)).
parent(kronos, zeus).
parent(zeus, ares).
:- end_object.
Here, we use inheritance (the individual worlds inherit the common knowledge) and messages to self (the ::/1 control construct) when common predicates need to access world specific predicate definitions (self is the object/world that received the message - grandparent/2 in the example).
Assuming the code is saved in a worlds.lgt file and that you're using SWI-Prolog as the backend:
$ swilgt
...
?- {worlds}.
% [ /Users/pmoura/worlds.lgt loaded ]
% (0 warnings)
true.
?- greece::grandparent(kronos, X).
X = ares.
P.S. If running on windows, use the "Logtalk - SWI-Prolog" shortcut from the Start Menu after installing Logtalk.
I ultimately solved this by passing the module around explicitly and invoking predicates in it with the : operator. It reminds me a bit of doing OOP in C, where you do things like obj->vtable->method(obj, params) (note how obj is mentioned twice, just like the M in my code below).
Similar to the Logtalk solution, I need to explicitly call into the imported module when I want to consider its clauses. As an example, I've added the fact that a father is also a parent to the allworlds module.
allworlds:assertz(grandparent(M, X, Z) :- (M:parent(M, X, Y), M:parent(M, Y, Z))).
allworlds:assertz(parent(M, X, Y) :- M:father(M, X, Y)).
add_import_module(greece, allworlds, start).
greece:assertz(parent(_, kronos, zeus)).
% need to call into allworlds explicitly
greece:assertz(parent(M, X, Y) :- allworlds:parent(M, X, Y)).
greece:assertz(father(_, zeus, ares)).
After making these assertions, I can call greece:grandparent(greece, kronos, X). and get the expected result X = ares.
The use of context is briefly mentioned in the K tutorial as a way to customize the order evaluation. But I'm also seeing other context statements that contain rewrite arrows in them, like this one in the untyped simple language.
context ++(HOLE => lvalue(HOLE))
rule <k> ++loc(L) => I +Int 1 ...</k>
<store>... L |-> (I => I +Int 1) ...</store> [increment]
Could someone explain how exactly context work in K? In particular, I'm interested in:
Is there a more general usage of context in K than just stating the order of evaluation?
How does the order in which context statements are declared affect the semantics?
Thank you!
More detailed information about context declarations in K can be found in K's documentation here. In particular, contexts with rewrite arrows mean that heating and cooling will wrap the term to be heated or cooled in a particular symbol. In your example, that symbol is lvalue.
To answer your questions specifically:
Context declarations, like strictness attributes, are primarily used in order to specify the evaluation strategy. While in theory they can be used for other things, in practice this rarely happens. That said, evaluation strategies can be complex, which is part of why K has so many different features relating to evaluation strategy. In the example you mentioned, we use rewrites in a context declaration in order to provide a separate set of rules for evaluating lvalues (ie, to avoid actually evaluating all the way to a value, and only evaluate to a location).
K's sentences are unordered. Within a single module, you can reorder any of its sentences (except import statements, which must appear first) and there will not be an effect on the intended semantics (although backends may result in slightly different behavior for concrete execution if your semantics is nondeterministic). This includes context declarations.
I am writing a simple compiler as a school work. I am looking for an automated approach to generate both positive and negative testing data to test my compiler, given the formal grammar and other specification. The language I am dealing with is of mediate size with 38 or so non-terminals. For the sake of illustration, here is a snapshot of the grammar:
program: const_decl* declaration* ENDMARKER
# statement
stmt: flow_stmt | '{' stmt* '}' | NAME [stmt_trailer] ';' | ';'
stmt_trailer: arglist | ['[' expr ']'] '=' expr
flow_stmt: if_stmt | for_stmt | while_stmt | read_stmt ';' | write_stmt ';' | return_stmt ';'
return_stmt: 'return' ['(' expr ')']
if_stmt: 'if' '(' condition ')' stmt ['else' stmt]
condition: expr ('<'|'<='|'>'|'>='|'!='|'==') expr | expr
for_stmt: ('for' '(' NAME '=' expr ';' condition ';'
NAME '=' NAME ('+'|'-') NUMBER ')' stmt)
Is there any tools to generate input file with the help of the grammar? The hand-written tests are too tedious or too weak to discover problems. An example of this language here:
void main() {
int N;
int temp;
int i, j;
int array_size;
reset_heap;
scanf(N);
for (i = 0; i < N; i = i + 1) {
scanf(array_size);
if (array_size > max_heap_size) {
printf("array_size exceeds max_heap_size");
} else {
for (j = 0; j < array_size; j = j + 1) {
scanf(temp);
heap[j] = temp;
}
heap_sort(array_size);
print_heap(array_size);
}
}
}
Generating controllable testing data automatically can save the days. Given the simplicity of the language, there must be some way to effectively do this. Any pointer and insight is greatly appreciated.
Any pointer and insight is greatly appreciated.
This should have the subtopic of How to avoid combinatorial explosion when generating test data.
While I would not be surprised if there are tools to do this having had the same need to generate test data for grammars I have created a few one off applications.
One of the best series of articles I have found on this is by Eric Lippert, Every Binary Tree There Is, think BNF converted to binary operators then converted to AST when you read tree. However he uses Catalan (every branch has two leaves) and when I wrote my app I preferred Motzikin (a branch can have one or two leaves).
Also he did his in C# with LINQ and I did mine in Prolog using DCG.
Generating the data based on the BNF or DCG is not hard, the real trick is to limit the area of expansion and the size of the expansion and to inject bad data.
By area of expansion lets say you want to test nested if statements three levels deep, but have to have valid code that compiles. Obviously you need the boilerplate code to make it compile then you start changing the deeply nested if by adding or removing the else clause. So you need to put in constraints so that the boilerplate code is constant and the testing part is variable.
By size of expansion lets say that you want to test conditional expressions. You can easily calculate that if you have many operators and you want to test them all in combinations you soon run into combinatorial explosion. The trick is to ensure you test deep enough and with enough breadth but not every combination. Again the judicial use of constraints helps.
So the point of all of this is that you start with a tool that takes in the BNF and generates valid code. Then you modify the BNF to add constraints and modify the generator to understand the constraints to generate the code examples.
Then you modify the BNF for invalid data and likewise the generator to understand those rules.
After that is working you can then start layering on levels of automation.
If you do go this route and decide that you will have to learn Prolog, take a look at Mercury first. I have not done this with Mercury, but if I do it again Mercury is high on the list.
While my actual code is not public, this and this is the closest to it that is public.
Along the way I had some fun with it in Code Golf.
When generating terminals such as reserved words or values for types, you can use predefined list with both valid and invalid data, e.g. for if if the language is case sensitive I would include in the list if,If,IF,iF, etc. For value types such as unsigned byte I would include -1,0,255 and 256.
When I was testing basic binary math expressions with +, -, * and ^ I generated all the test for with three basic numbers -2,-1,0,1, and 2. I thought it would be useless since I already had hundreds of test cases, but since it only took a few minutes to generate all of the test cases and several hours to run it, to my surprise it found a pattern I did not cover. The point here is that contrary what most people say about having to many test cases, remember that it is only time on a computer by changing a few constraints so do the large number of test.
In HOTT and also in COQ one cannot prove UIP, i.e.
\Prod_{p:a=a} p = refl a
But one can prove:
\Prod_{p:a=a} (a,p) = (a, refl a)
Why is this defined as it is?
Is it, because one wants to have a nice homotopy interpretation?
Or is there some natural, deeper reason for this definition?
Today we know of a good reason for rejecting UIP: it is incompatible with the principle of univalence from homotopy type theory, which roughly says that isomorphic types can be identified. However, as far as I am aware, the reason that Coq's equality does not validate UIP is mostly a historical accident inherited from one of its ancestors: Martin-Löf's intensional type theory, which predates HoTT by many years.
The behavior of equality in ITT was originally motivated by the desire to keep type checking decidable. This is possible in ITT because it requires us to explicitly mark every rewriting step in a proof. (Formally, these rewriting steps correspond to the use of the equality eliminator eq_rect in Coq.) By contrast, Martin-Löf designed another system called extensional type theory where rewriting is implicit: whenever two terms a and b are equal, in the sense that we can prove that a = b, they can be used interchangeably. This relies on an equality reflection rule which says that propositionally equal elements are also definitionally equal. Unfortunately, there is a price to pay for this convenience: type checking becomes undecidable. Roughly speaking, the type-checking algorithm relies crucially on the explicit rewriting steps of ITT to guide its computation, whereas these hints are absent in ETT.
We can prove UIP easily in ETT because of the equality reflection rule; however, it was unknown for a long time whether UIP was provable in ITT. We had to wait until the 90's for the work of Hofmann and Streicher, which showed that UIP cannot be proved in ITT by constructing a model where UIP is not valid. (Check also these slides by Hofmann, which explain the issue from a historic perspective.)
Edit
This doesn' t mean that UIP is incompatible with decidable type checking: it was shown later that it can be derived in other decidable variants of Martin-Löf type theory (such as Agda), and it can be safely added as an axiom in a system like Coq.
Intuitively, I tend to think of a = a as pi_1(A,a), i.e. the class of paths from a to itself modulo homotopy equivalence; whereas I think of { x:A | a = x } as the universal covering space of A, i.e. paths from a to some other point of A modulo homotopy equivalence. So, while pi_1(A,a) is often non-trivial, we do have that the universal covering space of A is contractible.
I would like to have a clear example of precedence and one of associativity in yacc, but I find myself yet in having troubles separating these two concepts.
Perhaps this is due to the fact that I'm associating these two concepts to math and mathematical operation.. These are two old examples I built:
Associativity (*) is used to specify the kind of association to be applied (left,right, non assoc....)
In fact
%left '+' '*'
instruct that plus and multiplication are left associative. So far, so good. (not exactly but it serve the purpose of the example)
Precedence (**) is used to give precedence to one operator over another.
%left '+'
%left '*'
the multiplication has higher precedence than plus operation.
So we got the wanted parsing action for E+E*E
E+(E*E) in case of (**)
(E+E)*E in case of only (*) --> this is clearly wrong - but it's fine for the example
So question is, can I separate clearly associativity from precedence without using the concept of associativity?
Even non-associatity implies associativity knowledge… so.. how, if possible, can I talk separately about them?
No. In a parser definition, associativity is just a small detail within the precedence algorithm.
To understand that, it's important to understand what precedence actually means, in parsing terms.
A left-to-right shift-reduce parser has a stack and an input stream. Initially, the stack is empty, and the input stream contains the input to be parsed. The SR parser repeatedly does one of the following two actions until the stack consists only of the start symbol and the input stream is empty (in which case the parse has succeeded), or neither action is possible (in which case the parse has failed):
reduce the production whose right-hand side is on the top of the stack by popping the right-hand side off of the stack and pushing the left-hand side non-terminal;
shift one input symbol from the input onto the stack.
It's an important feature of this framework that reductions can only occur when the production's right-hand side is on the top of the stack.
The shift action is always possible unless the input stream is exhausted, but a reduce action can only be taken if the top of the stack precisely matches the right-hand side of some production.
Different ways of building SR parsers will involve different mechanisms for deciding which action to take in any given stack configuration. One such mechanism is the precedence algorithm. Some very simple languages can be SR parsed only with the precedence algorithm. In other cases, it can be used as an auxiliary decision algorithm in order to resolve ambiguous grammar specifications; this is the use case for precedence in yacc-derived parser generators.
For precedence to work, it is necessary that at most one reduction action be possible in any stack configuration, which means that there cannot be two productions with the same right-hand side. [Note 1]
Given that there is at most one possible reduction action and at most one possible shift action (since the next input symbol, if any, is given), the only issue is deciding whether to shift or reduce. The precedence algorithm involves a precedence function PREC(A→α, a) ⇒ { SHIFT, REDUCE }, whose arguments are a production A→α and a terminal symbol a, which are mapped onto either SHIFT or REDUCE.
Although the precedence relationship is usually written as though it were a comparison, it is not a normal comparison operator because the two arguments are from different domains. It always involves a production and a terminal.
In simple cases, however, it is possible to implement PREC using numeric comparisons. To do that, we define two functions which map productions and terminals, respectively, onto integers: f(A→α) and g(a). We use those to compute PREC:
PREC(A→α, a) ≡
REDUCE if f(A→α) > g(a)
SHIFT if f(A→α) < g(a)
[Note 2]
In any event, the precedence algorithm for a given stack configuration is:
Identify the production P (=A→α) of the possible reduce action, if any.
If only a shift or only a reduce is possible, do that. Otherwise, if both a reduce and a shift are possible, compute PREC(P, input) and reduce using P if the result is REDUCE; otherwise, shift input.
Now that might seem confusing, since most descriptions of precedence relations describe them as though they compared terminals, rather than a production with a terminal. That's because it is normal to "name" each production using the last terminal in the production. Usually, that is unambiguous, because of the restriction on production right-hand sides: since two right-hand side must differ, it is likely that all production right-hand sides have different terminal symbols. [Note 3]
Although that short-hand allows us to say, for example, that "* has higher precedence than +" instead of the somewhat more cumbersome "the production E→E*E has precedence over the terminal +", it is important to remember that the latter statement is what we really mean.
Precedence also applies to single operators. With most operators, we prefer to group from left to right, so that E-E-E should be parsed as though it had been written (E-E)-E. However, some operators like exponentiation group to the right, meaning that E**E**E should be parsed as E**(E**E). This is simple to define using the PREC function; for a left-grouping operator ⊕, we'll have:
PREC(E→E⊕E, ⊕) ≡ REDUCE
while a right-grouping operator ⊗ would have
PREC(E→E⊗E, ⊗) ≡ SHIFT
That's clear when we use the actual arguments to PREC, but it becomes confusing when we use the shorthand notation, which leaves us trying to say that ⊕ has higher precedence than ⊕ while ⊗ has lower precedence than ⊗. To avoid the ambiguity and still let us get away with the shorthand, we describe ⊕ as "left-associative" (%left) and ⊗ as "right-associative" (%right). But the implementation is simply an application of the normal precedence algorithm.
As an example, consider the simple expression language:
E → E + E
E → E * E
E → E ** E
E → id
Here we expect * to bind more tightly than + with ** binding tightest; the first two group to the left while exponentiation groups to the right. To achieve that, we can assign f and g functions as follows:
Production f(Production) Terminal g(Terminal)
E → E + E 2 + 1
E → E * E 4 * 3
E → E ** E 5 ** 6
E → id 8 id 7
Yacc-generated grammars don't use precedence to decide when to reduce the E→id production, but the above will work since the grammar can be parsed completely using only the precedence algorithm.
Parentheses can easily be added; I'll leave that as an exercise.
Notes
There might be some other mechanism to decide between reduction actions, so the restriction is only absolute for a parser which only uses precedence. There might also be some other mechanism to restrict possible shift actions. For example, for a shift to be feasible, the tokens on the top of the stack need to eventually be reduced, which means that some suffix of the stack must be a prefix of the right-hand side of some production. Similarly, a reduction is only feasible if, post-reduce, some suffix of the stack is the prefix of the right-hand side of some production.
You'll see formulations using < and ≥ (or ≤ and >), but to avoid confusion, I'm assuming that the ranges of f and g are different sets of integers. Since the functions are arbitrary, this does not restrict generality.
That's not always the case. For example, languages which allow - to be either a unary or a binary operator will have productions with right-hand sides - E and E - E. Yacc-derived parser generators use the %prec TERMINAL declaration to associate a production with a terminal other than the default.
This is all very confused.
Associativity ... Is used to give precedence to one operator over another
No. Absolutely not. Associativity is used to determine which order two adjacent instances of the same operator are evaluated in. (E+E)+E or E+(E+E). All arithmetic operators except exponentiation are left-associative in mathematics.
%left '+' '*'
This says that + and * are both left-associative and have the same precedence, because they are both on the same line. And it is therefore wrong.
can I separate clearly associativity from precedence without using the concept of associativity
I'm sorry but this is just meaningless.