Please have a look at the following parser I implemented in Kotlin using the parser combinator library better-parse. It parses—should parse, rather—simple arithmetic expressions:
object CalcGrammar: Grammar<Int>() {
val num by regexToken("""\d+""")
val pls by literalToken("+")
val min by literalToken("-")
val mul by literalToken("*")
val div by literalToken("/")
val lpr by literalToken("(")
val rpr by literalToken(")")
val wsp by regexToken("\\s+", ignore = true)
// expr ::= term + expr | term - expr | term
// term ::= fact * term | fact / term | fact
// fact ::= (expr) | -fact | int
val fact: Parser<Int> by
skip(lpr) and parser(::expr) and skip(rpr) or
(skip(min) and parser(::fact) map { -it }) or
(num map { it.text.toInt() })
val term by
leftAssociative(fact, mul) { a, _, b -> a * b } or
leftAssociative(fact, div) { a, _, b -> a / b } or
fact
val expr by
leftAssociative(term, pls) { a, _, b -> a + b } or
leftAssociative(term, min) { a, _, b -> a - b } or
term
override val rootParser by expr
}
However, when I parse -2 + 4 - 5 + 6, I get this ParseException:
Could not parse input: UnparsedRemainder(startsWith=min#8 for "-" at 7 (1:8))
At first, I thought the issue was that I have not codified the self-recursion in the expr productions, i.e., the code does not accurately represent the grammar:
// Reference to `expr` is missing on the right-hand side...
val expr = ... leftAssociative(term, min) ...
// ...although the corresponding production defines it.
// expr ::= ... term - expr ...
But then I noticed that the official example for an arithmetic parser provided as part of the library's documentation—which turns out to be almost identical afaict—also omits this.
What did I do wrong if not this? And how can I make it work?
I am not entirely familiar with this library, but it seems your translation from grammar to code is too literal for the library's syntax and the library actually implicitly handles much of what you are explicitly writing, and it seems that as a part of this "ease of use" it breaks what is seemingly correct code.
For starters, you will find that your code behaves the exact same way whether or not you chain or term on to the end of expr, and or fact on to the end of term.
Based on this I was able to come to the conclusion that these ors are not working as expected when chaining together the leftAssociatives, in fact you would run in to the same problem had you attempted to parse a division. I believe it is for this reason that the example you provided a link to combines addition and subtraction (as with multiplication and division) into a single, more dynamic, leftAssociative call. And if you copy that same work to your own code, it runs flawlessly:
object CalcGrammar: Grammar<Int>() {
val num by regexToken("""\d+""")
val pls by literalToken("+")
val min by literalToken("-")
val mul by literalToken("*")
val div by literalToken("/")
val lpr by literalToken("(")
val rpr by literalToken(")")
val wsp by regexToken("\\s+", ignore = true)
// expr ::= term + expr | term - expr | term
// term ::= fact * term | fact / term | fact
// fact ::= (expr) | -fact | int
val fact: Parser<Int> by
skip(lpr) and parser(::expr) and skip(rpr) or
(skip(min) and parser(::fact) map { -it }) or
(num map { it.text.toInt() })
private val term by
leftAssociative(fact, div or mul use { type }) { a, op, b ->
if (op == div) a / b else a * b
}
private val expr by
leftAssociative(term, pls or min use { type }) { a, op, b ->
if (op == pls) a + b else a - b
}
override val rootParser by expr
}
Related
I am playing with kotlin language and I tried the following code:
data class D( val n: Int, val s: String )
val lst = listOf( D(1,"one"), D(2, "two" ) )
val res = lst.reduce { acc:String, d:D -> acc + ", " + d.toString() }
The last statement causes the following errors:
Expected parameter of type String
Expected parameter of type String
Type mismatch: inferred type is D but String was expected
while this version of the last statement works:
val res = lst.map { e -> e.toString() }.reduce { acc, el -> acc + ", " + el }
I do not understand why the first version does not work. The formal definition of the reduce function, found here, is the following:
inline fun <S, T : S> Iterable<T>.reduce(
operation: (acc: S, T) -> S
): S
But this seems in contrast with the following sentence, on the same page:
Accumulates value starting with the first element and applying
operation from left to right to current accumulator value and each
element.
That is, as explained here:
The difference between the two functions is that fold() takes an
initial value and uses it as the accumulated value on the first step,
whereas the first step of reduce() uses the first and the second
elements as operation arguments on the first step.
But, to be able to apply the operation on first and second element, and so on, it seems to me tha the operation shall have both arguments of the base type of the Iterable.
So, what am I missing ?
Reduce is not the right tool here. The best function in this case is joinToString:
listOf(D(1, "one"), D(2, "two"))
.joinToString(", ")
.let { println(it) }
This prints:
D(n=1, s=one), D(n=2, s=two)
reduce is not designed for converting types, it's designed for reducing a collection of elements to a single element of the same type. You don't want to reduce to a single D, you want a string. You could try implementing it with fold, which is like reduce but takes an initial element you want to fold into:
listOf(D(1, "one"), D(2, "two"))
.fold("") { acc, d -> "$acc, $d" }
.let { println(it) }
However, this will add an extra comma:
, D(n=1, s=one), D(n=2, s=two)
Which is exactly why joinToString exists.
You can see the definition to understand why its not working
To make it work, you can simply create an extension function:
fun List<D>.reduce(operation: (acc: String, D) -> String): String {
if (isEmpty())
throw UnsupportedOperationException("Empty list can't be reduced.")
var accumulator = this[0].toString()
for (index in 1..lastIndex) {
accumulator = operation(accumulator, this[index])
}
return accumulator
}
you can use it as:
val res = lst.reduce { acc:String, d:D -> acc + ", " + d.toString() }
or simply:
val res = lst.reduce { acc, d -> "$acc, $d" }
You can modify the function to be more generic if you want to.
TL;DR
Your code acc:String is already a false statement inside this line:
val res = lst.reduce { acc:String, d:D -> acc + ", " + d.toString() }
Because acc can only be D, never a String! Reduce returns the same type as the Iterable it is performed on and lst is Iterable<D>.
Explanation
You already looked up the definition of reduce
inline fun <S, T : S> Iterable<T>.reduce(
operation: (acc: S, T) -> S
): S
so lets try to put your code inside:
lst is of type List<D>
since List extends Iterable, we can write lst : Iterable<D>
reduce will look like this now:
inline fun <D, T : D> Iterable<T>.reduce(
operation: (acc: D, T) -> D //String is literally impossible here, because D is not a String
): S
and written out:
lst<D>.reduce { acc:D, d:D -> }
Is there any way to convert a char, lets say with a value of '+', into the operator +? Something like this:
println(1 charOperator 1);
output:
2
You can use something like this:
fun operatorFromChar(charOperator: Char):(Int, Int)->Int
{
return when(charOperator)
{
'+'->{a,b->a+b}
'-'->{a,b->a-b}
'/'->{a,b->a/b}
'*'->{a,b->a*b}
else -> throw Exception("That's not a supported operator")
}
}
and later call:
println(operatorFromChar('+').invoke(1,1))
Operators are, at the end of the way, functions. If you return a function with the operator's job, you can invoke it as it was the operator itself, but it will never be as "pretty" as calling the operator directly.
This isn't really possible. Maybe you should add your current solution and there's another way to help you out.
Here's a sneaky solution for calculating expressions with + and - only:
val exp = "10+44-12+3"
val result = exp.replace("-", "+-").split("+").sumBy { it.toInt() }
You can do something like
infix fun Int.`_`(that: Int) = this + that
where the backtick is unnecessary to this character but maybe necessary for other character. Then you can try:
println(2 _ 3) // output: 5
Update according to the comment:
I mean something like
val expr = input.split(' ')
when (expr[1])
{
'+' -> return expr[0].toInt() + expr[2].toInt()
'-' -> return expr[0].toInt() - expr[2].toInt()
'*' -> return expr[0].toInt() * expr[2].toInt()
'/' -> return expr[0].toInt() / expr[2].toInt()
// add more branches
}
However, I was wondering whether there is a better and tricky solution from the grammar of Kotlin.
What you basically want is an Char to result of an operation mapping. So, I decided to return the result right away and not a lambda.
fun Int.doOperation(charOperator: Char, x: Int) = when(charOperator) {
'+' -> this + x
'-' -> this - x
'/' -> this / x
'*' -> this * x
else -> throw IllegalArgumentException("Not supported")
}
Using an extension function maybe (?) makes the syntax a little nicer. You decide.
Call site:
println(5.doOperation('+', 6))
You can use the interpreter class in beanshell library
to convert string text automatically to result
for example
interpreter.eval("equal=2*3")
println(interpreter.get("equal").toString().toDouble().toString())
or can use expression class that does the same thing
fun String.stringToConditionalOperators(): (Boolean, Boolean) -> Boolean {
return when (this.lowercase(Locale.getDefault())) {
"and" -> {
{ a: Boolean, b: Boolean ->
a && b
}
}
"or" -> {
{ a: Boolean, b: Boolean ->
a || b
}
}
// You can add more operator 🤩
else -> {
return { a: Boolean, b: Boolean ->
a || b
}
}
}
}
Usage..
val operator = "AND"
operator.stringToConditionalOperators().invoke(one, two)
I have a trait
trait B {
type Index: Sized + Copy;
fn bounds(&self) -> (Self::Index, Self::Index);
}
I want to get all the Indexes within bounds:
fn iterate<T: B>(it: &T) {
let (low, high) = it.bounds();
for i in low..high {}
}
This won't work since there's no constraint that the type T can be "ranged" over, and the compiler says as much:
error[E0277]: the trait bound `<T as B>::Index: std::iter::Step` is not satisfied
--> src/main.rs:8:5
|
8 | for i in low..high {}
| ^^^^^^^^^^^^^^^^^^^^^ the trait `std::iter::Step` is not implemented for `<T as B>::Index`
|
= help: consider adding a `where <T as B>::Index: std::iter::Step` bound
= note: required because of the requirements on the impl of `std::iter::Iterator` for `std::ops::Range<<T as B>::Index>`
I tried adding the Step bound to Index
use std::iter::Step;
trait B {
type Index: Sized + Copy + Step;
fn bounds(&self) -> (Self::Index, Self::Index);
}
but apparently it isn't stable:
error: use of unstable library feature 'step_trait': likely to be replaced by finer-grained traits (see issue #42168)
--> src/main.rs:1:5
|
1 | use std::iter::Step;
| ^^^^^^^^^^^^^^^
error: use of unstable library feature 'step_trait': likely to be replaced by finer-grained traits (see issue #42168)
--> src/main.rs:4:32
|
4 | type Index: Sized + Copy + Step;
| ^^^^
Am I missing something or is it just not possible to do so right now?
If you want to require that a Range<T> can be iterated over, just use that as your trait bound:
trait Bounded {
type Index: Sized + Copy;
fn bounds(&self) -> (Self::Index, Self::Index);
}
fn iterate<T>(it: &T)
where
T: Bounded,
std::ops::Range<T::Index>: IntoIterator,
{
let (low, high) = it.bounds();
for i in low..high {}
}
fn main() {}
To do this kind of thing generically the num crate is helpful.
extern crate num;
use num::{Num, One};
use std::fmt::Debug;
fn iterate<T>(low: T, high: T)
where
T: Num + One + PartialOrd + Copy + Clone + Debug,
{
let one = T::one();
let mut i = low;
loop {
if i > high {
break;
}
println!("{:?}", i);
i = i + one;
}
}
fn main() {
iterate(0i32, 10i32);
iterate(5u8, 7u8);
iterate(0f64, 10f64);
}
So what I want to do is to convert a string into an int and do some error catching on it. I would also like to know where I would put what I want it to do after it fails if it does.
I know how to convert, but I am not sure how to catch it and where the code will jump to after the error
I believe the method for converting it Int.fromString(x)
Thank you.
SML has two approaches to error handling. One, based on raise to raise errors and handle to catch the error, is somewhat similar to how error handling works in languages like Python or Java. It is effective, but the resulting code tends to lose some of its functional flavor. The other method is based on the notion of options. Since the return type of Int.fromString is
string -> int option
it makes the most sense to use the option-based approach.
An int option is either SOME n, where n is and integer, or it is NONE. The function Int.fromString returns the latter if it fails in its attempt to convert the string to an integer. The function which calls Int.fromString can explicitly test for NONE and use the valOf to extract the value in the case that the return value is of the form SOME n. Alternatively, and somewhat more idiomatically, you can use pattern matching in a case expression. Here is a toy example:
fun squareString s =
case Int.fromString(s) of
SOME n => Int.toString (n * n) |
NONE => s ^ " isn't an integer";
This function has type string -> string. Typical output:
- squareString "4";
val it = "16" : string
- squareString "Bob";
val it = "Bob isn't an integer" : string
Note that the clause which starts NONE => is basically an error handler. If the function that you are defining isn't able to handle such errors, it could pass the buck. For example:
fun squareString s =
case Int.fromString(s) of
SOME n => SOME (Int.toString (n * n))|
NONE => NONE;
This has type string -> string option with output now looking like:
- squareString "4";
val it = SOME "16" : string option
- squareString "Bob";
val it = NONE : string option
This would make it the responsibility of the caller to figure out what to do with the option.
The approach to error handling that John explains is elaborated in the StackOverflow question 'Unpacking' the data in an SML DataType without a case statement. The use-case there is a bit different, since it also involves syntax trees, but the same convenience applies for smaller cases:
fun squareString s = Int.fromString s >>= (fn i => SOME (i*i))
Assuming you defined the >>= operator as:
infix 3 >>=
fun NONE >>= _ = NONE
| (SOME a) >>= f = f a
The drawback of using 'a option for error handling is that you have to take into account, every single time you use a function that has this return type, whether it errored. This is not unreasonable. It's like mandatory null-checking. But it comes at the cost of not being able to easily compose your functions (using e.g. the o operator) and a lot of nested case-ofs:
fun inputSqrt s =
case TextIO.inputLine TextIO.stdIn of
NONE => NONE
| SOME s => case Real.fromString s of
NONE => NONE
| SOME x => SOME (Math.sqrt x) handle Domain => NONE
A workaround is that you can build this constant error handling into your function composition operator, as long as all your functions share the same way of expressing errors, e.g. using 'a option:
fun safeSqrt x = SOME (Math.sqrt x) handle Domain => NONE
fun inputSqrt () =
TextIO.inputLine TextIO.stdIn >>=
(fn s => Real.fromString s >>=
(fn x => safeSqrt x))
Or even shorter by applying Eta conversion:
fun inputSqrt () = TextIO.inputLine TextIO.stdIn >>= Real.fromString >>= safeSqrt
This function could fail either because of a lack of input, or because the input didn't convert to a real, or because it was negative. Naturally, this error handling isn't smart enough to say what the error was, so you might want to extend your functions from using an 'a option to using an ('a, 'b) either:
datatype ('a, 'b) either = Left of 'a | Right of 'b
infix 3 >>=
fun (Left msg) >>= _ = Left msg
| (Right a) >>= f = f a
fun try (SOME x) _ = Right x
| try NONE msg = Left msg
fun inputLine () =
try (TextIO.inputLine TextIO.stdIn) "Could not read from stdIn."
fun realFromString s =
try (Real.fromString s) "Could not derive real from string."
fun safeSqrt x =
try (SOME (Math.sqrt x) handle Domain => NONE) "Square root of negative number"
fun inputSqrt () =
inputLine () >>= realFromString >>= safeSqrt
And trying this out:
- inputSqrt ();
9
> val it = Right 3.0 : (string, real) either
- inputSqrt ();
~42
> val it = Left "Square root of negative number" : (string, real) either
- inputSqrt ();
Hello
> val it = Left "Could not derive real from string." : (string, real) either
- (TextIO.closeIn TextIO.stdIn; inputSqrt ());
> val it = Left "Could not read from stdIn." : (string, real) either
(Disclaimer: I am fairly certain that this is not idiomatic in any way. If there is some alternative tree-traversal idiom in OCaml, I'm all ears :) )
I am writing a toy compiler in OCaml, and I would like to have a visitor for my large syntax tree type. I wrote one using classes, but I thought it would be fun to try and implement one using modules/functors. My type hierarchy is massive, so let me illustrate what I'm trying to do.
Consider the following type definitions (making these up on the spot):
type expr =
SNum of int
| SVarRef of string
| SAdd of expr * expr
| SDo of stmt list
and stmt =
SIf of expr * expr * expr
| SAssign of string * expr
Let me briefly illustrate usage. Say, for example, I wanted to collect all of the SVarRefs inside of the program. If I had a mapping visitor (one which visits every node of the tree and does nothing by default), I could do the following (in a perfect world):
module VarCollector : (sig
include AST_VISITOR
val var_refs : (string list) ref
end) = struct
include MapVisitor
let var_refs = ref []
let s_var_ref (s : string) =
var_refs := s::!var_refs
SVarRef(s)
end
(* Where my_prog is a stmt type *)
let refs = begin
let _ = VarCollector.visit_stmt my_prog in
VarCollector.!var_refs
end
I should note that the advantage of having functions for each specific variant is that my actual codebase has a type which both has a large amount of variants and does not make sense to fragment. Variant-specific functions allow the avoiding of repeated iteration implementations for the other variants of a type.
Seems simple enough, but here's the catch: there are different types of visitors. MapVisitor returns the original syntax tree, so it has type
sig
(** Dispatches to variant implementations *)
val visit_stmt : stmt -> stmt
val visit_expr : expr -> expr
(** Variant implementations *)
val s_num : int -> expr
val s_var_ref : string -> expr
val s_add : (expr * expr) -> expr
val s_do : stmt list -> expr
val s_if : (expr * expr * expr) -> stmt
val s_assign : (string * expr) -> stmt
end
At the same time, one might imagine a folding visitor in which the return type is some t for every function. Attempting to abstract this as much as possible, here is my attempt:
module type AST_DISPATCHER = sig
type expr_ret
type stmt_ret
val visit_expr : expr -> expr_ret
val visit_stmt : stmt -> stmt_ret
end
(** Concrete type designation goes in AST_VISITOR_IMPL *)
module type AST_VISITOR_IMPL = sig
type expr_ret
type stmt_ret
val s_num : int -> expr_ret
val s_var_ref : string -> expr_ret
val s_add : (expr * expr) -> expr_ret
val s_do : stmt list -> expr_ret
val s_if : (expr * expr * expr) -> stmt_ret
val s_assign : (string * expr) -> stmt_ret
end
module type AST_VISITOR = sig
include AST_VISITOR_IMPL
include AST_DISPATCHER with type expr_ret := expr_ret
and type stmt_ret := stmt_ret
end
(** Dispatcher Implementation *)
module AstDispatcherF(IM : AST_VISITOR_IMPL) : AST_DISPATCHER = struct
type expr_ret = IM.expr_ret
type stmt_ret = IM.stmt_ret
let visit_expr = function
| SNum(i) -> IM.s_num i
| SVarRef(s) -> IM.s_var_ref s
| SAdd(l,r) -> IM.s_add (l,r)
| SDo(sl) -> IM.s_do sl
let visit_stmt = function
| SIf(c,t,f) -> IM.s_if (c,t,f)
| SAssign(s,e) -> IM.s_assign (s,e)
end
module rec MapVisitor : AST_VISITOR = struct
type expr_ret = expr
type stmt_ret = stmt
module D : (AST_DISPATCHER with type expr_ret := expr_ret
and type stmt_ret := stmt_ret)
= AstDispatcherF(MapVisitor)
let visit_expr = D.visit_expr
let visit_stmt = D.visit_stmt
let s_num i = SNum i
let s_var_ref s = SVarRef s
let s_add (l,r) = SAdd(D.visit_expr l, D.visit_expr r)
let s_do sl = SDo(List.map D.visit_stmt sl)
let s_if (c,t,f) = SIf(D.visit_expr c, D.visit_expr t, D.visit_expr f)
let s_assign (s,e) = SAssign(s, D.visit_expr e)
end
Running this gives me the following error message, however:
Error: Signature Mismatch:
Values do not match:
val visit_expr : expr -> expr_ret
is not included in
val visit_expr : expr -> expr_ret
I know this means that I am not expressing the relationship between the types correctly, but I cannot figure out what the fix is in this case.
Disclaimer: Modules are just records of values accompanied with type definitions. Since there are no types in your modules there is no need to use them at all, just use plain old record types, and you will get one of the idiomatic AST traversal pattern. Soon, you will find out that you need an open recursion, and will switch to class-based approach. Anyway, this was the main reason, why classes were added to OCaml. You may also wrap your classes into a state monad, to fold AST with arbitrary user data.
What concerning your error message, then it is simple, you hide your types with signatures, a common mistake. The easiest solution is to omit an annotation of the return type of a functor at all, or to propagate type equality with a with type expr = expr annotations.
If you need examples of more idiomatic approaches, then for records you can go for ppx mappers, here is an example of different visitors implemented with classes, including those that are wrapped into a state monad.