how to add relational operations to my code
Thanks
My code is
grammar denem1;
options {
output=AST;
}
tokens {
ROOT;
}
parse
: stat+ EOF -> ^(ROOT stat+)
;
stat
: expr ';'
;
expr
: Id Assign expr -> ^(Assign Id expr)
| add
;
add
: mult (('+' | '-')^ mult)*
;
mult
: atom (('*' | '/')^ atom)*
;
atom
: Id
| Num
| '('! expr ')' !
;
Assign : '=' ;
Comment : '//' ~('\r' | '\n')* {skip();};
Id : 'a'..'z'+;
Num : '0'..'9'+;
Space : (' ' | '\t' | '\r' | '\n')+ {skip();};
Like this:
...
expr
: Id Assign expr -> ^(Assign Id expr)
| rel
;
rel
: add (('<=' | '<' | '>=' | '>')^ add)?
;
add
: mult (('+' | '-')^ mult)*
;
...
If possible, use ANTLR v4 instead of the old v3. In v4, you can simply do this:
stat
: expr ';'
;
expr
: Id Assign expr
| '-' expr
| expr ('*' | '/') expr
| expr ('+' | '-') expr
| expr ('<=' | '<' | '>=' | '>') expr
| Id
| Num
| '(' expr ')'
;
Related
I have the following the expressions I need to parse
and(true, false)
or(true, false, true)
not(or(true, false, true))
and(and(true, false), false)
or(or(true, false, true), true)
so far I have the following grammar
expr
: orExpr
;
orExpr
: OR '(' andExpr (',' andExpr)+ ')'
| andExpr
;
andExpr
: AND '(' equalityExpr (',' equalityExpr)+ ')'
| equalityExpr
;
equalityExpr
: comparison ((EQUALS | NOT_EQUALS) comparison)*
;
comparison
: notExpr ((GREATER_THAN_EQUALS | LESS_THAN_EQUALS | GREATER_THAN | LESS_THAN ) notExpr)?
;
notExpr
: NOT '(' expr ')'
| primary
;
primary
: '(' expr ')'
| true
| false
;
I am not able to parse and(and(true, false), false) with the above grammar? where am I going wrong?
Please assume there is precedence between AND OR NOT (although I understand it may look not necessary)
But for now, I need to be able to parse true=and(5=5, 4=4, 3>2) or vice-versa and(5=5, 4=4, 3>2)=true ?
In that case, there is absolutely no need to complicate things. All you have to do is this:
grammar Test;
parse
: expr EOF
;
expr
: '(' expr ')'
| OR '(' expr (',' expr)+ ')'
| AND '(' expr (',' expr)+ ')'
| NOT '(' expr ')'
| expr (LT | LTE | GT | GTE) expr
| expr (EQ | NEQ) expr
| TRUE
| FALSE
| INT
;
LT : '<';
LTE : '<=';
GT : '>';
GTE : '>=';
NEQ : '!=';
EQ : '=';
NOT : 'not';
TRUE : 'true';
FALSE : 'false';
AND : 'and';
OR : 'or';
INT
: [0-9]+
;
SPACES
: [ \t\r\n] -> skip
;
I am creating a the parser for DECAF with Antlr
grammar DECAF ;
//********* LEXER ******************
LETTER: ('a'..'z'|'A'..'Z') ;
DIGIT : '0'..'9' ;
ID : LETTER( LETTER | DIGIT)* ;
NUM: DIGIT(DIGIT)* ;
COMMENTS: '//' ~('\r' | '\n' )* -> channel(HIDDEN);
WS : [ \t\r\n\f | ' '| '\r' | '\n' | '\t']+ ->channel(HIDDEN);
CHAR: (LETTER|DIGIT|' '| '!' | '"' | '#' | '$' | '%' | '&' | '\'' | '(' | ')' | '*' | '+'
| ',' | '-' | '.' | '/' | ':' | ';' | '<' | '=' | '>' | '?' | '#' | '[' | '\\' | ']' | '^' | '_' | '`'| '{' | '|' | '}' | '~'
'\t'| '\n' | '\"' | '\'');
// ********** PARSER *****************
program : 'class' 'Program' '{' (declaration)* '}' ;
declaration: structDeclaration| varDeclaration | methodDeclaration ;
varDeclaration: varType ID ';' | varType ID '[' NUM ']' ';' ;
structDeclaration : 'struct' ID '{' (varDeclaration)* '}' ;
varType: 'int' | 'char' | 'boolean' | 'struct' ID | structDeclaration | 'void' ;
methodDeclaration : methodType ID '(' (parameter (',' parameter)*)* ')' block ;
methodType : 'int' | 'char' | 'boolean' | 'void' ;
parameter : parameterType ID | parameterType ID '[' ']' ;
parameterType: 'int' | 'char' | 'boolean' ;
block : '{' (varDeclaration)* (statement)* '}' ;
statement : 'if' '(' expression ')' block ( 'else' block )?
| 'while' '(' expression ')' block
|'return' expressionA ';'
| methodCall ';'
| block
| location '=' expression
| (expression)? ';' ;
expressionA: expression | ;
location : (ID|ID '[' expression ']') ('.' location)? ;
expression : location | methodCall | literal | expression op expression | '-' expression | '!' expression | '('expression')' ;
methodCall : ID '(' arg1 ')' ;
arg1 : arg2 | ;
arg2 : (arg) (',' arg)* ;
arg : expression;
op: arith_op | rel_op | eq_op | cond_op ;
arith_op : '+' | '-' | '*' | '/' | '%' ;
rel_op : '<' | '>' | '<=' | '>=' ;
eq_op : '==' | '!=' ;
cond_op : '&&' | '||' ;
literal : int_literal | char_literal | bool_literal ;
int_literal : NUM ;
char_literal : '\'' CHAR '\'' ;
bool_literal : 'true' | 'false' ;
When I give it the input:
class Program {
void main(){
return 3+5 ;
}
}
The parse tree is not building correctly since it is not recognizing the 3+5 as an expression. Is there anything wrong with my grammar that is causing the problem?
Lexer rules are matched from top to bottom. When 2 or more lexer rules match the same amount of characters, the one defined first will win. Because of that, a single digit integer will get matched as a DIGIT instead of a NUM.
Try parsing the following instead:
class Program {
void main(){
return 33 + 55 ;
}
}
which will be parsed just fine. This is because 33 and 55 are matched as NUMs, because NUM can now match 2 characters (DIGIT only 1, so NUM wins).
To fix it, make DIGIT a fragment (and LETTER as well):
fragment LETTER: ('a'..'z'|'A'..'Z') ;
fragment DIGIT : '0'..'9' ;
ID : LETTER( LETTER | DIGIT)* ;
NUM: DIGIT(DIGIT)* ;
Lexer fragments are only used internally by other lexer rules, and will never become tokens of their own.
A couple of other things: your WS rule matches way too much (it now also matches a | and a '), it should be:
WS : [ \t\r\n\f]+ ->channel(HIDDEN);
and you shouldn't match a char literal in your parser: do it in the lexer:
CHAR : '\'' ( ~['\r\n\\] | '\\' ['\\] ) '\'';
If you don't, the following will not get parsed properly:
class Program {
void main(){
return '1';
}
}
because the 1 wil be tokenized as a NUM and not as a CHAR.
I try to modify the grammar of the sqlite syntax (I'm interested in a variant of the where clause only) and I'm keep having a weird error when substituting AND to it's own token.
grammar wtfql;
/*
SQLite understands the following binary operators, in order from highest to
lowest precedence:
||
* / %
+ -
<< >> & |
< <= > >=
= != <> IS IS NOT IN LIKE GLOB MATCH REGEXP
AND
OR
*/
start : expr EOF?;
expr
: literal_value
//BIND_PARAMETER
| ( table_name '.' )? column_name
| unary_operator expr
| expr '||' expr
| expr ( '*' | '/' | '%' ) expr
| expr ( '+' | '-' ) expr
| expr ( '<' | '<=' | '>' | '>=' ) expr
| expr ( '=' | '<>' | K_IN ) expr
| expr K_AND expr
| expr K_OR expr
| function_name '(' ( expr ( ',' expr )* )? ')'
| '(' expr ')'
| expr K_NOT expr
| expr ( K_NOT K_NULL )
| expr K_NOT? K_IN ( '(' ( expr ( ',' expr )* ) ')' )
;
unary_operator
: '-'
| '+'
| K_NOT
;
literal_value
: NUMERIC_LITERAL
| STRING_LITERAL
| K_NULL
;
function_name
: IDENTIFIER
;
table_name
: any_name
;
column_name
: any_name
;
any_name
: IDENTIFIER
| keyword
// | '(' any_name ')'
;
keyword
: K_AND
| K_NOT
| K_NULL
| K_IN
| K_OR
;
IDENTIFIER
: [a-zA-Z_] [a-zA-Z_0-9]* // TODO check: needs more chars in set
;
NUMERIC_LITERAL
: DIGIT+ ( '.' DIGIT* )? ( E [-+]? DIGIT+ )?
| '.' DIGIT+ ( E [-+]? DIGIT+ )?
;
STRING_LITERAL
: '\"' ( ~'\"' | '\"\"' )* '\"'
;
SPACES
: [ \u000B\t\r\n] -> channel(HIDDEN)
;
DOT : '.';
OPEN_PAR : '(';
CLOSE_PAR : ')';
COMMA : ',';
STAR : '*';
PLUS : '+';
MINUS : '-';
TILDE : '~';
DIV : '/';
MOD : '%';
AMP : '&';
PIPE : '|';
LT : '<';
LT_EQ : '<=';
GT : '>';
GT_EQ : '>=';
EQ : '=';
NOT_EQ2 : '<>';
K_AND : A N D;
K_NOT : N O T;
K_NULL : N U L L;
K_OR : O R;
K_IN : I N;
fragment DIGIT : [0-9];
fragment A : [aA];
fragment B : [bB];
fragment C : [cC];
fragment D : [dD];
fragment E : [eE];
fragment F : [fF];
fragment G : [gG];
fragment H : [hH];
fragment I : [iI];
fragment J : [jJ];
fragment K : [kK];
fragment L : [lL];
fragment M : [mM];
fragment N : [nN];
fragment O : [oO];
fragment P : [pP];
fragment Q : [qQ];
fragment R : [rR];
fragment S : [sS];
fragment T : [tT];
fragment U : [uU];
fragment V : [vV];
fragment W : [wW];
fragment X : [xX];
fragment Y : [yY];
fragment Z : [zZ];
writing
| expr K_AND expr
with the input
field1=1 and field2 = 2
results in
line 1:8 mismatched input 'and' expecting {<EOF>, '||', '*', '+', '-', '/', '%', '<', '<=', '>', '>=', '=', '<>', K_AND, K_NOT, K_OR, K_IN}
while
| expr 'and' expr
works like a charm:
$ antlr4 wtfql.g4 && javac -classpath /usr/local/Cellar/antlr/4.4/antlr-4.4-complete.jar wtfql*.java && cat test.txt | grun wtfql start -tree -gui
(start (expr (expr (expr (column_name (any_name feld1))) = (expr (literal_value 1))) and (expr (expr (column_name (any_name feld2))) = (expr (literal_value 2)))) <EOF>)
What am I missing?
I presume "and" is an IDENTIFIER since the rule for IDENTIFIER comes before the rule for AND and thus wins.
If you write 'and' in the parser rule this implicitly creates a token (not AND!) which comes before IDENTIFIER and thus wins.
Rule of thumb: More specific lexer rules first. Don't create new lexer tokens implicitly in parser rules.
If you check the token type, you'll get a clue what's going on.
My Grammar file (see below) parses queries of the type:
(name = Jon AND age != 16 OR city = NY);
However, it doesn't allow something like:
(name = 'Jon Smith' AND age != 16);
ie, it doesn't allow assign to a field values with more than one word, separated by White Spaces. How can I modify my grammar file to accept that?
options
{
language = Java;
output = AST;
}
tokens {
BLOCK;
RETURN;
QUERY;
ASSIGNMENT;
INDEXES;
}
#parser::header {
package pt.ptinovacao.agorang.antlr;
}
#lexer::header {
package pt.ptinovacao.agorang.antlr;
}
query
: expr ('ORDER BY' NAME AD)? ';' EOF
-> ^(QUERY expr ^('ORDER BY' NAME AD)?)
;
expr
: logical_expr
;
logical_expr
: equality_expr (logical_op^ equality_expr)*
;
equality_expr
: NAME equality_op atom -> ^(equality_op NAME atom)
| '(' expr ')' -> ^('(' expr)
;
atom
: ID
| id_list
| Int
| Number
;
id_list
: '(' ID (',' ID)* ')'
-> ID+
;
NAME
: 'equipType'
| 'equipment'
| 'IP'
| 'site'
| 'managedDomain'
| 'adminState'
| 'dataType'
;
AD : 'ASC' | 'DESC' ;
equality_op
: '='
| '!='
| 'IN'
| 'NOT IN'
;
logical_op
: 'AND'
| 'OR'
;
Number
: Int ('.' Digit*)?
;
ID
: ('a'..'z' | 'A'..'Z' | '_' | '.' | '-' | Digit)*
;
String
#after {
setText(getText().substring(1, getText().length()-1).replaceAll("\\\\(.)", "$1"));
}
: '"' (~('"' | '\\') | '\\' ('\\' | '"'))* '"'
| '\'' (~('\'' | '\\') | '\\' ('\\' | '\''))* '\''
;
Comment
: '//' ~('\r' | '\n')* {skip();}
| '/*' .* '*/' {skip();}
;
Space
: (' ' | '\t' | '\r' | '\n' | '\u000C') {skip();}
;
fragment Int
: '1'..'9' Digit*
| '0'
;
fragment Digit
: '0'..'9'
;
indexes
: ('[' expr ']')+ -> ^(INDEXES expr+)
;
Include the String token as an alternative in your atom rule:
atom
: ID
| id_list
| Int
| Number
| String
;
I'm using Antlr 4 to build a compiler for a made up language. I'm having problems with eliminating whitespace properly. It will get rid of whitespace between tokens but it also delete whitespace within the string token which is obviously not what I want. I've tried using modes to clear this issue up with no avail.
Lexer.g4
lexer grammar WaccLexer;
SEMICOLON: ';' ;
WS: [ \n\t\r\u000C]+ -> skip;
EOL: '\n' ;
BEGIN: 'begin' ;
END: 'end' ;
SKIP: 'skip' ;
READ: 'read' ;
FREE: 'free' ;
RETURN: 'return' ;
EXIT: 'exit' ;
IS: 'is' ;
PRINT: 'print' ;
PRINTLN: 'println' ;
IF: 'if' ;
THEN: 'then' ;
ELSE: 'else' ;
FI: 'fi' ;
WHILE: 'while' ;
DO: 'do' ;
DONE: 'done' ;
NEWPAIR: 'newpair' ;
CALL: 'call' ;
FST: 'fst' ;
SND: 'snd' ;
INT: 'int' ;
BOOL: 'bool' ;
CHAR: 'char' ;
STRING: 'string' ;
PAIR: 'pair' ;
EXCLAMATION: '!' ;
LEN: 'len' ;
ORD: 'ord' ;
TOINT: 'toInt' ;
DIGIT: '0'..'9' ;
LOWCHAR: 'a'..'z' ;
R: 'r' ;
F: 'f' ;
N: 'n' ;
T: 't' ;
B: 'b' ;
ZERO: '0' ;
MULTI: '*' ;
DIVIDE: '/' ;
MOD: '%' ;
PLUS: '+' ;
MINUS: '-' ;
GT: '>' ;
GTE: '>=' ;
LT: '<' ;
LTE: '<=' ;
DOUBLEEQUAL: '==' ;
EQUAL: '=' ;
NOTEQUAL: '!=' ;
AND: '&&' ;
OR: '||' ;
UNDERSCORE: '_' ;
UPCHAR: 'A'..'Z' ;
OPENSQUARE: '[' ;
CLOSESQUARE: ']' ;
OPENPARENTHESIS: '(' ;
CLOSEPARENTHESIS: ')' ;
TRUE: 'true' ;
FALSE: 'false' ;
SINGLEQUOT: '\'' ;
DOUBLEQUOT: '\"' ;
BACKSLASH: '\\' ;
COMMA: ',' ;
NULL: 'null' ;
OPENSTRING : DOUBLEQUOT -> pushMode(STRINGMODE) ;
COMMENT: '#' ~[\r\n]* '\r'? '\n' -> skip ;
mode STRINGMODE ;
CLOSESTRING : DOUBLEQUOT -> popMode ;
CHARACTER : ~[\"\'\\] | (BACKSLASH ESCAPEDCHAR) ;
STRLIT : (CHARACTER)* ;
ESCAPEDCHAR : ZERO
| B
| T
| N
| F
| R
| DOUBLEQUOT
| SINGLEQUOT
| BACKSLASH
;
Parser.g4
parser grammar WaccParser;
options {
tokenVocab=WaccLexer;
}
program : BEGIN (func)* stat END EOF;
func : type ident OPENPARENTHESIS (paramlist)? CLOSEPARENTHESIS IS stat END ;
paramlist : param (COMMA param)* ;
param : type ident ;
stat : SKIP
| type ident EQUAL assignrhs
| assignlhs EQUAL assignrhs
| READ assignlhs
| FREE expr
| RETURN expr
| EXIT expr
| PRINT expr
| PRINTLN expr
| IF expr THEN stat ELSE stat FI
| WHILE expr DO stat DONE
| BEGIN stat END
| stat SEMICOLON stat
;
assignlhs : ident
| expr OPENSQUARE expr CLOSESQUARE
| pairelem
;
assignrhs : expr
| arrayliter
| NEWPAIR OPENPARENTHESIS expr COMMA expr CLOSEPARENTHESIS
| pairelem
| CALL ident OPENPARENTHESIS (arglist)? CLOSEPARENTHESIS
;
arglist : expr (COMMA expr)* ;
pairelem : FST expr
| SND expr
;
type : basetype
| type OPENSQUARE CLOSESQUARE
| pairtype
;
basetype : INT
| BOOL
| CHAR
| STRING
;
pairtype : PAIR OPENPARENTHESIS pairelemtype COMMA pairelemtype CLOSEPARENTHESIS ;
pairelemtype : basetype
| type OPENSQUARE CLOSESQUARE
| PAIR
;
expr : intliter
| boolliter
| charliter
| strliter
| pairliter
| ident
| expr OPENSQUARE expr CLOSESQUARE
| unaryoper expr
| expr binaryoper expr
| OPENPARENTHESIS expr CLOSEPARENTHESIS
;
unaryoper : EXCLAMATION
| MINUS
| LEN
| ORD
| TOINT
;
binaryoper : MULTI
| DIVIDE
| MOD
| PLUS
| MINUS nus
| GT
| GTE
| LT
| LTE
| DOUBLEEQUAL
| NOTEQUAL
| AND
| OR
;
ident : (UNDERSCORE | LOWCHAR | UPCHAR) (UNDERSCORE | LOWCHAR | UPCHAR | DIGIT)* ;
intliter : (intsign)? (digit)+ ;
digit : DIGIT ;
intsign : PLUS
| MINUS
;
boolliter : TRUE
| FALSE
;
charliter : CHARACTER;
strliter : OPENSTRING STRLIT CLOSESTRING;
arrayliter : OPENSQUARE (expr (COMMA expr)*)? CLOSESQUARE ;
Please also remember that comment starting with # need to be ignored. Thanks in advance.
The OPENSTRING lexer rule will never be matched in your grammar because the DOUBLEQUOT rule matches exactly the same input sequence and appears before it in the grammar. If you want to define a lexer rule, but you do not actually want that lexer rule to create a token on its own, then you need to define the rule with the fragment modifier.
fragment DOUBLEQUOT : '"';
In addition, you need to correct the warnings that appear when you generate code for your grammar. At least one of them (defined as EPSILON_TOKEN) indicates a major mistake that you made that used to be an error in ANTLR 4.0 but was changed to a warning in ANTLR 4.1 since there is an edge case where it can be used without problems.