The following works fine:
testRoot
: sqlStatement EOF ?
;
However, if I add in the following:
testRoot
: sqlStatement (SEMI | EOF) ?
;
The same input now gives me the following error message:
line 1:8 extraneous input '<EOF>' expecting {<EOF>, SEMI}
Is there something that I'm missing here, or why is that error popping up?
EOF really isn't something that's optional. A token stream will always have an EOF token at the end of the stream.
testRoot: sqlStatement SEMI? EOF;
You always want an EOF at the end of a start rule. Without it, ANTLR may be able to recognize a portion of your input, and will discard the remainder without an error. With it, the rule says... this rule has to end with the EOF token, so it will consume all of the input to get to the EOF and will report any syntax errors it encounters.
Related
I'm doing some experiments with ANTLR4 with this grammar:
srule
: '(' srule ')'
| srule srule
| '(' ')';
this grammar is for the language of balanced parentheses.
The problem is that when I run antlr with this string: (()))(
This string is obviously wrong but antlr simply return this AST:
It seems to stop when it finds the wrong parenthesis, but no error message returns. I would like to know more about this behavior. Thank you
The parser recognises (())) and then stops. If you want to force the parser to consume all tokens, "anchor" your test rule with the EOF token:
parse_all
: srule EOF
;
Btw, it's always a good idea to include the EOF token in the entry point (entry rule) of your grammar.
i have following grammar
root
: sql_statements EOF | EOF
;
sql_statements
:( sql_statement )+
;
sql_statement
: ddl_statement semicolon
;
ddl_statement
: alter_statement
;
This is a initial snapshot of my grammar
i have following test case
aleter table user;alter table group_user;
first statement has error hence i am getting error for both statements under sql_statements rule
Failed to parse at line 1:1 due to mismatched input 'aleter' expecting ALTER
Now i want to use Defaulterrorhandler functionality to skip first statement till semicolon.
Then it should start from second statement starting with rule sql_statements.
Question :
How to tell which rule to start after consuming error tokens ?
Thanks in advance.
I'm trying to recover from an error in an If-Else statement.
In my grammar an If is always followed by an Else.
statement: OBRACES statements CBRACES
| IF OPAR exp CPAR statement ELSE statement
| IF OPAR exp CPAR statement error '\n' { yyerrok; yyclearin;}
;
The error found is in the commented else in the last lines:
public boolean Equal(Element other){
if (!this.Compare(aux01,Age))
ret_val = false ;
//else
//nt = 0 ;
}
error: syntax error, unexpected CBRACES, expecting ELSE -> } # line 29
It is not recovering from that error ignoring the errors that come after.
Maybe i'm not understanding well how this error works but i can only find 2 examples on every site about error recovery: "error '\n'" and "'(' error ')'"
Anyone have an idea how to recover from this error (when an if is not followed by an else).
Thanks
You've provided not enough context to know exactly, but my guess is that the lexer/tokenizer, which feeds tokens to your parser, skips white space - including '\n'. So the parser never sees the newline and b/c of that never reduces the
IF OPAR exp CPAR statement error '\n'
production and so its action
{ yyerrok; yyclearin;}
never gets executed and so the error is not recovered.
Probably (although it is hard to say for sure without seeing more of the grammar) you don't need to skip any tokens in the case that else is not found.
The most likely case is that the program is simply lacking an else clause (perhaps because its author is used to other programming languages in which else is optional), and parsing can simply continue as though there were an empty else clause. So you should be able to just use:
IF OPAR exp CPAR statement error { yyerrok; }
(Note: I removed yyclearin because you almost certainly don't want to do that. In the case of the error in the OP, the result would be to ignore the '}' token, leading to extraneous errors later in the parse.)
You probably should take advantage of the action in this error production to produce a clear error message ("if statements must have else clauses"), although the default message is reasonably clear as well.
It is certainly the case that whatever token(s) are used as error context must be produceable by the scanner. That generally precludes error recovery techniques such as "skip to the end of the line", except in the case of languages in which newlines are syntactically significant.
I found out the problem. Thanks for the help anyway.
I put the error inside the braces and now is working.
statement: OBRACES statements CBRACES
| OBRACES error CBRACES { yyerrok; yyclearin;}
| IF OPAR exp CPAR statement ELSE statement
;
My question is in regards to running the following grammar in ANTLRWorks:
INT :('0'..'9')+;
SEMICOLON: ';';
NEWLINE: ('\r\n'|'\n'|'\r');
STMTEND: (SEMICOLON (NEWLINE)*|NEWLINE+);
statement
: STMTEND
| INT STMTEND
;
program: statement+;
I get the following results with the following input (with program as the start rule), regardless of which newline NL (CR/LF/CRLF) or integer I choose:
"; NL" or "32; NL" parses without error.
";" or "45;" (without newlines) result in EarlyExitException.
"NL" by itself parses without error.
"456 NL", without the semicolon, results in MismatchedTokenException.
What I want is for a statement to be terminated by a newline, semicolon, or semicolon followed by newline, and I want the parser to eat as many contiguous newlines as it can on a termination, so "; NL NL NL NL" is just one termination, not four or five. Also, I would like the end-of-file case to be a valid termination as well, but I don't know how to do that yet.
So what's wrong with this, and how can I make this terminate nicely at EOF? I'm completely new to all of parsing, ANTLR, and EBNF, and I haven't found much material to read on it at a level somewhere in between the simple calculator example and the reference (I have The Definitive ANTLR Reference, but it really is a reference, with a quick start in the front which I haven't yet got to run outside of ANTLRWorks), so any reading suggestions (besides Wirth's 1977 ACM paper) would be helpful too. Thanks!
In case of input like ";" or "45;", the token STMTEND will never be created.
";" will create a single token: SEMICOLON, and "45;" will produce: INT SEMICOLON.
What you (probably) want is that SEMICOLON and NEWLINE never make it to real tokens themselves, but they will always be a STMTEND. You can do that by making them so called "fragment" rules:
program: statement+;
statement
: STMTEND
| INT STMTEND
;
INT : '0'..'9'+;
STMTEND : SEMICOLON NEWLINE* | NEWLINE+;
fragment SEMICOLON : ';';
fragment NEWLINE : '\r' '\n' | '\n' | '\r';
Fragment rules are only available for other lexer rules, so they will never end up in parser (production) rules. To emphasize: the grammar above will only ever create either INT or STMTEND tokens.
ANTLR gives me the following error when my input file has either no newline at the EOF, or more than one.
line 0:-1 mismatched input '' expecting NEWLINE
How would I go about taking into account the possibilities of having multiple or no newlines at the end of the input file. Preferably I'd like to account for this in the grammar.
The rule:
parse
: (Token LineBreak)+ EOF
;
only parses a stream of tokens, separated by exactly one line break, ending with exactly one line break.
While the rule:
parse
: Token (LineBreak+ Token)* LineBreak* EOF
;
parses a stream of tokens separated by one or more line breaks, ending with zero, one or more line breaks.
But do you really need to make the line breaks visible in the parser? Couldn't you put them on a "hidden channel" instead?
If this doesn't answer your question, you'll have to post your grammar (you can edit your original question for that).
HTH