Antlr production producing a bunch of single element arrays - antlr

Question

My grammar is working, but I have a bunch of elements in the tree that are single element arrays, and I don't really understand why. I tried reading the information about visitors, but I'm pretty sure the "problem" is with the grammar and perhaps its verbosity. Does anything jump out here? Or perhaps I'm just visiting things incorrectly. In the example below I do not react to visitFnArgs or visitArgs, but just visitFunctionCall. Things like function arguments and statements seem to sometimes be wrapped in single element arrays.
grammar Txl;
root: program;
// High level language
program: stmt (NEWLINE stmt)* NEWLINE? EOF # Statement
;
stmt: require # Condition
| entry # CreateEntry
| assignment # Assign
;
require: REQUIRE valueExpression;
entry: (CREDIT | DEBIT) journal valueExpression (IF valueExpression)? (LPAREN 'id:' valueExpression RPAREN)?;
assignment: IDENT ASSIGN valueExpression;
journal: IDENT COLON IDENT;
valueExpression: expr # Expression;
expr: expr (MULT | DIV) expr # MulDiv
| expr (PLUS | MINUS) expr # AddSub
| expr MOD expr # Mod
| expr POW expr # Pow
| MINUS expr # Negative
| expr AND expr # And
| expr OR expr # Or
| NOT expr # Not
| expr EQ expr # Equality
| expr NEQ expr # Inequality
| expr (LTE | GTE) expr # CmpEqual
| expr (LT | GT) expr # Cmp
| expr QUESTION expr COLON expr # Ternary
| LPAREN expr RPAREN # Parens
| NUMBER # NumberLiteral
| IDENT LPAREN args RPAREN # FunctionCall
| IDENT # Identifier
| STRING_LITERAL # StringLiteral
;
fnArg: expr | journal;
args: (fnArg (',' fnArg)*)?;
// Reserved words
CREDIT: 'credit';
DEBIT: 'debit';
IF: 'if';
REQUIRE: 'require';
// Operators
MULT: '*';
DIV: '/';
MINUS: '-';
PLUS: '+';
POW: '^';
MOD: '%';
LPAREN: '(';
RPAREN: ')';
LBRACE: '[';
RBRACE: ']';
COMMA: ',';
EQ: '==';
NEQ: '!=';
GTE: '>=';
LTE: '<=';
GT: '>';
LT: '<';
ASSIGN: '=';
QUESTION: '?';
COLON: ':';
AND: 'and';
OR: 'or';
NOT: 'not';
HASH: '#';
NEWLINE : [\r\n];
WS: [ \t] + -> skip;
// Entities
NUMBER: ('0' .. '9') + ('.' ('0' .. '9') +)?;
IDENT: [a-zA-Z]+[0-9a-zA-Z]*;
EXTID: [a-zA-Z0-9-]+;
STRING_LITERAL : '"' (~('"' | '\\' | '\r' | '\n') | '\\' ('"' | '\\'))* '"';
This input:
require balance(assets:cash) + balance(assets:earnings) > AMT
Produces the following single element arrays:
SINGLE ELEMENT INSTRUCTION MathOperation (>)
SINGLE ELEMENT INSTRUCTION JournalReference { identifier: 'assets:cash' }
SINGLE ELEMENT INSTRUCTION JournalReference { identifier: 'assets:earnings' }
I wonder if partly my problem is I'm not visiting things properly. Here's my Math visitor:
visitMath(ctx) {
const visited = this.visitChildren(ctx);
return new MathOperation(
visited[0],
ctx.getChild(1).getText(),
visited[2],
);
}
But I assume the problem is in the thing that contains the math operation, which I think is visitRequire:
visitRequire(ctx) {
return new Condition(this.visitExpression(ctx.getChild(1)));
}
Or perhaps in visitValueExpression or visitCondition, which are not overridden in my visitor.

Answer 1

Really short answer: There's nothing wrong with single element arrays. If there was only one instance of a thing that could exist multiple times, then it has to be an array (or List), and that list will have only the one item, if that's how many there are.
Antlr won't "unwrap" a single item to not be in an array. (That would only be valid in untyped languages or languages that allow Union types, and would be a pain to use as you'd always have to check whether you had a "thing" or a list of "thing"s)
Any time the "same type of thing" can exist more than once when matching a rule, ANTLR will make that available as an Array/List of that type.
Eample:
journal: IDENT COLON IDENT;
has 2 IDENT tokens, so it'll be made accessible via the context as a List of those types
(in Java, I'm not positive which language you're using).
public List<TerminalNode> IDENT() { return getTokens(TxlParser.IDENT); }
Two of your examples are of "JournalReference" so this would explain getting a list (if you use the ctx.IDENT() or the ctx.getChild(n) methods).
If I change the Journal rule to be:
journal: j1=IDENT COLON j2=IDENT;
I've given names to each IDENT so I get individual accessors for them (in addition to the IDENT() accessor that returns a list:
public static class JournalContext extends ParserRuleContext {
public Token j1;
public Token j2;
public TerminalNode COLON() { return getToken(TxlParser.COLON, 0); }
public List<TerminalNode> IDENT() { return getTokens(TxlParser.IDENT); }
With the labels you can use cox.j1 or cox.j2 to get individual tokens. (of course you'd name them as appropriate to your use case).
since the FunctionCall alternative of the expr rule uses the args rule
args: (fnArg (',' fnArg)*)?;
and that rule can have more than one fnArg, the it will necessarily be a list of fnArgs in the context:
public static class ArgsContext extends ParserRuleContext {
public List<FnArgContext> fnArg() {
return getRuleContexts(FnArgContext.class);
}
There's really not much you can do (or should want to do to not have that in a List, there can be one or more of them.
Since non of the code you present shows where you're writing your output, its a bit difficult to be more specific than that.
Your visitMath(cox) example is also a bit perplexing as math is not a rule in your grammar, so it would not exist in the Visitor interface.
I would suggest taking a closer look at the *Context classes that are generated for you. They'll provide utility methods that will be much easy to use and read in the future than getChild(n). getChild(n) is obscure, in that you'll have to refer back to the rule and diligently count rule members to determine which child to get, and it is also VERY brittle, in that n will change with any modification to your grammar. (Maintainers, or future you, will appreciate using the utility methods instead.)