How to implement an AST given a grammar with JJTree - grammar

I'm trying to implement a parser from LaTeX to HTML, to complete my exercise I need to write a JavaCC grammar, generate the Abstract syntax tree and implement a visitor to parse the code.
I've written my .jj grammar file, now I'm confused about how to use jjtree to generate the AST based on grammar file. Anyone can help me?
Here you are my grammar file, if it can help.
ArrayList<LaTeXObject> LaTeX() :
{
ArrayList<LaTeXObject> objects;
}
{
objects = ObjectList() <EOF>
{
return objects;
}
}
ArrayList<LaTeXObject> ObjectList() :
{
ArrayList<LaTeXObject> objects = new ArrayList<LaTeXObject>();
LaTeXObject object;
}
{
( object = Object() { objects.add(object); } )*
{
return objects;
}
}
LaTeXObject Object() :
{
LaTeXObject object;
}
{
(
object = Command()
|
object = Group()
|
object = String()
)
{
return object;
}
}
LaTeXCommand Command() :
{
String name;
}
{
<BACKSLASH>
(
name = Name() Whitespace()
|
name = SpecialCharacter()
|
name = NonSpecialCharacter()
)
{
return new LaTeXCommand(name);
}
}
String Name() :
{
StringBuilder sb = new StringBuilder();
Token token;
}
{
token = <ASCII_LETTER> { sb.append(token.image); } ( LOOKAHEAD( <ASCII_LETTER> ) token = <ASCII_LETTER> { sb.append(token.image); } )*
{
return sb.toString();
}
}
void Whitespace() :
{}
{
( LOOKAHEAD( WhitespaceCharacter() ) WhitespaceCharacter() )*
}
String WhitespaceCharacter() :
{
Token token;
}
{
token = <WHITESPACE>
{
return token.image;
}
}
String SpecialCharacter() :
{
Token token;
}
{
(
token = <BACKSLASH>
|
token = <LBRACE>
|
token = <RBRACE>
|
token = <SPECIAL>
)
{
return token.image;
}
}
String NonSpecialCharacter() :
{
Token token;
}
{
token = <NON_SPECIAL>
{
return token.image;
}
}
LaTeXGroup Group() :
{
ArrayList<LaTeXObject> objects;
}
{
<LBRACE> objects = ObjectList() <RBRACE>
{
return new LaTeXGroup(objects);
}
}
LaTeXString String() :
{
StringBuilder sb = new StringBuilder();
String string;
}
{
string = TextCharacter() { sb.append(string); } ( LOOKAHEAD( TextCharacter() ) string = TextCharacter() { sb.append(string); } )*
{
return new LaTeXString(sb.toString());
}
}
String TextCharacter() :
{
Token token;
}
{
(
token = <WHITESPACE>
|
token = <NON_SPECIAL>
|
token = <SPECIAL>
|
token = <ASCII_LETTER>
|
token = <ASCII_DIGIT>
|
token = <LATIN_SUPPLEMENT>
|
token = <UNICODE_LETTER>
)
{
return token.image;
}
}

So what you have now does build an abstract syntax tree. If you are happy with that, you don't need JJTree at all.
If you really want to use JJTree, you should strip out all the code that makes LatexObjectobjects. Just make all your nonterminal productions return void. Also rename the file from .jj to .jjt. Now run JJTree with your .jjt files as input. Low and behold you'll have a new .jj file that builds an abstract syntax tree. Now fiddle with the .jjt file until the abstract syntax tree it produces is one you are happy with.

Related

Nextflow dynamic includes and output

I'm trying to use dynamic includes but I have problem to manage output files:
/*
* enables modules
*/
nextflow.enable.dsl = 2
include { requestData } from './modules/get_xapi_data'
include { uniqueActors } from './modules/unique_actors'
include { compileJson } from './modules/unique_actors'
if (params.user_algo) {
include { userAlgo } from params.user_algo
}
workflow {
dataChannel = Channel.from("xapi_data.json")
requestData(dataChannel)
uniqueActors(requestData.out.channel_data)
if (params.user_algo) {
user_algo = userAlgo(requestData.out.channel_data)
} else {
user_algo = null
}
output_json = [user_algo, uniqueActors.out]
// Filter output
Channel.fromList(output_json)
.filter{ it != null } <--- problem here
.map{ file(it) }
.set{jsonFiles}
compileJson(jsonFiles)
}
The problem is userAlgo can be dynamically loaded. And I don't know how I can take care of it. With this solution, I got a Unknown method invocation getFileSystem on ChannelOut type error.
The problem is that fromList expects a list of values, not a list of channels. If you use an empty Channel instead of checking for a null value, you could use:
if( params.user_algo ) {
user_algo = userAlgo(requestData.out.channel_data)
} else {
user_algo = Channel.empty()
}
user_algo
| concat( uniqueActors.out )
| map { file(it) }
| compileJson

Flyweight pattern in this simple net core Api uses more memory ram

I'm trying to aplicate Flyweight method pattern in a simple .net core Api to see how much memory is saved compared to not using the pattern.
I have two methods, the first one creates 5000 objects without uses the pattern and the another creates 5000 object using the pattern. After each of them create the objects, then they call a method that returns the current memory used by the App.
public class MemoryService : IMemoryService
{
private readonly TreeFactory _treeFactory;
public MemoryService()
{
_treeFactory = new TreeFactory();
}
//create without pattern
public long SetObjectsMemory()
{
List<Tree> trees = new List<Tree>();
for (int i = 0; i < 5000; i++)
{
var tree = new Tree()
{
Id = new Random().Next(1, 9999999),
Part = new PartTree()
{
Name = "Nameany",
Bark = "Barkany",
Color = "Colorany"
}
};
trees.Add(tree);
};
return Utilities.GetCurrentMemoryUsed();
}
//crete with flyweight pattern
public long SetObjectsMemoryFactory()
{
List<Tree> trees = new List<Tree>();
for (int i = 0; i < 5000; i++)
{
var tree = new Tree()
{
Id = new Random().Next(1, 9999999),
Part = _treeFactory.GetPartTree("Nameany", "Barkany", "Colorany")
};
trees.Add(tree);
}
return Utilities.GetCurrentMemoryUsed();
}
}
I use the pattern like a class that uses a list of Parts and return a part object if exists.
public class TreeFactory
{
private static List<PartTree> _parts;
public TreeFactory() {
_parts = new List<PartTree>();
}
public PartTree GetPartTree(string name, string bark, string color)
{
if (_parts.Any(x => x.Name == name && x.Bark == bark && x.Color == color))
{
return _parts.Where(x => x.Name == name && x.Bark == bark && x.Color == color).FirstOrDefault();
}
else {
var newpart = new PartTree()
{
Name = name,
Bark = bark,
Color = color
};
_parts.Add(newpart);
return newpart;
}
}
}
The way to get the current memory used by the App is using Process of this way (in Utilities class):
public static long GetCurrentMemoryUsed() {
Int64 memory;
using (Process proc = Process.GetCurrentProcess())
{
memory = proc.PrivateMemorySize64 / (1024 * 1024);
}
return memory;
}
And in my Startup i inject the MemoryService like a Singleton. In the controller i use 3 methods for call the functions:
[HttpGet, Route(nameof(WeatherForecastController.GenerateMemory))]
public IActionResult GenerateMemory()
{
var total=_memoryService.SetObjectsMemory();
return Ok(total);
}
[HttpGet, Route(nameof(WeatherForecastController.GenerateLiftMemory))]
public IActionResult GenerateLiftMemory()
{
var total = _memoryService.SetObjectsMemoryFactory();
return Ok(total);
}
[HttpGet, Route(nameof(WeatherForecastController.GetMemory))]
public IActionResult GetMemory()
{
var total = Utilities.GetCurrentMemoryUsed();
return Ok(total);
}
The problem is: When i call in the navigator the method in controller without pattern (/weatherforecast/GenerateMemory), then this returns (current)+2mb, but when i call the method
with pattern (/weatherforecast/GenerateLiftMemory) this returns (current)+3mb.
Why the method with pattern flyweight returns more used MB (growing) than the methods without the pattern ??
The repository with the code for test it. Gitlab repository memory api
The code which uses TreeFactory consumes more memory because its GetPartTree method called many times in a loop so as Linq methods Any and Where inside it. Both of these methods create additional Iterator objects under the hood in order to iterate through the collection and it causes additional memory consumption.
I wrote simple benchmark using BenchmarkDotNet with more options to demonstrate the issue
Extended MemoryService
public class MemoryService : IMemoryService
{
private const int TreeCount = 50000;
private readonly TreeFactory _treeFactory;
public MemoryService()
{
_treeFactory = new TreeFactory();
}
//crea objetos en memoria sin patrones
public decimal SetObjectsMemory()
{
List<Tree> trees = new List<Tree>();
for (int i = 0; i < TreeCount; i++)
{
var tree = new Tree()
{
Id = 1,
Part = new PartTree()
{
Name = "Nameany",
Bark = "Barkany",
Color = "Colorany"
}
};
trees.Add(tree);
};
return Utilities.GetCurrentMemoryUsed();
}
//crea objetos en memoria usando patron flyweight
public decimal SetObjectsMemoryFactory()
{
List<Tree> trees = new List<Tree>();
for (int i = 0; i < TreeCount; i++)
{
var tree = new Tree()
{
Id = 1,
Part = _treeFactory.GetPartTree("Nameany", "Barkany", "Colorany")
};
trees.Add(tree);
}
return Utilities.GetCurrentMemoryUsed();
}
public decimal SetObjectsMemoryFactoryImproved()
{
List<Tree> trees = new List<Tree>();
for (int i = 0; i < TreeCount; i++)
{
var tree = new Tree()
{
Id = 1,
Part = _treeFactory.GetPartTreeImproved("Nameany", "Barkany", "Colorany")
};
trees.Add(tree);
}
return Utilities.GetCurrentMemoryUsed();
}
//crea objetos en memoria usando patron flyweight
public decimal SetObjectsMemoryFactoryWithoutLambda()
{
List<Tree> trees = new List<Tree>();
for (int i = 0; i < TreeCount; i++)
{
var tree = new Tree()
{
Id = 1,
Part = _treeFactory.GetPartTreeWithoutLambda("Nameany", "Barkany", "Colorany")
};
trees.Add(tree);
}
return Utilities.GetCurrentMemoryUsed();
}
}
Extended TreeFactory
public class TreeFactory
{
private static List<PartTree> _parts;
public TreeFactory()
{
_parts = new List<PartTree>();
}
public PartTree GetPartTree(string name, string bark, string color)
{
if (_parts.Any(x => x.Name == name && x.Bark == bark && x.Color == color))
{
return _parts.Where(x => x.Name == name && x.Bark == bark && x.Color == color).FirstOrDefault();
}
var newpart = new PartTree()
{
Name = name,
Bark = bark,
Color = color
};
_parts.Add(newpart);
return newpart;
}
public PartTree GetPartTreeImproved(string name, string bark, string color)
{
var existingPart = _parts.Where(x => x.Name == name && x.Bark == bark && x.Color == color).FirstOrDefault();
if (existingPart != null)
return existingPart;
var newpart = new PartTree()
{
Name = name,
Bark = bark,
Color = color
};
_parts.Add(newpart);
return newpart;
}
public PartTree GetPartTreeWithoutLambda(string name, string bark, string color)
{
for (int i = 0; i < _parts.Count; i++)
{
var x = _parts[i];
if (x.Name == name && x.Bark == bark && x.Color == color)
return x;
}
var newpart = new PartTree()
{
Name = name,
Bark = bark,
Color = color
};
_parts.Add(newpart);
return newpart;
}
}
Benchmark in a separate console project
class Program
{
static void Main(string[] args)
{
var result = BenchmarkRunner.Run<MemoryBenchmark>();
}
}
[MemoryDiagnoser]
public class MemoryBenchmark
{
private IMemoryService memoryService;
[GlobalSetup]
public void Setup()
{
memoryService = new MemoryService();
}
[Benchmark]
public object SimpleTrees()
{
var trees = memoryService.SetObjectsMemory();
return trees;
}
[Benchmark]
public object FlyTrees()
{
var trees = memoryService.SetObjectsMemoryFactory();
return trees;
}
[Benchmark]
public object FlyTreesImproved()
{
var trees = memoryService.SetObjectsMemoryFactoryImproved();
return trees;
}
[Benchmark]
public object FlyTreesWithoutLambda()
{
var trees = memoryService.SetObjectsMemoryFactoryWithoutLambda();
return trees;
}
}
And its results
Method
Mean
Error
StdDev
Gen 0
Gen 1
Gen 2
Allocated
SimpleTrees
9.040 ms
0.1804 ms
0.2346 ms
718.7500
453.1250
265.6250
4.44 MB
FlyTrees
19.701 ms
0.1716 ms
0.1521 ms
2500.0000
906.2500
437.5000
15.88 MB
FlyTreesImproved
18.075 ms
0.2869 ms
0.2684 ms
1781.2500
625.0000
312.5000
10.92 MB
FlyTreesWithoutLambda
4.919 ms
0.0273 ms
0.0242 ms
421.8750
281.2500
281.2500
2.53 MB

How to create url with complex query

I use dart and flutter for mobile app. I use my api to get data from server. But I found a problem, maybe its dart core problem.
I need to add complex queryParams to my URL like
Map<String, Map<String, List<String>>>{"a": {"b": ["c","d"]}, "e": {}}
I use Uri.parse(url).replace(queryParams: myQueryParams).toString()
But Uri.replace() accepts only Map<String, Iterable<String>> and throws an error
Unhandled Exception: type '_InternalLinkedHashMap<String, List<String>>' is not a subtype of type 'Iterable<dynamic>'
I found method which throws this error
static String _makeQuery(String query, int start, int end,
Map<String, dynamic /*String|Iterable<String>*/ > queryParameters) {
if (query != null) {
if (queryParameters != null) {
throw ArgumentError('Both query and queryParameters specified');
}
return _normalizeOrSubstring(query, start, end, _queryCharTable,
escapeDelimiters: true);
}
if (queryParameters == null) return null;
var result = StringBuffer();
var separator = "";
void writeParameter(String key, String value) {
result.write(separator);
separator = "&";
result.write(Uri.encodeQueryComponent(key));
if (value != null && value.isNotEmpty) {
result.write("=");
result.write(Uri.encodeQueryComponent(value));
}
}
queryParameters.forEach((key, value) {
if (value == null || value is String) {
writeParameter(key, value);
} else {
Iterable values = value;
for (String value in values) {
writeParameter(key, value);
}
}
});
return result.toString();
}
So my question is there is some method in dart to add my queryParams to url or I need to create it by my own?
I have modified original method and now its work.
class UrlCreator {
static String addQueryParams(String url, Map<String, dynamic> queryParams) {
var result = StringBuffer();
var separator = "";
void writeParameter(String key, String value) {
result.write(separator);
separator = "&";
result.write(Uri.encodeQueryComponent(key));
if (value != null && value.isNotEmpty) {
result.write("=");
result.write(Uri.encodeQueryComponent(value));
}
}
void buildQuery(Map queryParams, {parentKey}){
queryParams.forEach((key, value){
print("parentKey = $parentKey Key = $key value = $value");
if (value == null || value is String) {
var newKey = parentKey != null ? "$parentKey[$key]" : key;
writeParameter(newKey, value);
} else if (value is Map) {
buildQuery(value, parentKey: key);
} else {
Iterable values = value;
var newKey = parentKey != null ? "$parentKey[$key][]" : "$key[]";
for (String value in values) {
writeParameter(newKey, value);
}
}
});
}
buildQuery(queryParams);
return url + "?" + result.toString();
}
}

Precedence and Associativity - Grammar Mistake using JavaCC

I'm having a problem with my grammar and I don't really know how to solve it. I'm facing the issue with precedence and associativity in operations. I included the whole grammar but I avoided to put all the Tokens otherwise it'd be too long.
PARSER_BEGIN(UcParse)
Node Start() :
{
Node tree = new Node(Id.PROGRAM);
Node td;
}
{
( td = TopLevelDeclaration() { tree.add(td); }
)*
<EOF> { return tree; }
}
Node TopLevelDeclaration() :
{
Node n;
}
{
LOOKAHEAD(3)
n = Declaration() <SEMI> { return n; }
| n = Function() { return n; }
| n = IncludeFile() {return n; }
}
Node Function() :
{
Node tld = new Node(Id.FUNC);
Node bt;
Node dr;
Node body;
Node formals;
Node s;
}
{
bt = ReturnType() { tld.add(bt); }
Declarator(tld)
(
FunctionParameters(tld)
(
body = CompoundStatement() { tld.add(body); }
|
<SEMI>
)
| { }
)
{ return tld; }
}
//List FunctionParameters () :
void FunctionParameters (Node func) :
{
Node f;
}
{
<LPAREN>
(
<VOID> { func.add(new Node(Id.VOID)); }
|
f = Declaration() { func.add(f); }
( <COMMA>
f = Declaration() { func.add(f); }
)*
)
<RPAREN>
}
Node Declaration () :
{
Node d = new Node(Id.VARDEC);
Node bt;
Node dr;
}
{
bt = DeclarationType() { d.add(bt); }
Declarator(d) { return d; }
}
Node SimpleDeclaration () :
{
Node d = new Node(Id.VARDEC);
Node bt;
Node id;
}
{
bt = DeclarationType() { d.add(bt); }
id = Identifier() { d.add(id); }
<SEMI>
{ return d; }
}
Node ReturnType () :
{}
{
<CHAR> { return new Node(Id.CHAR); }
| <INT> { return new Node(Id.INT); }
| <VOID> { return new Node(Id.VOID); }
}
Node DeclarationType () :
{}
{
<CHAR> { return new Node(Id.CHAR); }
|
<INT> { return new Node(Id.INT); }
}
void Declarator (Node p) :
{
Node id;
Node r;
}
{
id = Identifier() { p.add(id); }
( <LBRACK>
( r = IntegerLiteral() { p.add(r); } ) *
<RBRACK>
|
{ }
)
}
Node CompoundStatement () :
{
Node cs = new Node(Id.COMPOUND_STMNT);
Node d;
Node s;
}
{
<LBRACE>
( d = Declaration() { cs.add(d); }
<SEMI>
)*
( s = Statement() { cs.add(s); }
)*
<RBRACE>
{ return cs; }
}
Node Statement() :
{
Node stmt = new Node(Id.STMNT);
Node s;
Token t;
Node c;
Node s1;
Node s2;
}
{
(s = SimpleCompoundStatement() { stmt.add(s); }
|
Expression(stmt)
<SEMI> // expr;
|
<SEMI> { stmt.add(new Node(Id.EMPTY_STMNT)); } //;
|
t = { s = new Node(Id.IF); stmt.add(s); }
<LPAREN>
Expression(s)
<RPAREN>
s1 = Statement() { s.add(s1); }
( LOOKAHEAD(1)
<ELSE>
s2 = Statement() { s.add(s2); } ) *
|
t = <WHILE> { s = new Node(Id.WHILE); stmt.add(s); }
<LPAREN>
Expression(s)
<RPAREN>
s2 = Statement() { s.add(s2); }
|
(Expression(stmt)) *
<SEMI>) //return expr*;
{ return stmt; }
}
Node SimpleCompoundStatement() :
{
Node scs = new Node(Id.SIMPLE_COMPOUND_STMNT);
Token left;
Token right;
Node s;
}
{
left = <LBRACE>
( s = Statement() { scs.add(s); }
)*
right = <RBRACE>
{ return scs; }
}
void Expression (Node e) :
{
Node exp;
Node id;
Node p;
Node op;
}
{
(
op = IntegerLiteral()
OperatorExpression(e, op)
|
exp = CharLiteral()
OperatorExpression(e, exp)
|
<LPAREN> { p = new Node(Id.PAREN); }
Expression(p)
<RPAREN>
OperatorExpression(e, p)
| id = Identifier()
( <LBRACK>
Expression(e)
<RBRACK>
| <LPAREN>
(
Expression(e)
( <COMMA>
Expression(e)
)*
| { } )
<RPAREN>
| { }
)
OperatorExpression(e, id)
|
exp = Unary()
Expression(e)
OperatorExpression(e, exp)
)
}
void OperatorExpression(Node par, Node op) :
{
Node n;
Node p;
}
{
( LOOKAHEAD(2)
n = BinaryMulDiv() { par.add(n); }
{ n.add(op); }
Expression(n)
| OperatorExpressionPlusMin(par, op) )
}
void OperatorExpressionPlusMin(Node par, Node op) :
{
Node n;
}
{
( LOOKAHEAD(2)
n = BinaryPlusMin() { par.add(n); }
{ n.add(op); }
Expression(n)
| OperatorExpressionComp(par, op))
}
void OperatorExpressionComp(Node par, Node op) :
{
Node n;
}
{
( LOOKAHEAD(2)
n = BinaryComp() { par.add(n); }
{ n.add(op); }
Expression(par)
| {} {par.add(op);} )
}
Node BinaryComp () :
{
Token t;
}
{
(t = <LT> // >
|
t = <GT> // // // >=
|
t = <EQ> // =
|
t = <EQEQ> // ==
|
t = <NOTEQ> // !=
|
t = <ANDAND> // &&
|
t = <OROR>) // ||
{return new Node(Id.BINARY, t.image); }
}
Node BinaryMulDiv () :
{
Token t;
}
{
(t = <MUL> // *
|
t = <DIV>) // /
{return new Node(Id.BINARY, t.image); }
}
Node BinaryPlusMin () :
{
Token t;
}
{
(t = <PLUS> // +
|
t = <MINUS>) // -
{return new Node(Id.BINARY, t.image); }
}
Node Unary() :
{
Token t;
}
{
t = <MINUS> { return new Node(Id.UNARY, t.image); }
|
t = <NOT> { return new Node(Id.UNARY, t.image); } // !
}
Node Identifier() :
{
Token t;
}
{
t = <IDENT> { return new Node(Id.IDENT, t.image); }
}
Node IntegerLiteral() :
{
Token t;
}
{
t = <INTEGER_LITERAL>
{ return new Node(Id.INTEGER_LITERAL, t.image); }
}
Node CharLiteral() :
{
Token t;
}
{
t = <CHAR_LITERAL>
{ return new Node(Id.CHAR_LITERAL, t.image); }
}
Node FileName() :
{
Token dot;
}
{
(<IDENT> <DOT> <IDENT>)
{ return new Node(Id.FILE_NAME); }
}
Node IncludeFile() :
{
Node include = new Node(Id.INCLUDE);
Node name;
Token incl;
Token lt;
Token gt;
}
{
incl = <INCLUDE>
lt = <LT>
name = FileName() { include.add(name); }
gt = <GT>
{ return include; }
}
This is the program I'm using for my tests.
int main(void) {
int i;
1!=!3;
4&&(6);
7* 8+10; // wrong tree
10+8*7; // right tree
(11-12)+(12/16);
17=27>28;
}
I guess that the part that doesn't work in my grammar is the OperatorExpression() because when I print the syntax tree for the code above, I obtain two different branches for those line I put a comment in.
Here the syntax tree
PROGRAM
FUNC
INT
IDENT ( main )
VOID
COMPOUND_STMNT
VARDEC
INT
IDENT ( i )
STMNT
BINARY ( != )
INTEGER_LITERAL ( 1 )
INTEGER_LITERAL ( 3 )
UNARY ( ! )
STMNT
BINARY ( && )
INTEGER_LITERAL ( 4 )
PAREN
INTEGER_LITERAL ( 6 )
STMNT
BINARY ( * )
INTEGER_LITERAL ( 7 )
BINARY ( + )
INTEGER_LITERAL ( 8 )
INTEGER_LITERAL ( 10 )
STMNT
BINARY ( + )
INTEGER_LITERAL ( 10 )
BINARY ( * )
INTEGER_LITERAL ( 8 )
INTEGER_LITERAL ( 7 )
STMNT
BINARY ( + )
PAREN
BINARY ( - )
INTEGER_LITERAL ( 11 )
INTEGER_LITERAL ( 12 )
PAREN
BINARY ( / )
INTEGER_LITERAL ( 12 )
INTEGER_LITERAL ( 16 )
STMNT
BINARY ( = )
INTEGER_LITERAL ( 25 )
BINARY ( > )
INTEGER_LITERAL ( 27 )
INTEGER_LITERAL ( 28 )
Any help is really appreciate! Thanks
There is a write up called Parsing Expressions by Recursive Descent, which outlines 3 approaches to getting precedence and associativity right in recursive descent parsers. All three techniques can be applied to JavaCC parsers. The second one (the "classic" algorithm) is probably simplest to use.

How to catch list of tokens in tree grammar of antlr3?

I took a dummy language for example:
It simply accepts one or more '!'.
its lexer and grammar rules are:
grammar Ns;
options {
output=AST;
ASTLabelType=CommonTree;
}
tokens {
NOTS;
}
#header {
package test;
}
#lexer::header {
package test;
}
ns : NOT+ EOF -> ^(NOTS NOT+);
NOT : '!';
ok, as you can see, this represents a language which accept '!' or '!!!' or '!!!!!'...
and I defined some meaningful classes to build ASTs:
public class Not {
public static final Not SINGLETON = new Not();
private Not() {
}
}
public class Ns {
private List<Not> nots;
public Ns(String nots) {
this.nots = new ArrayList<Not>();
for (int i = 0; i < nots.length(); i++) {
this.nots.add(Not.SINGLETON);
}
}
public String toString() {
String ret = "";
for (int i = 0; i < this.nots.size(); i++) {
ret += "!";
}
return ret;
}
}
and here's the tree grammar:
tree grammar NsTreeWalker;
options {
output = AST;
tokenVocab = Ns;
ASTLabelType = CommonTree;
}
#header {
package test;
}
ns returns [Ns ret] : ^(NOTS n=NOT+) {$ret = new Ns($n.text);};
and the main class code with some sample data to test the generated classes:
public class Test {
public static void main(String[] args) throws Exception {
ANTLRInputStream input = new ANTLRInputStream(new ByteArrayInputStream("!!!".getBytes("utf-8")));
NsLexer lexer = new NsLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);
NsParser parser = new NsParser(tokens);
CommonTree root = (CommonTree) parser.ns().getTree();
NsTreeWalker walker = new NsTreeWalker(new CommonTreeNodeStream(root));
try {
NsTreeWalker.ns_return r = walker.ns();
System.out.println(r.ret);
} catch (RecognitionException e) {
e.printStackTrace();
}
}
}
but the final output printed is '!', other than the expecting '!!!'.
that's mainly because this line of code :
ns returns [Ns ret] : ^(NOTS n=NOT+) {$ret = new Ns($n.text);};
the $n above captured only one '!', I don't know how to capture all three tokens of '!', in other words , a list of '!' with $n.
Is there some one could help?thanks!
The fact that only one ! gets printed is because your rule:
ns returns [Ns ret]
: ^(NOTS n=NOT+) {$ret = new Ns($n.text);}
;
gets more or less translated as:
Token n = null
LOOP
n = match NOT_token
END
return new Ns(n.text)
Therefor, n.text will always be just a single !.
What you need to do is collect these NOT tokens in a list. In ANTLR you can create a list of tokens using the += operator instead of the "single token" operator =. So change your ns rule into:
ns returns [Ns ret]
: ^(NOTS n+=NOT+) {$ret = new Ns($n);}
;
which gets translated as:
List n = null
LOOP
n.add(match NOT_token)
END
return new Ns(n)
Be sure to change the constructor of your Ns class to take a List instead:
public Ns(List nots) {
this.nots = new ArrayList<Not>();
for (Object o : nots) {
this.nots.add(Not.SINGLETON);
}
}
after which the output of your test class would be:
!!!
Good luck!