I would like to parse a file where the first line may or may not contain a definition of a "project" name (like with Pascal's program keyword), and if not, use the name of the file that is being parsed as default. Simplified:
#members{ String projectName; }
project : {projectName = ...} // name of parsed file as default
( PROJECTNAMEKEYWORD ID {projectName = $ID.text;} )?
otherstuff {/*...*/};
Is this even possible? I found this surprisingly hard to find out using google or the antlr manual. By its documentation, input.getSourceName() should be what I am looking for, but it always returns null, and debugging lead me to the class ANTLRStringStream that has a name field whose value is returned by this method but never set.
Simply create a custom constructor in your parser that takes a string that represents your file.
A demo:
grammar T;
#parser::members {
private String projectName;
public TParser(String fileName) throws java.io.IOException {
super(new CommonTokenStream(new TLexer(new ANTLRFileStream(fileName))));
projectName = fileName;
}
public String getProjectName() {
return projectName;
}
}
parse
: (PROJECT ID {projectName = $ID.text;})? EOF
;
PROJECT : 'project';
ID : ('a'..'z' | 'A'..'Z')+;
SPACE : (' ' | '\t' | '\r' | '\n') {skip();};
which can be tested with:
import org.antlr.runtime.*;
public class Main {
public static void main(String[] args) throws Exception {
// 'empty.txt' is, as the name suggests, an empty file
TParser parser = new TParser("empty.txt");
parser.parse();
System.out.println(parser.getProjectName());
// 'test.txt' contains a single line with the words: 'project JustATest'
parser = new TParser("test.txt");
parser.parse();
System.out.println(parser.getProjectName());
}
}
If you run the Main class, the following will be printed to your console:
empty.txt
JustATest
Related
I have a sample model i am using as below
grammar org.xtext.example.testdsl.TestDsl with org.eclipse.xtext.common.Terminals
generate testDsl "http://www.xtext.com/test/example/TestDsl"
Model:
prog+=Program*;
Program: g=Greeting de+=DataEntry* s+=Statement*;
Greeting: 'Hello' t=ProgPara '!';
ProgPara: 'PROGRAM' pname=Progname ';';
Progname : name=ID;
DataEntry: a=INT (v=Varname| in=Indexname) ';';
Varname : name = ID;
Statement: c=CopyStmt ';';
CopyStmt: 'COPY' 'TO' qname=[IndexVarname|ID] ;
IndexVarname : (Indexname|Varname);
Indexname : '(' name = ID ')';
Named:Progname|Indexname|Varname;
I have added caching support for Scoping like in the code below:
class TestDslScopeProvider extends AbstractTestDslScopeProvider {
#Inject
IResourceScopeCache cache;
override getScope(EObject context, EReference reference) {
if (context instanceof CopyStmt) {
if (reference.featureID == TestDslPackage.COPY_STMT__QNAME) {
val candidates = cache.get(
"COPY_STMT__QNAME_scope",
reference.eResource,
[|findQNameCandidates(context, reference)]
);
return Scopes.scopeFor(candidates);
}
}
return super.getScope(context, reference);
}
def findQNameCandidates(EObject context, EReference reference) {
val rootElement = EcoreUtil2.getRootContainer(context);
val candidates1 = EcoreUtil2.getAllContentsOfType(rootElement, IndexVarname);
return candidates1;
}
}
Now i have a sample test case as below:
Hello PROGRAM test;!
1 test1;
2 (test4);
3 test3;
COPY TO test4;
COPY TO test4;
When i try to rename test4 using the Rename Element, only the variable in the definition is getting renamed. The references are not getting renamed. Without caching it works fine, but when caching is done, i hit this issue. What am i missing here?
Thanks,
Anitha
you do cache wrong
reference.eResource
you need to use the context resource as cache not the metamodels
context.eResource,
I try to test a case that has to random a string to input in field by using xebium with fitnesse.
I try to use below command but it doesn't work.
| $fname= | is | storeValue | on | var chars = 'ABCDEFGHIJKLMNOPQRSTUVWXTZ'; var fname = ''; for (var i=0; i<6; i++)var rnum = Math.floor(Math.random()*chars.length); fname += chars.substring(rnum,rnum+1); |
Thank you for attention :)
We are using special type parser and template strings for this purpose.
For example, function:
bool LastLogonTimeLessThan( DateTimeWrapper time)
{
return time.Value < GetLastAccessTime();
}
Then you can add custom parser for this type, see tutorial here: https://github.com/imanushin/NetRunner/wiki/Parsing
Parser can be:
public static void DateTimeWrapper(string inputLine)
{
if("{today}".Equals(inputLine)
return new DateTimeWrapper(DateTime.Now)
return new DateTimeWrapper(DateTime.Parse(inputLine))
}
I have a "statement" definition from the Java language definition as follows.
statement
: block
| ASSERT expression (':' expression)? ';'
| 'if' parExpression statement ('else' statement)?
| 'for' '(' forControl ')' statement
| 'while' parExpression statement
| 'do' statement 'while' parExpression ';'
| 'try' block
( catches 'finally' block
| catches
| 'finally' block
)
| 'switch' parExpression switchBlock
| 'synchronized' parExpression block
| 'return' expression? ';'
| 'throw' expression ';'
| 'break' Identifier? ';'
| 'continue' Identifier? ';'
| ';'
| statementExpression ';'
| Identifier ':' statement
;
When doing the parser, i want to print the full user-written statements also (inculding the spaces in the statements), such as:
Object o = Ma.addToObj(r1);
if(h.isFull() && !h.contains(true)) h.update(o);
But when i use the function "getText()" in "exitStatement", i can only get the statements with all the spaces been deleted, such as:
Objecto=Ma.addToObj(r1);
if(h.isFull()&&!h.contains(true))h.update(o);
How can i get the full user-written statements (inculding the spaces in the statements) in a easy way? Thanks a lot!
The full codes as follows:
public class PrintStatements {
public static class GetStatements extends sdlParserBaseListener {
StringBuilder statements = new StringBuilder();
public void exitStatement(sdlParserParser.StatementContext ctx){
statements.append(ctx.getText());
statements.append("\n");
}
}
public static void main(String[] args) throws Exception{
String inputFile = null;
if ( args.length>0 ) inputFile = args[0];
InputStream is = System.in;
if ( inputFile!=null ) {
is = new FileInputStream(inputFile);
}
ANTLRInputStream input = new ANTLRInputStream(is);
sdlParserLexer lexer = new sdlParserLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);
sdlParserParser parser = new sdlParserParser(tokens);
ParseTree tree = parser.s();
// create a standard ANTLR parse tree walker
ParseTreeWalker walker = new ParseTreeWalker();
// create listener then feed to walker
GetStatements loader = new GetStatements();
walker.walk(loader, tree); // walk parse tree
System.out.println(loader.statements.toString());
}
}
I've solved this problem by using tokens.getText() in the upper level of the statement, like this:
public void exitE(sdlParserParser.EContext ctx) {
TokenStream tokens = parser.getTokenStream();
String Stmt = null;
Stmt = tokens.getText(ctx.statement());
...
}
I'm pretty new with ANTLR, so maybe i'm wrong with something...
I don't know easy way to do this but you can try something like this.
In your grammar file you probably have something like this:
WS : (' '|'\r'|'\t'|'\u000C'|'\n')
{
if (!preserveWhitespacesAndComments) {
skip();
} else {
$channel = HIDDEN;
}
}
This lexer rule tells parser to ignore whitespaces. More exactly this tokens are sent on HIDDEN channel (parser don't see them). If you comment this lines of code
WS : (' '|'\r'|'\t'|'\u000C'|'\n')
{
if (!preserveWhitespacesAndComments) {
// skip();
} else {
// $channel = HIDDEN;
}
}
all whitespaces will be sent to parser but then you need to rewrite parser rules so he can expect whitespaces at any place.
Object(EXPECT WHITESPACE)o(EXPECT WHITESPACE)=(EXPECT WHITESPACE)Ma.addToObj(r1);
Otherwise parser will report errors.
You need one of two things:
The ability to take file position data for the first and last tokens accepted by a statement parse (either the lexemes or the tree nodes should do), and go to the source file, and extract the text. That will get you the original whitespace.
A prettyprinter, which will regenerate text from the AST, inserting appropriate whitespacing. See my SO answer on how to build a prettyprinter here.
in terms of Antlr4 and Python3, the code looks as follows:
def exitSomeDecl(self, ctx: yourParser.SomeDeclContext):
start_index = ctx.start.tokenIndex
stop_index = ctx.stop.tokenIndex
user_text = self.token_stream.getText(interval=(start_index, stop_index))
here, the self.token_stream: CommonTokenStream is assigned during init:
input_stream = FileStream(file_name)
lexer = sdplLexer(input_stream)
token_stream = CommonTokenStream(lexer)
I'm using StringTemplate 4 to generate some Java source files.
The templates are really simple, e.g.:
QueryHandler(method) ::="
public class Obtenir<method.name>Handler extends QueryHandler\<List\<<method.name>Db>> implements IQueryHandler\<List\<<method.name>>>{
private IQuery\<List\<<method.name>Db>> query;
private <method.name>Converter converter;
#Inject
public Obtenir<method.name>Handler(IQuery\<List\<<method.name>Db>> query, <method.name>Converter converter, IStatisticsCollecter theStatsCollecter){
super(theStatsCollecter);
if(query == null){
throw new IllegalArgumentException(\"The query argument cannot be null\");
}
if(converter == null){
throw new IllegalArgumentException(\"Illegal argument for converter(null)\");
}
this.query = query;
this.converter = converter;
}
public List\<<method.name>> handle(Object... params) throws JdbcException {
final String method = \"obtenir<method.name>\";
DaoQueryStatusCallable status = new DaoQueryStatusCallable();
List\<<method.name>Db> result = invoke(query, status, method);
return converter.convert(result);
}
}
"
The code is even simpler:
STGroup group = new STGroupFile("src/main/resources/QueryHandler.stg");
ST wsTemplate = group.getInstanceOf("QueryHandler");
wsTemplate.add("method", m);
System.out.println(wsTemplate.render());
The template lines are separated by Unix EOLs (\n).
When I execute the code, StringTemplate is emitting a warning "QueryHandler.stg 1:25: \n in string".
The result is correct, but I'd still like to get rid of this message.
Anybody ever had this problem and knows how to solve it?
t() ::= "..." is meant only for single lines. Please use
t() ::= <<
...
>>
to get multi-line templates.
Ter
heres my test in java
public class person
{
public String name;
public int age;
public String getName() { return name; }
public int getAge() { return age; }
}
In my function I create a number of person objects
and add it into an list
ArrayList<person> arr = new ArrayList<person>()
arr.add(person1);
arr.add(person2); etc etc
in the string template group file I have got
test(arr) ::= <<
<table>
$arr: {a|
<tr><td>$a.name$</td><td>$a.age$</td></tr>
}$
</table>
>>
this is called from my list template
list (arr) ::= <<
$test(arr)$
... and other page details etc
>>
in version4 for I get template not found message with some stack trace as follows
Caused by: java.lang.ClassCastException: java.util.ArrayList
at org.antlr.runtime.tree.RewriteRuleTokenStream.nextNode(RewriteRuleTokenStream.java:58)
at org.stringtemplate.v4.compiler.STParser.subtemplate(STParser.java:1563)
at org.stringtemplate.v4.compiler.STParser.mapTemplateRef(STParser.java:3692)
but I can list an array as follows and it prints the object representation in string format
(also I can use a map - ie key value pairs works ok too)
in string template
test(arr) :: =<<
<p>
$arr; seperator="</br>"$
</p>
>>
how do we iterate list of object to print field values using string template grop in V4
am I using it in a wrong format/syntax?
any help/points would be greateful
note: we set $ as delimiter using new STGroupString("", templateGroup, '$', '$');
You are missing the terminating $ on your expression: $a.age should be $a.age$