I am parsing Java with Antlr, using ready Java9 grammar from antlr-grammars repo.
In the Antlr contexts I often see methods with "lfno" or "lf" suffics in the name, for instance:
classInstanceCreationExpression_lfno_primary
arrayAccess_lfno_primary,
or even like this - PrimaryNoNewArray_lfno_primary_lf_arrayAccess_lfno_primaryContext
I wonder what does that mean, because I coundn't find any information on that
Any ideas would be appreciated
Related
I am using lhs2TeX for my literate Agda files and I'd like them to be syntax highlighted. I know I can achieve some highlighting via %format instructions but that is a bit too much. I have tried using lhs2Tex-hl as instructed at http://foswiki.cs.uu.nl/foswiki/pub/FPOld/CourseLiterature/presentation.pdf but to no avail.
If anyone can help with getting PDF's with coloured Agda, that'd be appreciated!
Edit
The Agda latex backend is great but has some issues. Biggest issue is spec-environment. It says that spec is undefined, so making it a synonym for code should fix things: \newenvironment{spec}{\begin{code}}{\end{code}}, but instead now there are type-checking issues and missing $ errors. Using sed to remove spec blocks altogether results in missing $ errors; likewise \newenvironment{spec}{\verbatim}{\endverbatim}. Similar trouble with using |...|. Also, I honestly liked the %format of lhs2tex ...
I am using ANTLR4 to parse code in my Netbeans Platform application. I have successfully implemented syntax highlighting using ANTLR4 and Netbeans mechanisms.
I have also implemented a simple code completion for two of my tokens. At the moment I am using a simple implementation from a tutorial, which searches for a whitespace and starts the completion process from there. This works, but it deems the user to prefix a whitespace before starting code completion.
My question: is it possible or even contemplated using ANTLR's lexer to determine which tokens are currently read from the input to determine the correct completion item?
I would appreciate every pointer in the right direction to improve this behaviour.
not really an answer, but I do not have enough reputation points to post comments.
is it possible or even contemplated using ANTLR's lexer to determine which tokens are currently read from the input to determine the correct completion item?
Have a look here: http://www.antlr3.org/pipermail/antlr-interest/2008-November/031576.html
and here: https://groups.google.com/forum/#!topic/antlr-discussion/DbJ-2qBmNk0
Bear in mind that first post was written in 2008 and current antlr v4 is very different from the one available at the time, which is why Sam’s opinion on this topic appear to have evolved.
My personal experience - most of what you are asking is probably doable with antlr, but you would have to know antlr very well. A more straightforward option is to use antlr to gather information about the context and use your own heuristics to decide what needs to be shown in this context.
The ANTLRv3 grammar https://sourceware.org/git/?p=frysk.git;a=blob_plain;f=frysk-core/frysk/expr/CExpr.g;hb=HEAD implements context sensitive completion of C expressions (no macros).
For instance, if fed the string:
a_struct->a<tab>
it would just lists the fields of "a_struct" starting with "a" (tab could, technically be any character or marker).
The technique it used was to:
modify a C grammar to recognize both IDENT and IDENT_TAB tokens
for IDENT_TAB capture the partial expression AST and "TOKEN_TAB" and throw them back to 'main' (there are hacks to help capture the AST)
'main' then performs a type-eval on the partial expression (compute the expression's type not value) and use that to expand TOKEN_TAB
the same technique, while not exactly ideal, can certainly be used in ANTLRv4.
I don't know, if this question is valid since i'm not very familiar with source code parsing. My goal is to write a source code completion function for one existing programming language (Language "X") for learning purposes.
Is Antlr(v4) suitable for such a task or should the necessary AST/Parse Tree creation and parsing be done by hand, assuming no existing solutions exists?
I haven't found much information about that specific topic, except a list of compiler books, except a compiler is not what i'm after for.
The code completion in GoWorks is completely implemented using ANTLR 4. The following video shows the level of completion of this code completion engine. The code completion example runs from 5 minutes through the end of the video.
Intro to Tunnel Vision Labs' GoWorks IDE (Preview Release)
I have been working on code completion algorithms for many years, and strongly believe that there is no better solution (automated or manual) for producing a code completion solution for a new language that meets the requirements for what I would call highly-responsive code completion. If you are not interested in that level of performance or accuracy, other solutions may be easier for you to get involved with (I don't work with those personally, because I am too easily disappointed in the results).
Xtext uses ANTLR3 and has good autocomplete facilities. The problem is, it generates a seperate parser (again using antlr3) for autocomplete processing which is derived from AbstractInternalContentAssistParser. This multi-thousand line code part shows that the error recovery of ANTLR3 alone found to be insufficient by the xtext team.
Meanwhile ANTLR4 has a function parser.getExpectedTokensWithinCurrentRule() which lists possible token types for given position. It works when used in a ParseTreeListener. Remaining is semantics, scoping etc which is out of ANTLRs scope.
...is there anything I could do about it?
To be more precise, I would like to replace the caret "^" with something like "§" - granted, there's not much left on the keyboard that's not in use already.
After thinking about it for a while (dismissed using run script build phases along the way) I think the only way to do it would be a custom llvm build.
While I don't quite think I'm ready to deal with the internals of compilers, I have the naive hope that replacing one symbol with another isn't too hard. And the idea of building and running my own version of a compiler tickles me, be it just for a good deal of childish fun.
So I started poking around in the llvm sources, but - surprise - got nowhere so far.
If someone is familiar with these kind of things, could you please point me to a place to look at?
That would be awesome! Thanks!
Extending LLVM can be a bit of a hassle, especially considering how fast-moving the compiler team is, so it's a good thing you don't have to. The C preprocessor exists to perform the exact same thing you've outlined (text replacement). I'm fairly sure § isn't aliased to anything important, so #define § ^ should work great. If you still want to write your own module, LLVM provides instructions on how to extend their compiler.
Actually the code relevant for such a change isn't a part of LLVM at all, but a part of its Objective-C frontend, called Clang. Confusingly, "Clang" is also the name of the entire C/C++/ObjC compiler based on both Clang and LLVM.
While I don't quite think I'm ready to deal with the internals of compilers, I have the naive hope that replacing one symbol with another isn't too hard.
And you'll be right. What you're trying to do is very simple change.
In fact, if ^ was only used for blocks, it would be a trivial change - just modify the lexer to generate the "caret" token from § instead of ^: take a look at the lexer code to see what I mean (search for ^).
Unfortunately it's used for xor as well, so we'll have to modify both the lexer and the parser. The lexer to add a new token type and create that token from §, the parser to actually do something with it, e.g. by adding:
case tok::section: // 'section' is the token type you've added
Res = ParseBlockLiteralExpression();
break;
(and then fixing the assert at the beginning of ParseBlockLiteralExpression()).
You might run into some issues, though, as § isn't in ASCII - though as far as I know Clang should be able to deal with UTF-8 encoded files.
Greetings
I'm working on a project that requires me to write queries in text form, then convert them to some easily processed nodes to be processed by some abiguous repository. Of everything there, the part I'm least interested is the part that converts the text to nodes. I'm hoping it's already done somewhere.
Because I'm making stuff up as I go, I chose to use a LINQish expression syntax.
from m in Movie select m.A, m.B
I started parsing it manually and got the basics, but it's pretty cheesy. I'm looking for the better solution. I made some progress using MGrammar, but it would be nice if such a thing already existed. Does anyone know of anything that already does this? I looked for existing ANTLR templates, but no luck.
Thanks for the help.
You could start with a full C# grammar and throw away everything but the LINQ syntax :-}
The DMS Software Reengineering Toolkit is a tool for building parsers/program analyzers/transformers that has a full C# 4.0 front end, including all the LINQ syntax.
Try this example from the Pyparsing wiki Examples page. It should give you a start.