Greetings
I'm working on a project that requires me to write queries in text form, then convert them to some easily processed nodes to be processed by some abiguous repository. Of everything there, the part I'm least interested is the part that converts the text to nodes. I'm hoping it's already done somewhere.
Because I'm making stuff up as I go, I chose to use a LINQish expression syntax.
from m in Movie select m.A, m.B
I started parsing it manually and got the basics, but it's pretty cheesy. I'm looking for the better solution. I made some progress using MGrammar, but it would be nice if such a thing already existed. Does anyone know of anything that already does this? I looked for existing ANTLR templates, but no luck.
Thanks for the help.
You could start with a full C# grammar and throw away everything but the LINQ syntax :-}
The DMS Software Reengineering Toolkit is a tool for building parsers/program analyzers/transformers that has a full C# 4.0 front end, including all the LINQ syntax.
Try this example from the Pyparsing wiki Examples page. It should give you a start.
Related
👋
I'm parsing AST with typescript-estree's parse function and traversing the tree with simpleTraverse to identify and collect metadata about different nodes. I read in this issue discussion that simpleTraverse is intentionally not documented as it's only intended for usage by other ESlint packages, which makes sense.
My question is whether it's fine to use as is, or whether there's another tool I should be using. I have tried other common traversal tools, but the types are incomplete and / or conflict with what's provided by #typescript-eslint/typescript-estree. Another option is to copy the simpleTraverse code.
Thanks in advance! I really appreciate this project and the community supporting it. ❤️
In a customer object, we found an SWC statement that our parser chokes on.
IF NOT ( pyparaid IS INITIAL OR dataset_exp IS INITIAL ).
swc_set_element container 'DATASET' dataset_exp+10.
ENDIF.
Although this page seems to imply that they are well known in the ABAP world, I cannot find a page where they are documented officially. (Similar to the ABAP keyword documentation).
Are these macros considered part of the language? In other words, if they are not covered, would you consider a parser incomplete? Please point me to their documentation.
Please try searching for yourself next time. The first hit when googling for "site:help.sap.com swc_set_element" would have lead you straight to the reference.
Yes, a parser that is unable to process macros is incomplete. You have been warned about that half a year ago... :-)
I am using ANTLR4 to parse code in my Netbeans Platform application. I have successfully implemented syntax highlighting using ANTLR4 and Netbeans mechanisms.
I have also implemented a simple code completion for two of my tokens. At the moment I am using a simple implementation from a tutorial, which searches for a whitespace and starts the completion process from there. This works, but it deems the user to prefix a whitespace before starting code completion.
My question: is it possible or even contemplated using ANTLR's lexer to determine which tokens are currently read from the input to determine the correct completion item?
I would appreciate every pointer in the right direction to improve this behaviour.
not really an answer, but I do not have enough reputation points to post comments.
is it possible or even contemplated using ANTLR's lexer to determine which tokens are currently read from the input to determine the correct completion item?
Have a look here: http://www.antlr3.org/pipermail/antlr-interest/2008-November/031576.html
and here: https://groups.google.com/forum/#!topic/antlr-discussion/DbJ-2qBmNk0
Bear in mind that first post was written in 2008 and current antlr v4 is very different from the one available at the time, which is why Sam’s opinion on this topic appear to have evolved.
My personal experience - most of what you are asking is probably doable with antlr, but you would have to know antlr very well. A more straightforward option is to use antlr to gather information about the context and use your own heuristics to decide what needs to be shown in this context.
The ANTLRv3 grammar https://sourceware.org/git/?p=frysk.git;a=blob_plain;f=frysk-core/frysk/expr/CExpr.g;hb=HEAD implements context sensitive completion of C expressions (no macros).
For instance, if fed the string:
a_struct->a<tab>
it would just lists the fields of "a_struct" starting with "a" (tab could, technically be any character or marker).
The technique it used was to:
modify a C grammar to recognize both IDENT and IDENT_TAB tokens
for IDENT_TAB capture the partial expression AST and "TOKEN_TAB" and throw them back to 'main' (there are hacks to help capture the AST)
'main' then performs a type-eval on the partial expression (compute the expression's type not value) and use that to expand TOKEN_TAB
the same technique, while not exactly ideal, can certainly be used in ANTLRv4.
I don't know, if this question is valid since i'm not very familiar with source code parsing. My goal is to write a source code completion function for one existing programming language (Language "X") for learning purposes.
Is Antlr(v4) suitable for such a task or should the necessary AST/Parse Tree creation and parsing be done by hand, assuming no existing solutions exists?
I haven't found much information about that specific topic, except a list of compiler books, except a compiler is not what i'm after for.
The code completion in GoWorks is completely implemented using ANTLR 4. The following video shows the level of completion of this code completion engine. The code completion example runs from 5 minutes through the end of the video.
Intro to Tunnel Vision Labs' GoWorks IDE (Preview Release)
I have been working on code completion algorithms for many years, and strongly believe that there is no better solution (automated or manual) for producing a code completion solution for a new language that meets the requirements for what I would call highly-responsive code completion. If you are not interested in that level of performance or accuracy, other solutions may be easier for you to get involved with (I don't work with those personally, because I am too easily disappointed in the results).
Xtext uses ANTLR3 and has good autocomplete facilities. The problem is, it generates a seperate parser (again using antlr3) for autocomplete processing which is derived from AbstractInternalContentAssistParser. This multi-thousand line code part shows that the error recovery of ANTLR3 alone found to be insufficient by the xtext team.
Meanwhile ANTLR4 has a function parser.getExpectedTokensWithinCurrentRule() which lists possible token types for given position. It works when used in a ParseTreeListener. Remaining is semantics, scoping etc which is out of ANTLRs scope.
There is the simple interpretive programming language and, actually, console interpreter.exe.
Need to make colorizing of syntax, autocomplete and executing by press F5.
(if it is possible to make 'debug' - that will be awesome!)
I never did such things.
There are many IDE, which allow to add lang.: eclipse, NetBeans, emacs, ...
But I did not found complete instruction to add or they are ununderstandable.
What IDE is best to use? to add lang. as easy as possible?
(it will be cool, if IDE can work in Windows)
How to add my language there?
Please, if it is possible to give complete instruction.
Depending on how far you really want to go there are multiple options:
Dumb Autocompletion for text editors:
There are editors like scite aka Notepad++, that take a simple textfile with all the keywords to give you autocompletion, but they don't take into account the syntax nor the context. All they do is to highlight the words they know (e.g. you have given to them) and to autocomplete just these terms.
Smarter Syntax Highlighting:
This would require you to get used to the tools lex and yacc, if we are talking open source. I don't know which proprietary source tools are out there. If you want to get into that, there are several good pages on that topic, and this is one of them.
Compile it all the time:
A simple but effective method for small projects would be just to compile it once every few seconds, and interpret the output. This would be the messy version, but might be fun to look into.
The documentation for adding a new editor to Eclipse looks fairly straightforward:
http://wiki.eclipse.org/FAQ_How_do_I_write_an_editor_for_my_own_language%3F
This covers syntax color highlighting and autocomplete. I imagine you can also create a launch profile in the same plugin