I am currently debugging an ABNF grammar. It is currently very big. I am finding it difficult to debug. An IDE or syntax highlighter would be very helpful.
Is there any recommended ABNF grammar IDE or plugin available? Not the BNF ides and plugins. My online research gave me only the BNF plugins which don't recognise ABNF syntax like the slant operator.
I have the same need and have not found an answer. However, I find Lark (https://github.com/lark-parser/lark) very easy to use.
As a workaround, I am developing in Lark's EBNF-derived grammar language. Once the parser is working, hand-translating the Lark grammar to ABNF is relatively straightforward. Unfortunately even ABNF parsers are in short supply, so it's not possible to check the translation for functional correctness. I settle for syntactic correctness using https://tools.ietf.org/tools/bap/abnf.cgi.
If anyone else knows of a Python parser that uses ABNF, I'd love to hear about it.
Related
I have built an antlr v4 grammar for parsing Cobol files. It is tested and fully functional. Now I need to adapt it to be used within a XText project (with unfortunately uses antlrv3). How can I achieve it without backporting my grammar (and loosing all already built Listeners and Visitors)?
After a few thinking about the problem I am wondering if there is a way to generate antlr v3 interface adaptors to use antlr v4 Parser and Lexer. If so, i could "tweak" XText, so it would be using my already built antlrv4 classes through this adaptor interface.
Anyone already had done something like this?
We are currently working on this problem as well. Is there a possibility to look at your grammar file?
I'm trying to use Antlr4.5 to generate a lexer/parser (in C#) for a SQL grammar. What sort of tools would you recommend in order to write the grammar and to test it?
I'm trying antlrworks2 but I find it a bit confusing (can't find a way to enter sample text and see the parse tree - not sure if it's not there or I'm blind or just plain dumb, but I'm pretty sure antlrworks v1 had such a feature).
The Visual Studio VSIX plugin from Sam Harwell looks great but it also confuses the heck out of me, the lexer and parser get generated in the obj\ folder instead of inside the project. Also, it won't help me much when writing the grammar as far as showing me a syntax diagram, letting me try out the grammar with sample text and so on.
I'm pretty confident that antlr is a good way to go, but I'm really not sure about the tooling and the approach I should take when defining and testing my grammar. I would appreciate any suggestions.
My question wasn't brilliant because tooling is pretty independent of the antlr target language, but FWIW I ended up using the free version of IntelliJ IDEA with the antlr plugin. I started liking IntelliJ, it's very good. The antl plugin is also very good for the most part, it a bit rough around the edges here and there, but all the important stuff works very well and in general it's amazingly useful.
Has somebody used both for Language implementation and
is able to compare them, pointing out strengths and
weaknesses? I seek a RAD tool with support for
AST Walker Code generation. SableCC is LALR and thus
supports ´Left recursion´, whereas ANTLR is LL(*).
Is this important for typical grammars or DSLs? I need
to perform some domain-specific analysis as well. (The target
language of my compiler will be OpenCL C). As this will be
for a student project it is important that I do not lose
that much time on the tedious side, that is implementing
the Front-End of the language.
I cannot say much about ANTLR, but maybe some information about SableCC.
Design
It generate a parser, which generated code and hand-written code are clean separated using Visitor pattern, and integrates the transform from Concrete Syntax Tree to Abstract Syntax Tree. As a result the designer can get a AST after the parser parses successful the input, and he can walk through the tree and make action on corresponding nodes.
The designer can first write and debug his grammar, try to optimize the transform from Concrete Syntax Tree to Abstract Syntax Tree. After he has a solid AST he can write action code in separated class. So the designer write grammar only once and can write more type of action for the grammar, for example once for Syntax Highlight, once for Semantic analysis and code generator. I have done it in a productive system. It works very well.
With ANTLR the designer can construct the AST tree by adding action code in grammar t generate the AST, then reuses it for different manner. But there a not a clean separation between generated code and hand-written code.
An other aspect maybe support of IDE. Since with SableCC you have separated code, you can easy use auto-complete function of IDE.
grammar
SableCC is a LR(1) parser generator, so it is IMO easier to write grammar for ANTLR, which is a LL(k) parser generator, (without trick). I think (aber not sure) SableCC is the only one LR(1) java parser generator, which is so popular.
output parser
ANTLR can generate parser in many languages, while SableCC can only generate parser in Java (mainstream). There some plugin / adapter to generate parser in other language, however according to the author (http://www.mare.ee/indrek/sablecc/) they are too old. SableCC 4 can generate more, but it is in beta, which is not recommend for serious project.
Development Support
ANTLR hat a IDE to write grammar. It is ANTLRWorks, which can visual grammar, navigate in source (like jump to definition of token or production). SableCC hat no such tools. There are primitive Syntax Highlight script for VIM and a poor feature plugin for Netbeans.
Conclusion
IMO I think for big project, required long term maintenance SableCC is more suitable than ANTLR.
Martin Fowler has a informative about SableCC, you can find it here.
http://martinfowler.com/bliki/HelloSablecc.html
I'm working on a project that requires me to create a series of editors for languages that are quite different. The syntaxes are defined by us.
I'm looking for a solution for this.
Is there a shortcut to take in this problem?
You could use XText:
a framework for development of textual domain specific languages (DSLs).
Just describe your very own DSL using Xtext's simple EBNF grammar language and the generator will create a parser, an AST-meta model (implemented in EMF) as well as a full-featured Eclipse text editor from that.
Alternatives to XText are Rascal or Spoofax, both less popular than XText but interesting for they support more general context-free grammars, among other things. Nice to check out.
If you are looking for a more low level, programmable solution, then Eclipse's IDE Meta-tooling platform is a good choice (IMP).
What IMP gives you is API to connect your existing parsers to Eclipse without much hassle. You need to implement an IParseController interface, to call your parser and ITokenIterator to produce tokens and some other interface to assign fonts to each kind of token.
The aforementioned Rascal and Spoofax are both build on top of IMP.
Not mentioned is DLTK (proposed also in Tutorial regarding the development of a custom Eclipse editor)
There are Ruby, bash that are implemented with it.
If I were looking to create my own language are there any tools that would help me along? I have heard of yacc but I'm wondering how I would implement features that I want in the language.
Closely related questions (all taken by searching on [compiler] on stackoverflow):
Learning Resources on Parsers, Interpreters, and Compilers
Learning to write a compiler
Constructing a simple interpreter
...
And similar topics (from the same search):
Bootstrapping a language
How much of the compiler should we know?
Writing a compiler in its own language
...
Edit: I know the stackoverflow related question search isn't what we'd like it to be, but
did we really need the nth iteration of this topic? Meh!
The first tool I would recommend is the Dragon Book. That is the reference for building compilers. Designing a language is no easy task, implementing it is even more difficult. The dragon book helps there. The book even reference to the standard unix tools lex and yacc. The gnu equivalent tools are called flex and bison. They both generate lexer and parser. There exist also more modern tools for generating lexer and parser, e.g. for java there are ANTLR (I also remember javacc and CUP, but I used myself only ANTLR). The fact that ANTLR combines parser and lexer and that eclipse plugin is availabe make it very comfortable to use. But to compare them, the type of parser you need, and know for what you need them, you should read the Dragon book. There are also other things you have to consider, like runtime environment, programming paradigm, ....
If you have already certain design ideas and need help for a certain step or detail the anwsers could be more helpful.
ANTLR is a very nice parser generator written in Java. There's a terrific book available, too.
I like Flex (Fast Lex) [Lexical scanner]
and Bison (A Hairy Yacc) [Yet another compiler compiler]
Both are free and available on all *NIX installations. For Windows just install cygwin.
But I old school.
By using these tools you can also find the lex rules and yacc gramers for a lot of popular languages on the internet. Thus providing you with a quick way to get up and running and then you can customize the grammers as you go.
Example: Arithmetic expression handling [order of precedence etc is a done to death problem] you can quickly get the grammer for this from the web.
An alternative to think about is to write a front-end extension to GCC.
Non Trivial but if you want a compiled language it saves a lot of work in the code generation section (you will still need to know love and understand flex/bison).
I never finished the complete language, I had used rply and llvmlite implements a simple foxbase language, in https://github.com/acekingke/foxbase_compiler
so if you want use python, rply or llvmlite is helpful.
if you want use golang, goyacc maybe useful. But you should write a lexical analyzer by hard coding by hand. Or you can use https://github.com/acekingke/lexergo to simplify it.