I'm developing a python library for domain specific languages and I want to include as many language definitions as possible. For regular expressions it's easy, since the format is the same across all implementation and there are several repositories out there like regexlib.
Is there any online library/repository available with loads of BNF/eBNF language examples (like regexlib.com)?
The Grammar Zoo from the Software Language Processing Suite project might help you.
Related
I want to bind a different programming language to the Godot game engine. Is there an instructional document or video on this topic? For example, how was this project done: godot-rust. If I can learn the basics, I can succeed in working in a different language. Thanks.
In this answer I show you the different approaches to add language support in Godot 3.x (the situation will be somewhat different with Godot 4.0 and GDExtension - which replaces GDNative and hopefully means less custom builds), and I mention some languages that are supported by each of these approaches. However, this is not an exhaustive list of the languages.
First of all, Godot has official build-in support for GDScript and Godot's VisualScript (and Godot's shading language and its visual counterpart if those counts for you).
There are a few ways to use C++:
You can use it to create GDNative scripts (which are basically a wrapper around native calls that allow you to use them as scripts in Godot).
Or you can create modules (which are static libraries you can add in a custom Godot build).
And since Godot source is in C++, you don't have to restrict yourself to making modules if you are making custom builds.
In web builds Godot can interface with JavaScript via the JavaScript class. However, this approach does not allow you to add JavaScript scripts to Nodes, and so on.
Then there are languages that can only be added in custom builds of Godot, which is currently the official support for C#.
There are other non-official custom builds that offer language binding for languages such as Lua, Kotlin, TypeScript and JavaScript (this time allowing you to make scripts).
If you need to add a runtime, you would probably do this.
Some language take advantage of the fact that Godot's has official Mono support in order to support C#. This way you can, for example, use F# and Clojure.
They start by adding a C# project and then modify it so it uses another language. This is viable if your language already compiles to .NET.
Some other languages can be added as plugins that implement the PluginScript class via GDNative. This is the case of Python and Lua (again) which you can get from the asset library.
This is the most user friendly way to add language support to Godot, but it is limited to what you can do with PluginScript.
Addendum: Gil Barbosa Reis, author of the aforementioned Lua bindings, has an article series about its implementation stuffed away in the repository (in English and Portugueses): godot-lua-pluginscript/extras/articles/. It is probably the most comprehensive tutorial to date.
Other languages are added by means of taking advantage of GDNative (They basically mimic what you would do with C++). This is the case of Nim, Rust, D, Haskell, Go, Swift…
So that's how godot-rust works: make native libraries using rust and the godot-rust create and add them as if they were made in C++. For any language for which there are the means to make native libraries already, this is a good option.
Finally there is another way to add support for a language: a transpiler from that language to GDScript, which can be automated with an addon that might also be written in GDScript. This is the case of Lisp.
This last approach is mostly used for domain specific languages.
The official docs here provide your answer:
Godot officially supports GDScript, C/C++, C#.
Some 3rd party languages that can be used are: Rust, D, Python, Nim, and Go.
my project this year is to develop a text mining tool (with new features)
so we need a mini script language in this tool to add annotation to texts
this language should be simple and like lisp grammars (left and right side) .
what i need is how to design this language ,i know how to constract the compiler ,
but how to write language grammars ? , and i want to use some mini open source language or any language bnf
please advice me and if there is a language i can use and customize to meet my needs ?
EDIT : if anybody can give a link for grammar (bnf) for lisp or any language like it
many thanks
You can consider us this
http://www.antlr.org/
to design your language.
If possible, I would recommend just using Lua. If you use their interpreter, which is designed to be easily embedded in other programs, it will save you the time of designing, implementing, and testing your own language.
Otherewise, you may be able to re-use the Lua parser code to parse your own Lua-like language.
I'm looking for an editor/IDE that can provide features that are nice to have while coding (example: being able to click-through to function definitions) for languages that it is not specifically built for. By these, I have in mind languages designed for a very specific purpose and often only used by a small community. In other words, ones that would not have native support in most software.
I realize this would require a fair bit of fairy dust but I don't think it's out of the scope of what's possible. Basically, the editor would have to be smart enough to recognize the commonly used syntax and semantics that many declarative languages have in common. It's quite possible this would require some amount of configuration.
Does something like this exist? If not, what challenges do you think there would be in creating it?
If you need only the feature to jump of to the definition of a specific function or class, then VIM (and many other editors, like Emacs, Epsilon and JOE) can read the jump location from the ctags file. You just have to write a ctags file generator for your custom language.
For programmable editors (like VIM, Emacs, Epsilon, Eclipse and gedit), you can write your own plugin for your custom language, but it may quickly become time-consuming and a maintenance nightmare, because new versions of editors tend to change the plugin interface.
Please note that adding support for syntax highlighting is usually much easier than adding ctags-like support for symbol lookups. More advanced features, like refactoring and context-sensitive symbol completion (like Ctrl-Space and Tab in modern IDEs) are even harder to implement.
GNU Emacs has a pretty good infrastructure for this sort of thing. Until recently Haskell was a relatively unknown language used primarily by researchers. Nevertheless, in a few thousand lines of Emacs Lisp, we have
Syntax highlighting with colors
Automatic indentation
Package support
Automatic highlighting of type and other information when placing the cursor over library functions
Meta-dot on an identifier to jump to its definition (through the standard emacs tags mechanism)
The nice thing about Emacs is that (a) there are many models to follow, and (b) you can build up the environment gradually, starting with those aspects that are most important to you.
I'm suprised no one has mentioned Notepad++ yet:
http://notepad-plus-plus.org/
It offers syntax support for a great many languages and offers the user to add new languages, and an active community that adds many languages that are not included out-of-the-box.
Most good IDE's are language agnostic and supports several if not many programming languages. If you are talking about DSL's, eclipse has a solution that seems pretty awsome - Xtext
EditPadPro comes with a set of tools that allow you to build your own syntax highlighting, code folding and file navigation schemes, based on a very powerful regex syntax. So if your language is not among the many that have already been provided out-of-the-box or can be downloaded off the website, you can roll your own quite easily (and share it with the community).
Visual Studio is designed to allow for this, but it requires the language to add explicit support. For example, Delphi Prism will install into Visual Studio, and provide full language support.
This is far above and beyond "configuration", however, and requires quite a bit of custom development to support.
SciTE and Scintilla offer a generic editor/platform for different languages. The library contains several parsers that work with many programming languages and you can reuse one of these for your own language to add formatting and syntax highlighting.
They don't offer advanced features like click-throughs, but you could build it on top of the library.
Visual Studio and Eclipse also support language plug-ins.
Zeus is a language neutral IDE for the Windows platform and it provides this go to definition/declaration functionality for any language supported by ctags.
To make it work you just create a Zeus project/workspace and then add the files to this workspace.
I'm working on a project that requires me to create a series of editors for languages that are quite different. The syntaxes are defined by us.
I'm looking for a solution for this.
Is there a shortcut to take in this problem?
You could use XText:
a framework for development of textual domain specific languages (DSLs).
Just describe your very own DSL using Xtext's simple EBNF grammar language and the generator will create a parser, an AST-meta model (implemented in EMF) as well as a full-featured Eclipse text editor from that.
Alternatives to XText are Rascal or Spoofax, both less popular than XText but interesting for they support more general context-free grammars, among other things. Nice to check out.
If you are looking for a more low level, programmable solution, then Eclipse's IDE Meta-tooling platform is a good choice (IMP).
What IMP gives you is API to connect your existing parsers to Eclipse without much hassle. You need to implement an IParseController interface, to call your parser and ITokenIterator to produce tokens and some other interface to assign fonts to each kind of token.
The aforementioned Rascal and Spoofax are both build on top of IMP.
Not mentioned is DLTK (proposed also in Tutorial regarding the development of a custom Eclipse editor)
There are Ruby, bash that are implemented with it.
If I were looking to create my own language are there any tools that would help me along? I have heard of yacc but I'm wondering how I would implement features that I want in the language.
Closely related questions (all taken by searching on [compiler] on stackoverflow):
Learning Resources on Parsers, Interpreters, and Compilers
Learning to write a compiler
Constructing a simple interpreter
...
And similar topics (from the same search):
Bootstrapping a language
How much of the compiler should we know?
Writing a compiler in its own language
...
Edit: I know the stackoverflow related question search isn't what we'd like it to be, but
did we really need the nth iteration of this topic? Meh!
The first tool I would recommend is the Dragon Book. That is the reference for building compilers. Designing a language is no easy task, implementing it is even more difficult. The dragon book helps there. The book even reference to the standard unix tools lex and yacc. The gnu equivalent tools are called flex and bison. They both generate lexer and parser. There exist also more modern tools for generating lexer and parser, e.g. for java there are ANTLR (I also remember javacc and CUP, but I used myself only ANTLR). The fact that ANTLR combines parser and lexer and that eclipse plugin is availabe make it very comfortable to use. But to compare them, the type of parser you need, and know for what you need them, you should read the Dragon book. There are also other things you have to consider, like runtime environment, programming paradigm, ....
If you have already certain design ideas and need help for a certain step or detail the anwsers could be more helpful.
ANTLR is a very nice parser generator written in Java. There's a terrific book available, too.
I like Flex (Fast Lex) [Lexical scanner]
and Bison (A Hairy Yacc) [Yet another compiler compiler]
Both are free and available on all *NIX installations. For Windows just install cygwin.
But I old school.
By using these tools you can also find the lex rules and yacc gramers for a lot of popular languages on the internet. Thus providing you with a quick way to get up and running and then you can customize the grammers as you go.
Example: Arithmetic expression handling [order of precedence etc is a done to death problem] you can quickly get the grammer for this from the web.
An alternative to think about is to write a front-end extension to GCC.
Non Trivial but if you want a compiled language it saves a lot of work in the code generation section (you will still need to know love and understand flex/bison).
I never finished the complete language, I had used rply and llvmlite implements a simple foxbase language, in https://github.com/acekingke/foxbase_compiler
so if you want use python, rply or llvmlite is helpful.
if you want use golang, goyacc maybe useful. But you should write a lexical analyzer by hard coding by hand. Or you can use https://github.com/acekingke/lexergo to simplify it.