Looking for sample for building ANTLR4 grammar based Code Editor with intellisense. SharpDevelop provides all Code Editor features, however if we need to provide the intellisense and Code Completion details, then we need to write own parser.
Need sample where ANTLR4, SharpDevelop is used for building the Code Editor for custom language.
Thanks.
I could get the expected Tokens from ANTLR4 using GetExpectedTokensWithinRule API in the Listener and converting them to tokens.
pseudo code looks like this
public class MyGrammarListener : MyGrammarBaseListener
{
public MyGrammarListener(MyGrammarParser parser)
{
this.Parser = parser;
}
public override void EnterXXXXX(XXXXX_Context context)
{
IntervalSet set = Parser.GetExpectedTokensWithinCurrentRule();
base.EnterXXXXX(context);
foreach (int token in set.ToIntegerList())
{
// Returns the expected tokens.
string data = Parser.Vocabulary.GetLiteralName(token);
}
}
}
I have used Jide CodeEditor with antlr4, it seems to be working OK but took some time to get it together. I generate the errors and keywords for highlighting from the parser. I use listeners for parsing etc. and a visitor for executing the language. Not familiar with SharpDevelop
Related
I'm pretty new to Kotlin, and I've come across both of these representations:
Car(name = "CarName")
and
car {
name = "CarName"
}
Is there any guidelines about when which one should be used? The docs don't seem to be too clear on this.
The second snippet is an example of how you could build a DSL for your domain. For simple cases like this, it is a bit overkill to create a DSL, but when your objects get larger it might be cleaner to design a DSL.
In fact, using the DSL style to create simple instances might even be confusing.
For example, the documentation on DSLs shows the following code:
fun result(args: Array<String>) =
html {
head {
title {+"XML encoding with Kotlin"}
}
body {
h1 {+"XML encoding with Kotlin"}
p {+"this format can be used as an alternative markup to XML"}
// an element with attributes and text content
a(href = "http://kotlinlang.org") {+"Kotlin"}
// mixed content
p {
+"This is some"
b {+"mixed"}
+"text. For more see the"
a(href = "http://kotlinlang.org") {+"Kotlin"}
+"project"
}
p {+"some text"}
// content generated by
p {
for (arg in args)
+arg
}
}
}
This is an excellent example of when you could use a DSL: The syntax enables a clean structure of how you create your models. Anko for another provides a DSL to defines UI's.
Currently in Web-Essentials (for Visual Studio 2015), if a .ts file from the Solution Explorer is dragged and dropped into an open .ts file, a reference path is automatically inserted at the top:
/// <reference path="../playback/key.ts" />
This is fine when a project is being developed using internal modules (namespaces), but is virtually useless when going external. How could I make it so that an ES6 import statement is generated instead? That would be awesome. Such as:
import {} from "../playback/key";
It turns out what we are seeing is a built-in feature of Web-Essentials. I've filed a feature-request to get this available, hopefully in the near future.
In the meantime, the following Visual Commander automation snippet can replace all references to ES6 imports:
using EnvDTE;
using EnvDTE80;
using System.Text.RegularExpressions;
public class C : VisualCommanderExt.ICommand
{
// Called by Visual Commander extension.
public void Run(EnvDTE80.DTE2 DTE, Microsoft.VisualStudio.Shell.Package package)
{
TextDocument doc = (TextDocument)(DTE.ActiveDocument.Object("TextDocument"));
var p = doc.StartPoint.CreateEditPoint();
string s = p.GetText(doc.EndPoint);
p.ReplaceText(doc.EndPoint, this.ReplaceReferences(s), (int)vsEPReplaceTextOptions.vsEPReplaceTextKeepMarkers);
}
// Converts "reference" syntax to "ES6 import" syntax.
private string ReplaceReferences(string text)
{
string pattern = "\\/\\/\\/ *<reference *path *= *\"([^\"]*)(?:\\.ts)\" *\\/>";
var regex = new Regex(pattern);
var matches = Regex.Matches(text, pattern);
return Regex.Replace(text, pattern, "import {} from \"./$1\";");
}
}
The regex used to perform this replace is (without the double backslashes, more readable):
\/\/\/ *<reference *path *= *"([^"]*)(?:\.ts)" *\/>
This snippet will effectively convert all reference comments to ES6 imports in the currently active document, very useful.
Known issues:
a comment containing a reference tag is not recognized by Visual Studio if it is preceded by other text in the corresponding line, but it is still replaced by this snippet
additional text after a reference tag is valid, but after replace will cause syntax errors, since it is no longer a comment.
(Neither of these issues are relevant if the snippet is used on reference tags generated by Web-Essentials when a .ts file is drag-and-dropped.)
I'm a novice programmer trying to learn plug-in development. I'd like to upgrade the sample XML editor so that some words like "cat", "dog", "hamster", "rabbit" and "bird" would be highlighted when it appears in an XML file (it's just for learning purpose). Can anyone give me some implementation tips or suggestions? I am clueless.. (But I am carrying out my research on this as well, I'm not being lazy. You have my word.) Thanks in advance.
You can detect words in the plain text part of the XML by modifying the sample XML editor as follows.
We can use the provided WordRule class to detect the words. The XMLScanner class which scans the plain text needs to be updated to include the word rule:
public XMLScanner(final ColorManager manager)
{
IToken procInstr = new Token(new TextAttribute(manager.getColor(IXMLColorConstants.PROC_INSTR)));
WordRule words = new WordRule(new WordDetector());
words.addWord("cat", procInstr);
words.addWord("dog", procInstr);
// TODO add more words here
IRule [] rules = new IRule [] {
// Add rule for processing instructions
new SingleLineRule("<?", "?>", procInstr),
// Add generic whitespace rule.
new WhitespaceRule(new XMLWhitespaceDetector()),
// Words rules
words
};
setRules(rules);
}
I have used the existing processing instruction token here to reduce the amount of new code, but you should define a new color and use a new token.
The WordRule constructor requires an IWordDetector class, we can use a very simple detector here:
class WordDetector implements IWordDetector
{
#Override
public boolean isWordStart(final char c)
{
return Character.isLetter(c);
}
#Override
public boolean isWordPart(final char c)
{
return Character.isLetter(c);
}
}
This is just accepting letters in words.
I've written an Xtext-based plugin for some language. I'm now interested in creating a new independent view (as a separate plugin, though it requires my first plugin), which will interact with the currently-active DSL document - and specifically, interact with the model Xtext parsed (I think it's called the Ecore model?). How do I approach this?
I saw I can get an instance of XtextEditor if I do something like this when initializing my view:
getSite().getPage().addPartListener(new MyListener());
And then, in MyListener, override partActivated and partInputChanged to get an IWorkbenchPartReference, which is a reference to the XtextEditor. But what do I do from here? Is this even the right approach to this problem? Should I instead use some notification functionality from the Xtext side?
Found it out! First, you need an actual document:
IXtextDocument doc = editor.getDocument();
Then, if you want to access the model:
doc.modify(new IUnitOfWork.Void<XtextResource>() { // Can also use just IUnitOfWork
#Override public void process(XtextResource state) throws Exception {
...
}
});
And if you want to get live updates whenever it changes:
doc.addModelListener(new IXtextModelListener() {
#Override public void modelChanged(XtextResource resource) {
for (EObject model : resource.getContent()) {
...
}
}
});
I need to create a grammar for a language with forward references. I think that the easiest way to achieve this is to make several passes on the generated AST, but I need a way to store symbol information in the tree.
Right now my parser correctly generates an AST and computes scopes of the variables and function definitions. The problem is, I don't know how to save the scope information into the tree.
Fragment of my grammar:
composite_instruction
scope JScope;
#init {
$JScope::symbols = new ArrayList();
$JScope::name = "level "+ $JScope.size();
}
#after {
System.out.println("code block scope " +$JScope::name + " = " + $JScope::symbols);
}
: '{' instruction* '}' -> ^(INSTRUCTION_LIST instruction*)
;
I would like to put a reference to current scope into a tree, something like:
: '{' instruction* '}' -> ^(INSTRUCTION_LIST instruction* {$JScope::symbols})
Is it even possible? Is there any other way to store current scopes in a generated tree? I can generate the scope info in a tree grammar, but it won't change anything, because I still have to store it somewhere for the second pass on the tree.
To my knowledge, the syntax for the rewrite rules doesn't allows for directly assigning values as your tentative snippet suggests. This is in part due to the fact that the parser wouldn't really know to what part of the tree/node the values should be added to.
However, one of the cool features of ANTLR-produced ASTs is that the parser makes no assumptions about the type of the Nodes. One just needs to implement a TreeAdapator which serves as a factory for new nodes and as a navigator of the tree structure. One can therefore stuff whatever info may be needed in the nodes, as explained below.
ANTLR provides a default tree node implementation, CommonTree, and in most cases (as in the situation at hand) we merely need to
subclass CommonTree by adding some custom fields to it
subclass the CommonTreeAdaptor to override its create() method, i.e. the way it produces new nodes.
but one could also create a novel type of node altogher, for some odd graph structure or whatnot. For the case at hand, the following should be sufficient (adapt for the specific target language if this isn't java)
import org.antlr.runtime.tree.*;
import org.antlr.runtime.Token;
public class NodeWithScope extends CommonTree {
/* Just declare the extra fields for the node */
public ArrayList symbols;
public string name;
public object whatever_else;
public NodeWithScope (Token t) {
super(t);
}
}
/* TreeAdaptor: we just need to override create method */
class NodeWithScopeAdaptor extends CommonTreeAdaptor {
public Object create(Token standardPayload) {
return new NodeWithScope(standardPayload);
}
}
One then needs to slightly modify the way the parsing process is started, so that ANTLR (or rather the ANTLR-produced parser) knows to use the NodeWithScopeAdaptor rather than CommnTree.
(Step 4.1 below, the rest if rather standard ANTLR test rig)
// ***** Typical ANTLR pipe rig *****
// ** 1. input stream
ANTLRInputStream input = new ANTLRInputStream(my_input_file);
// ** 2, Lexer
MyGrammarLexer lexer = new MyGrammarLexer(input);
// ** 3. token stream produced by lexer
CommonTokenStream tokens = new CommonTokenStream(lexer);
// ** 4. Parser
MyGrammarParser parser = new MyGrammarParser(tokens);
// 4.1 !!! Specify the TreeAdapter
NodeWithScopeAdaptor adaptor = new NodeWithScopeAdaptor();
parser.setTreeAdaptor(adaptor); // use my adaptor
// ** 5. Start process by invoking the root rule
r = parser.MyTopRule();
// ** 6. AST tree
NodeWithScope t = (NodeWithScope)r.getTree();
// ** 7. etc. parse the tree or do whatever is needed on it.
Finally your grammar would have to be adapted with something akin to what follows
(note that the node [for the current rule] is only available in the #after section. It may however reference any token attribute and other contextual variable from the grammar-level, using the usual $rule.atrribute notation)
composite_instruction
scope JScope;
#init {
$JScope::symbols = new ArrayList();
$JScope::name = "level "+ $JScope.size();
}
#after {
($composite_instruction.tree).symbols = $JScope::symbols;
($composite_instruction.tree).name = $JScope::name;
($composite_instruction.tree).whatever_else
= new myFancyObject($x.Text, $y.line, whatever, blah);
}
: '{' instruction* '}' -> ^(INSTRUCTION_LIST instruction*)
;