Code Editor using ANTLR - ide

Question

I'm in the process of writing a code editor for a custom language. We're using ANTLR for the lexer and parser, and CodeMirror for the editor framework (running in a browser).
I can do some basic things like syntax coloring of keywords, as well as providing rudimentary code completion.
What I'm finding is that the user is frequently in the middle of editing something, so the ANTLR parser is not very useful since the current input stream is not fully parseable (and often leads ANTLR down the wrong path due to an incomplete input stream).
Therefore, I'm using the token stream to figure out what's going on and attempt to provide context-sensitive help.
I'm wondering if anyone can provide some guidance around using ANTLR as part of a code editor. Am I on the right track using the token stream instead of the parse tree?
Can the ANTLR API be leveraged to do things like looking ahead for tokens to figure out the overall context of what the user is currently editing?
Sorry if this is kind of vague. Just getting started on this project. :-)
Thanks for any help.

Answer 1

I find ANTRL great for syntax checking and easy details retrieval for valid input. For code completion however you have a different scenario. As you found out already the parser can often not give you good answers, because the input is not valid while the user is typing. Bart has linked to an answer where Sam described that he implemented a great solution using ANTLR 4, but unfortunately doesn't describe how.
But even if you can get the parser to give you a set of expected tokens, what are you going to make out of it? What do you wanna show if, say, an identifier is expected? That can be anything, like a class member, a var name etc. I don't believe this is the answer, hence I developed an own solution which I describe here: Universal Code Completion using ANTLR. This is for ANTLR 3, but can certainly be made working with 4 as well.
This article also contains several links to (C++) source code that shows how code completion is implemented in my application. It's amazing how simple the implementation is, after all, but still delivers very precise results.
Update
Meanwhile I developed a solution for ANTLR4 too, called [antlr4-c3][2]. This is a Typescript solution but comes with translations to [Java][3] and [C++][4].

Answer 2

If someone is interested in the integration of Codemirror 6 and Antlr4 grammar,
you can find an example on github page.
The main idea is here:
token: (stream, state) => {
// getting tokens for the current stream.string, using antlr4 TokenStream
const tokens = getTokensForText(stream.string);
// getting the next token
const nextToken = tokens.filter(t => t.startIndex >= stream.pos)[0];
// matching the next token in current stream
if (stream.match(nextToken.text)) {
let valueClass = getStyleNameByTag(tags.keyword);
switch (nextToken.type) {
case ...:
valueClass = getStyleNameByTag(tags.string);
break;
...
default:
valueClass = getStyleNameByTag(tags.keyword);
break;
}
return valueClass;
} else {
stream.next();
return null;
}
}