How does one remove shift/reduce errors caused by limited lookahead? - yacc

Question

In my grammar, it's usually possible to only use declaration which looks like:
int x, y, z = 23;
int i = 1;
int j;
In for I'd like to use a set of comma separated declarations of different types e.g.
for (int i = 0, double d = 2.0; i < 0; i++) { ... }
Using yacc, the limited lookahead creates problems. Here is the naive grammar:
variables_decl
: type_expression IDENT
| variables_decl ',' IDENT
;
declaration
: variables_decl
| variables_decl '=' initializer
;
declaration_list
: declaration
| declaration_list ',' declaration
;
This causes a shift/reduce error on ',':
state 149
100 variables_decl: variables_decl . ',' IDENT
101 declaration: variables_decl .
102 | variables_decl . '=' initializer
',' shift, and go to state 261
'=' shift, and go to state 262
',' [reduce using rule 101 (declaration)]
$default reduce using rule 101 (declaration)
I'd like fix this issue so that this actually works:
for (double x, y, int i, j = 0, long l = 1; i < 0; i++) { ... }
But it's not obvious to me how to do that.

Answer 1

In general terms, you avoid this type of shift/reduce conflict by avoiding forcing the parser to make a decision until absolutely necessary.
It's understandable why you have structured the grammar as you have; intuitively, a declaration list is a list of declarations, where each declaration is a type and a list of variables of that type. The problem is that this definition makes it impossible to know whether a comma belongs to an inner or outer list.
Moreover, one extra lookahead token might not be enough, since the following IDENT could be a typename or the name of a variable to be declared, assuming type-expression is the usual C syntax which can start with an identifier corresponding to a typename.
But that's not the only way to look at the declaration list syntax. Instead, you can think of it as a list of individual declarations, each of which starts with an optional type (except the first in the list, which must have an explicit type), using the semantic convention that an omitted type is the same as the type of the previous variable. That leads to the following conflict-free grammar:
declaration_list: explicit_decl
| declaration_list ',' declaration
declaration : explicit_decl
| implicit_decl
explicit_decl : type_expression implicit_decl
implicit_decl : IDENT opt_init
opt_init : %empty | '=' expr
That does not capture the syntax of C declarations, since not all C declarations have the form type_expression IDENT. The IDENT being defined can be buried inside the declaration, as with, for example, int a[4] or int f(int i); fortunately, these forms are of limited use in a for loop.
On the other hand, unlike your grammar it does allow all declared variables to be initialised, so
int a = 1, b = 0, double x = -1.0, y = 0.0
should work.
Another note: the first item in a C for clause can be empty, a declaration (possibly in the form of a list) or an expression. In the last case, a top-level , is an operator, not a list indicator.
In short, the above fragment might or might not be a solution in the context of your actual grammar. But it is conflict-free in a simple test framework where typed declarations are always of the form typename identifier.