gawk: Local variables typed differently in SYMTAB - awk

Question

Function parameters seem to be typed differently in the SYMTAB array than if you check them directly.
i.e. typeof(p) is not always equal to typeof(SYMTAB["p"])
I'm trying to write a generic function to initialize a one-dimensional array, with a variable number of elements. This issue is specific to gawk as it uses the SYMTAB array that doesn't exist in standard awk.
For example:
array_new(array1, 1, 2, "c", 4, "e")
array_new(array2, 99, 98, 97)
function array_new(arr, p1, p2, p3, p4, p5, p6) {
# initialization code
}
gawk doesn't allow for optional parameters, so I have set a reasonable upper limit - 6 for the example above but in practice I've been using 20 - and then within the function I only want to create elements for which a value has been given.
A first "brute force" approach that I've tried, and which works, is as follows :
function array_new(arr, p1, p2, p3, p4, p5, p6) {
delete arr
if (typeof(p1) != "untyped") arr[1] = p1
if (typeof(p2) != "untyped") arr[2] = p2
if (typeof(p3) != "untyped") arr[3] = p3
if (typeof(p4) != "untyped") arr[4] = p4
if (typeof(p5) != "untyped") arr[5] = p5
if (typeof(p6) != "untyped") arr[6] = p6
return
}
But the more parameters I want to allow, the more unwieldy this becomes, so I wanted to loop through the parameters and exit as soon as an uninitialized one is encountered.
So I tried the following, but it doesn't work :
function array_new_2(arr, p1, p2, p3, p4, p5, p6, n) {
for (n=1; n<=6; n++) {
if (typeof(SYMTAB["p"n]) != "untyped") arr[n] = SYMTAB["p"n]
else break
}
return
}
Trying to understand why I ran some tests, and it turns out that typeof(p...) is "string", "number" or "untyped" depending on what is passed to the function, or not, but typeof(SYMTAB["p"n]) is always "unassigned".
This is my complete code, with some debugging "print" statements added to see what is happening:
END {
array_new_1(aaa, 3, 2, 1)
array_walk(aaa, "aaa")
print ""
array_new_2(bbb, "a", "b", "c")
array_walk(bbb, "bbb")
print ""
}
function array_walk(arr, name, i) {
for (i in arr) {
if (isarray(arr[i]))
array_walk(arr[i], (name "[" i "]"))
else
printf("%s[%s] = '%s'\n", name, i, arr[i])
}
}
function array_new_1(arr, p1, p2, p3, p4, p5, p6) {
print "p1 : " typeof(p1), p1
print "p6 : " typeof(p6), p6
print "SYMTAB[\"p1\"] : " typeof(SYMTAB["p1"]), SYMTAB["p1"]
print "SYMTAB[\"p6\"] : " typeof(SYMTAB["p6"]), SYMTAB["p6"] # this creates an "unassigned" value
if (typeof(p1) != "untyped") arr[1] = p1
if (typeof(p2) != "untyped") arr[2] = p2
if (typeof(p3) != "untyped") arr[3] = p3
if (typeof(p4) != "untyped") arr[4] = p4
if (typeof(p5) != "untyped") arr[5] = p5
if (typeof(p6) != "untyped") arr[6] = p6
return
}
function array_new_2(arr, p1, p2, p3, p4, p5, p6, n) {
print "p1 : " typeof(p1), p1
print "p6 : " typeof(p6), p6
print "SYMTAB[\"p1\"] : " typeof(SYMTAB["p1"]), SYMTAB["p1"]
print "SYMTAB[\"p6\"] : " typeof(SYMTAB["p6"]), SYMTAB["p1"]
for (n=1; n<=6; n++) {
if (typeof(SYMTAB["p"n]) == "unassigned") arr[n] = SYMTAB["p"n]
else break
}
return
}
This produces the following output :
p1 : number 3
p6 : untyped
SYMTAB["p1"] : unassigned
SYMTAB["p6"] : unassigned
aaa[1] = '3'
aaa[2] = '2'
aaa[3] = '1'
aaa[6] = ''
p1 : string a
p6 : untyped
SYMTAB["p1"] : unassigned
SYMTAB["p6"] : unassigned
bbb[1] = ''
bbb[2] = ''
bbb[3] = ''
bbb[4] = ''
bbb[5] = ''
bbb[6] = ''
So, to sum up, can anyone explain why the SYMTAB versions of the parameters are typed differently from the variables themselves, and also, is there a better way to write my array_new function ?

Answer 1

In appears (to me) that the SYMTAB[]/typing question is a diversion from the main issue of figuring out how to (efficiently) provide a variable number of input parameters to an awk function. [NOTE: This is similar to the issue of wanting to feed a variable number of values on the command line via several -v var=val clauses.]
One common workaround is to concatenate the (variable) list of values into a single delimited string, then have the code split() this delimited string into a variable length array of values.
Applying this to OP's case:
array_new(array1, "1,2,c,4,e") # concatenate the list of values into a comma-delimited string
function array_new{arr, plist) {
n=split(plist,p,",") ... # split the comma-delimited string into an array named p[], with "n" elements in the array
for (i=1;i<=n;i++) { # loop through the indices of the p[] array (1 to n)
# do something with p[i]
}
}
If the sole purpose of OP's function is to populate an array then we can use the input parameter (arr) as the target of the split() call, eg:
function array_new(arr, plist) {
split(plist,arr,",")
}
Running some tests:
array_new(array1, "1,2,c,4,e")
print "########### array1"
for (i=1;i<=length(array1);i++)
print "array1[" i "] = " array1[i]
array_new(array2, "99,98,97")
print "########### array2"
for (i=1;i<=length(array2);i++)
print "array2[" i "] = " array2[i]
This generates:
########### array1
array1[1] = 1
array1[2] = 2
array1[3] = c
array1[4] = 4
array1[5] = e
########### array2
array2[1] = 99
array2[2] = 98
array2[3] = 97
NOTE: for this sample code I've used a comma as the delimiter; if the data could include commas then OP will (obviously) need to switch to a different delimiter (that doesn't show up in the data)
Then again, if OP's only need for the function is to populate the array, there's really no need for a user-defined function to serve as a wrapper for the awk supplied split() function.
In other words, all of this:
array_new(array1, "1,2,c,4,e")
array_new(array2, "99,98,97")
function array_new(arr, plist) {
split(plist,arr,",")
}
Can be replaced by this:
split("1,2,c,4,e", array1, ",")
split("99,98,97", array2, ",")

Answer 2

Looking solely at the SYMTAB[] issue ...
It doesn't appear that SYMTAB[] is aware of 'local' parameters as defined in the function's parameter list unless they are also referenced in the body of the function, but even then it appears that SYMTAB[<param>] doesn't contain the value of the parameter.
Consider the following:
awk '
function array_new_2(arr, p1, p2, p3, p4, p5, p6, n) {
print "p1",p1,SYMTAB["p1"],typeof(SYMTAB["p1"])
print "p6",p6,SYMTAB["p6"],typeof(SYMTAB["p1"])
print "XX",XX,SYMTAB["XX"],typeof(SYMTAB["XX"])
print "############# pX in SYMTAB ?"
for (i=1;i<=6;i++)
print "p"i ("p"i in SYMTAB ? "" : " not") " in SYMTAB[]; SYMTAB[p"i"]=" SYMTAB["p"i]
print "############# for loop:"
for (i in SYMTAB)
if (isarray(SYMTAB[i]) )
print i,"[ array ]"
else
print i,SYMTAB[i]
print "#############"
arr[1]=p1
}
BEGIN {
OFS=","
array_new_2(arrX,1,2,3) # provide values for p1/p2/p3
}
'
This generates:
p1,1,,unassigned # SYMTAB["p1"] is empty/undefined
p6,,,unassigned # no value supplied for p6, SYMTAB["p6"] is empty/undefined
XX,,,unassigned # undefined variable "XX"
############# pX in SYMTAB ?
p1 in SYMTAB[]; SYMTAB[p1]= # in SYMTAB[] because referenced in body of function, but value not in SYMTAB[] !!
p2 not in SYMTAB[]; SYMTAB[p2]= # value provided but not in SYMTAB[] !!
p3 not in SYMTAB[]; SYMTAB[p3]= # value provided but not in SYMTAB[] !!
p4 not in SYMTAB[]; SYMTAB[p4]=
p5 not in SYMTAB[]; SYMTAB[p5]=
p6 in SYMTAB[]; SYMTAB[p6]= # no value provided but in SYMTAB[] because referenced in body of function
############# for loop:
ARGV,[ array ]
i,i
ROUNDMODE,N
ORS,
OFS,,
LINT,0
FNR,0
ERRNO,
NR,0
IGNORECASE,0
p1, # referenced in body of function; SYMTAB[] is empty/undefined
TEXTDOMAIN,messages
NF,0
ARGIND,0
arrX,[ array ] # without "arr[1]=p1" the variable "arrX" is treated as a scalar and SYMTAB["arrX"] is empty/undefined
XX,
ARGC,1
PROCINFO,[ array ]
FIELDWIDTHS,
CONVFMT,%.6g
SUBSEP,
PREC,53
ENVIRON,[ array ]
RS,
FPAT,[^[:space:]]+
p6, # referenced in body of function; SYMTAB[] is empty/undefined
RT,
RLENGTH,0
OFMT,%.6g
FS,
RSTART,0
FILENAME,
BINMODE,0
#############
NOTES:
p1 (=1) and p6 (empty/undefined) show up in for loop output because they are referenced in the body of function
p2 (=2) and p3 (=3) have values supplied by the function call but they are never referenced in the body of the function soooo, they don't show up in the for loop output
p4(empty/undefined) and p5 (empty/undefined) are never referenced in the body of the function soooo, they also don't show up in the for loop output
From this small test it appears:
SYMTAB[] cannot be used to determine if a function's input parameters were supplied values
SYMTAB[] cannot be used to access the actual values passed in the function call