(Question by John Williams, from a Coursera forum, which I decided to share with the community, since I haven't been able to find this answered anywhere.)
The following code runs without error:
int _j = 1;
//int 2var = 2;
int var2 = 2;
int Kvar = 3; // first letter can be uppercase
int spec$var = 4;
int com_pound_var = 5; // compounding without camel case
int com$pound$two = 6;
int $var = 199;
println(_j);
println(var2);
println(Kvar);
println(spec$var);
println(com_pound_var);
println(com$pound$two);
println($var); //first character can be special
Since the compiler accepts _j, Kvar, and $var as valid variable names, it is clear that variable names do not need to start with a lowercase letter.
I was unable to locate the variable naming rules anywhere in the reference.
What are the variable naming rules for the Processing language?
Quick answer: can start with any letter, underscore and dollar signs, continue with letters, numbers, underscore and dollar signs. Details below.
I could also not find anything in the reference or the documentation at all. However, inspecting the source code, I found that Processing is not a language of its own, but rather a framework in which you run some commands. The difference is that you're actually writing a different language, and Processing just gives you some basic scaffolding where you build on top of.
For some technical details: Processing compiles a Java Build with some flags, spins up a virtual machine (Java VM, not same thing as a full fledged virtual machine) and connects to it to get input and output streams (this is why you can interact with the mouse or get the console output of your own program in a separate window). (Source.)
This language, which you may have guessed already, is Java.
With that said, the actual docs that answer this question is the Java Language Specification, which, to simplify things, is as close as you can get to an answer. (But if you really want to know, it's a mess.)
Specifically, the section on Identifiers, which I'll sum up below:
Can start with any letter (A-Z, a-z), underscore (_), dollar sign ($), or any unicode "letter" (accented, chinese, etc. Details.)
Can continue with any of the above, and can also continue with digits (0-9). Can also contain other unicode "letters" (Details.)
Can have unlimited length
Cannot be any Java keyword (list here)
Cannot be false, true, null
They can look the same and still be different if their codes are different (some Unicode letters look just like letters but are different ones)
I hope this helps! Investigating was fun.
Related
Edit/Update:
Thank you all for responding. I understand I was being too vague, but wasn't sure if posting naked lines of code would be useful in this case.
In my .vb file I have a pulldown control with its validation values as:
TempUnit.DataSource = {"°C", "°F", "°R", "K"}
...which is stored in a variable:
Dim unit As String = TempUnit.SelectedItem.ToString
...which gets passed into a function along with other variables:
Function xxx(..., ByVal unitT As String) As Double
... which finally calls the .fs file and gets evaluated using:
let tempConv t u =
match u with
|"°C" -> t * 9.0 / 5.0 + 32.0
|"°R" -> t - 459.67
|"K" -> t * 9.0 / 5.0 - 459.67
|_ -> t
If any temperature unit other than Kelvin is selected, the match fails and defaults to the else case (which is Fahrenheit in this context). I ended up bypassing the degree symbol entirely by evaluating the substring instead:
Dim unit As String = TempUnit.SelectedItem.ToString.Substring(1)
The program is working again, but I have no idea what I changed, if anything, to make the string match stop working. The first thing I tried was to copy/paste from one file to other to ensure they were identical strings, in addition to trying other symbols, but to no avail. The degree symbol is what caught my attention, but then I checked the pressure units and found the exact same issue with the micro prefix.
Thank you, Hans Passant, I had unicode in mind as a possible solution, but it didn't seem like an easy fix in the heat of the moment. I appreciate your link.
Original Post:
I have a VB program referencing a function stored in an F# library file whose arguments include unit of measure strings containing special characters (e.g. "°C" "µBar").
The strings are identical in the .vb and .fs files; and there was no issue until the F# library file stopped recognizing the Alt-Code characters for reasons unbeknownst to me.
The program works as intended if I remove the offending Alt-Code character from the string definitions in the F# and VB files.
What would cause a match to fail between two identical strings that happen to contain an Alt-Code character?
What is the proper way to handle Alt-Code characters in F# (and VB for that matter)?
The µ glyph is a bit infamous. Unicode has two codepoints that look like that: U+03BC = "Greek small letter Mu" and U+00B5 = "Micro sign". One is a letter in the Greek alphabet, the other is a symbol that often appears in math and units.
Compare μ and µ. Looks almost identical in most fonts (you can see the difference with Segoe UI) and very easily fools the human eye. Typographers insist they are not the same, particularly if they are Greek I'd imagine. Nor does a computer, the problem you are surely dealing with.
Copy/paste or re-type to fix. The Charmap.exe applet in Windows is very handy to get this right.
I've been breaking my head over this for a few days now and can't seem to be able to figure it out. Perhaps it's glaringly obvious, but I don't seem to be able to spot it. I've read up on all the basics of unicode, UTF-8, UTF-16, normalisation, etc, but to no avail. Hopefully somebody's able to help me out here...
I'm using Go's Value function from the testing/quick package to generate random values for the fields in my data structs, in order to implement the Generator interface for the structs in question. Specifically, given a Metadata struct, I've defined the implementation as follows:
func (m *Metadata) Generate(r *rand.Rand, size int) (value reflect.Value) {
value = reflect.ValueOf(m).Elem()
for i := 0; i < value.NumField(); i++ {
if t, ok := quick.Value(value.Field(i).Type(), r); ok {
value.Field(i).Set(t)
}
}
return
}
Now, in doing so, I'll end up with both the receiver and the return value being set with random generated values of the appropriate type (strings, ints, etc. in the receiver and reflect.Value in the returned reflect.Value).
Now, the implementation for the Value function states that it will return something of type []rune converted to type string. As far as I know, this should allow me to then use the functions in the runes, unicode and norm packages to define a filter which filters out everything which is not part of 'Latin', 'Letter' or 'Number'. I defined the following filter which uses a transform to filter out letters which are not in those character rangetables (as defined in the unicode package):
func runefilter(in reflect.Value) (out reflect.Value) {
out = in // Make sure you return something
if in.Kind() == reflect.String {
instr := in.String()
t := transform.Chain(norm.NFD, runes.Remove(runes.NotIn(rangetable.Merge(unicode.Letter, unicode.Latin, unicode.Number))), norm.NFC)
outstr, _, _ := transform.String(t, instr)
out = reflect.ValueOf(outstr)
}
return
}
Now, I think I've tried just about anything, but I keep ending up with a series of strings which are far from the Latin range, e.g.:
𥗉똿穊
𢷽嚶
秓䝏小𪖹䮋
𪿝ท솲
𡉪䂾
ʋ𥅮ᦸ
堮𡹯憨𥗼𧵕ꥆ
𢝌𐑮𧍛併怃𥊇
鯮
𣏲𝐒
⓿ꐠ槹𬠂黟
𢼭踁퓺𪇖
俇𣄃𔘧
𢝶
𝖸쩈𤫐𢬿詢𬄙
𫱘𨆟𑊙
欓
So, can anybody explain what I'm overlooking here and how I could instead define a transformer which removes/replaces non-letter/number/latin characters so that I can use the Value function as intended (but with a smaller subset of 'random' characters)?
Thanks!
Confusingly the Generate interface needs a function using the type not a the pointer to the type. You want your type signature to look like
func (m Metadata) Generate(r *rand.Rand, size int) (value reflect.Value)
You can play with this here. Note: the most important thing to do in that playground is to switch the type of the generate function from m Metadata to m *Metadata and see that Hi Mom! never prints.
In addition, I think you would be better served using your own type and writing a generate method for that type using a list of all of the characters you want to use. For example:
type LatinString string
const latin = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01233456789"
and then use the generator
func (l LatinString) Generate(rand *rand.Rand, size int) reflect.Value {
var buffer bytes.Buffer
for i := 0; i < size; i++ {
buffer.WriteString(string(latin[rand.Intn(len(latin))]))
}
s := LatinString(buffer.String())
return reflect.ValueOf(s)
}
playground
Edit: also this library is pretty cool, thanks for showing it to me
The answer to my own question is, it seems, a combination of the answers provided in the comments by #nj_ and #jimb and the answer provided by #benjaminkadish.
In short, the answer boils down to:
"Not such a great idea as you thought it was", or "Bit of an ill-posed question"
"You were using the union of 'Letter', 'Latin' and 'Number' (Letter || Number || Latin), instead of the intersection of 'Latin' with the union of 'Letter' and 'Number' ((Letter || Number) && Latin))
Now for the longer version...
The idea behind me using the testing/quick package is that I wanted random data for (fuzzy) testing of my code. In the past, I've always written the code for doing things like that myself, again and again. This meant a lot of the same code across different projects. Now, I could of course written my own package for it, but it turns out that, even better than that, there's actually a standard package which does just about exactly what I want.
Now, it turns out the package does exactly what I want very well. The codepoints in the strings which it generates are actually random and not just restricted to what we're accustomed to using in everyday life. Now, this is of course exactly the thing which you want in doing fuzzy testing in order to test the code with values outside the usual assumptions.
In practice, that means I'm running into two problems:
There's some limits on what I would consider reasonable input for a string. Meaning that, in testing the processing of a Name field or a URL field, I can reasonably assume there's not going to be a value like 'James Mc⌢' (let alone 'James Mc🙁') or 'www.🕸site.com', but just 'James McFrown' and 'www.website.com'. Hence, I can't expect a reasonable system to be able to support it. Of course, things shouldn't completely break down, but it also can't be expected to handle the former examples without any problems.
When I filter the generated string on values which one might consider reasonable, the chance of ending up with a valid string is very small. The set of possible characters in the set used by the testing/quick is just so large (0x10FFFF) and the set of reasonable characters so small, you end up with empty strings most of the time.
So, what do we need to take away from this?
So, whilst I hoped to use the standard testing/quick package to replace my often repeated code to generate random data for fuzzy testing, it does this so well that it provides data outside the range of what I would consider reasonable for the code to be able to handle. It seems that the choice, in the end, is to:
Either be able to actually handle all fuzzy options, meaning that if somebody's name is 'Arnold 💰💰' ('Arnold Moneybags'), it shouldn't go arse over end. Or...
Use custom/derived types with their own Generator. This means you're going to have to use the derived type instead of the basic type throughout the code. (Comparable to defining a string as wchar_t instead of char in C++ and working with those by default.). Or...
Don't use testing/quick for fuzzy testing, because as soon as you run into a generated string value, you can (and should) get a very random string.
As always, further comments are of course welcome, as it's quite possible I overlooked something.
My problem is that I have an identical pattern of characters that I wish to parse differently given their context. In one part of the file I need to parse a version number of the form "#.#" Obviously, this is identical to a floating point number. The lexer always opts to return a floating point number. I think I can switch the sequence of rules to give the version # precedence(?), but that doesn't do me any good when I need to later parse floating point numbers.
I suppose I could forget about asking the parser to return each piece of the version separately and split the floating point number into pieces, but I had hoped to have it done for me.
There is actually a bit more to the context of the version #. Its complete form is "ABC #.# XYZ" where "ABC" and "XYZ" never change. I have tried a couple of things to leverage the context of the version #, but haven't made it work.
Is there a way to provide some context to the lexer to parse the components of the version? Am I stuck with receiving a floating point number and parsing it myself?
You have a few possibilities.
The simplest one is to do the string to number conversion in the parser instead of the scanner. That requires making a copy of the number as a string, but the overhead should not be too high: malloc of short strings is well-optimized on almost all platforms. And the benefit is that the code is extremely straightforward and robust:
Parser
%union {
char* string;
double number;
// other types, including version
}
%token <string> DOTTED
%token <number> NUMBER
%type <number> number
%type <version> version
%%
number : NUMBER
| DOTTED { $$ = atod($1); free($1); }
version: DOTTED { $$ = make_version($1); free($1); }
Scanner
[[:digit:]]+\.[[:digit:]]+ { yylval.string = strdup(yytext); return DOTTED; }
[[:digit:]]+\.?|\.[[:digit:]]+ { yylval.number = atod(yytext); }
The above assumes that version numbers are always single-dotted, as in the OP. In applications where version numbers could have multiple dots or non-numeric characters, you would end up with three possible token types: unambiguous numbers, unambiguous version strings, and single-dotted numeric strings which could be either. Aside from adding the VERSION token type and the pattern for unambiguous version strings to the scanner, the only change is to add | VERSION to the version production in the parser.
Another possibility, if you can easily figure out in the scanner whether a number or a version is required, is to use a start condition. You can also change the condition from the parser but it's more complicated and you need to understand the parsing algorithm in order to ensure that you are communicating the state changes correctly.
Finally, you could do both conversions in the scanner, and pick the correct one when you reduce the token in the parser. If the version is a small and simple data structure, this might well turn out to be optimal.
I am studying YACC and the concept of a terminal symbol vs a token keeps coming up. Could someone explain to me what the difference is or point me to an article or tutorial that might help?
They are really two names for the same thing, but usually "terminal" is used to describe what the parser is working with, while "token" is used to describe the corresponding sequence of symbols in the source.
In a parser generator like yacc, the grammar of the language is defined in terms of an "alphabet" of "terminals". The word "alphabet" is a little confusing because they are strings, not letters. But from the parser's perspective, every terminal is an indivisible unit indistinguishable from any other use of the same kind of terminal. So the source code:
total = 17 + subtotal;
will be presented to the parser as something like:
ID EQUALS NUMBER PLUS ID SEMICOLON
There is a correspondence between the stream of terminals which the parser sees and substrings of the input language. So we say that the "token" total is an instance of the "terminal" ID. There may be an unlimited number of potential tokens corresponding to a given terminal (or they may be just one, as with the terminal EQUALS) but what the parser actually works with is a smallish finite set of terminals.
This question already has answers here:
Is there any language that allows spaces in its variable names [closed]
(2 answers)
Closed 9 years ago.
Related: Why can't variable names start with numbers?
Is there a technical reason why spaces aren't allowed in variable names or is it down to convention?
For example, what's stopping us from doing something like this?:
average score = sum of scores / number of scores
The only issue that comes to mind is keywords, but one could simply restrict the use of them in a variable name, and the lexer would be able to distinguish between part of a variable and a keyword.
There’s no fundamental reason, apart from the decisions of language designers and a history of single-token identifiers. Some languages in fact do allow multi-token identifiers: MultiMedia Fusion’s expression language, some Mac spreadsheet/notebook software whose name escapes me, and I’m sure of others. There are several considerations that make the problem nontrivial, though.
Presuming the language is free-form, you need a canonical representation, so that an identifier like account name is treated the same regardless of whitespace. A compiler would probably need to use some mangling convention to please a linker. Then you have to consider the effect of that on foreign exports—why C++ has the extern "C" linkage specifier to disable mangling.
Keywords are an issue, as you have seen. Most C-family languages have a lexical class of keywords distinct from identifiers, which are not context-sensitive. You cannot name a variable class in C++. This can be solved by disallowing keywords in multi-token identifiers:
if account age < 13 then child account = true;
Here, if and then cannot be part of an identifier, so there is no ambiguity with account age and child account. Alternatively, you can require punctuation everywhere:
if (account age < 13) {
child account = true;
}
The last option is to make keywords pervasively context-sensitive, leading to such monstrosities as:
IF IF = THEN THEN ELSE = THEN ELSE THEN = ELSE
The biggest issue is that juxtaposition is an extremely powerful syntactic construct, and you don’t want to occupy it lightly. Allowing multi-token identifiers prevents using juxtaposition for another purpose, such as function application or composition. Far better, I think, just to allow most nonwhitespace characters and thereby permit such identifiers as canonical-venomous-frobnicator. Still plenty readable but with fewer opportunities for ambiguity.
I think it is bacause the designers of the language have followed this convention.
I have searched on Google and found that while naming a variable this is a rule which is followed while naming a variable.
Some links are given below:-
SPSS notes
The following rules apply to variable names:
Variable names cannot contain spaces.
C Programming/Variables
Variable names by IBM
Java Variable Naming convention
Variable names are case-sensitive. A variable's name can be any legal
identifier — an unlimited-length sequence of Unicode letters and
digits, beginning with a letter, the dollar sign "$", or the
underscore character "". The convention, however, is to always begin
your variable names with a letter, not "$" or "". Additionally, the
dollar sign character, by convention, is never used at all. You may
find some situations where auto-generated names will contain the
dollar sign, but your variable names should always avoid using it. A
similar convention exists for the underscore character; while it's
technically legal to begin your variable's name with "_", this
practice is discouraged. White space is not permitted.
Wiki for Naming Convention
In all of the above links you will find that the designers have followed this naming convention for naming the variable.
Also check Is there any language that allows spaces in its variable names
This is forced from language designing.
Compiler needs to find out the meaning of words.
Compiler works on a "State Machine" method, and it needs to distinguish key words.
Maybe placing variable names in "[" and "]" give us some solution(like SQL).
But it will be harder to use it in coding...