I see that the Practical Common Lisp uses (defvar *db* nil) for setting up a global variable. Isn't it OK to use setq for the same purpose?
What are the advantages/disadvantages of using defvar vs. setq?
There are several ways to introduce variables.
DEFVAR and DEFPARAMETER introduce global dynamic variables. DEFVAR optionally sets it to some value, unless it is already defined. DEFPARAMETER sets it always to the provided value.
SETQ does not introduce a variable.
(defparameter *number-of-processes* 10)
(defvar *world* (make-world)) ; the world is made only once.
Notice that you likely never want to DEFVAR variables with names like x, y, stream, limit, ... Why? Because these variables then would be declared special and its difficult to undo that. The special declaration is global and all further uses of the variable would use dynamic binding.
BAD:
(defvar x 10) ; global special variable X, naming convention violated
(defvar y 20) ; global special variable Y, naming convention violated
(defun foo ()
(+ x y)) ; refers to special variables X and y
(defun bar (x y) ; OOPS!! X and Y are special variables
; even though they are parameters of a function!
(+ (foo) x y))
(bar 5 7) ; -> 24
BETTER: Always mark special variables with * in their names!
(defvar *x* 10) ; global special variable *X*
(defvar *y* 20) ; global special variable *Y*
(defun foo ()
(+ *x* *y*)) ; refers to special variables X and y
(defun bar (x y) ; Yep! X and Y are lexical variables
(+ (foo) x y))
(bar 5 7) ; -> 42
Local variables are introduced with DEFUN, LAMBDA, LET, MULTIPLE-VALUE-BIND and many others.
(defun foo (i-am-a-local-variable)
(print i-am-a-local-variable))
(let ((i-am-also-a-local-variable 'hehe))
(print i-am-also-a-local-variable))
Now, by default the local variables in above two forms are lexical, unless they are declared SPECIAL. Then they would be dynamic variables.
Next, there are also several forms to set a variable to new values. SET, SETQ, SETF and others. SETQ and SETF can set both lexical and special (dynamic) variables.
It is required for portable code that one sets variables that are already declared. The exact effect of setting a not declared variable is undefined by the standard.
So, if you know what your Common Lisp implementation does, you can use
(setq world (make-new-world))
in the Read-Eval-Print-Loop at the toplevel. But don't use it in your code, since the effect is not portable. Typically SETQ will set the variable. But some implementation might also declare the variable SPECIAL when it doesn't know it (CMU Common Lisp does that by default). That's almost always not what one would want. Use it for casual use if you know what you do, but not for code.
Same here:
(defun make-shiny-new-world ()
(setq world (make-world 'shiny)))
First, such variables should be written as *world* (with the surrounding * characters), to make clear that it is a global special variable. Second, it should have been declared with DEFVAR or DEFPARAMETER before.
A typical Lisp compiler will complain that above variable is undeclared. Since global lexical variables don't exist in Common Lisp, the compiler has to generate code for a dynamic lookup. Some compiler then say, okay, we assume that this is a dynamic lookup, let's declare it to be special - since that is what we assume anyway.
defvar introduces a dynamic variable while setq is used to assign a value to a dynamic or lexical variable. The value of a dynamic variable is looked up in the environment that calls the function, while the value of a lexical variable is looked up in the environment where the function was defined. The following example will make the difference clear:
;; dynamic variable sample
> (defvar *x* 100)
*X*
> (defun fx () *x*)
FX
> (fx)
100
> (let ((*x* 500)) (fx)) ;; gets the value of *x* from the dynamic scope.
500
> (fx) ;; *x* now refers to the global binding.
100
;; example of using a lexical variable
> (let ((y 200))
(let ((fy (lambda () (format t "~a~%" y))))
(funcall fy) ;; => 200
(let ((y 500))
(funcall fy) ;; => 200, the value of lexically bound y
(setq y 500) ;; => y in the current environment is modified
(funcall fy)) ;; => 200, the value of lexically bound y, which was
;; unaffected by setq
(setq y 500) => ;; value of the original y is modified.
(funcall fy))) ;; => 500, the new value of y in fy's defining environment.
Dynamic variables are useful for passing around a default value. For instance, we can bind the dynamic variable *out* to the standard output, so that it becomes the default output of all io functions. To override this behavior, we just introduce a local binding:
> (defun my-print (s)
(format *out* "~a~%" s))
MY-PRINT
> (my-print "hello")
hello
> (let ((*out* some-stream))
(my-print " cruel ")) ;; goes to some-stream
> (my-print " world.")
world
A common use of lexical variables is in defining closures, to emulate objects with state. In the first example, the variable y in the binding environment of fy effectively became the private state of that function.
defvar will assign a value to a variable only if it is not already assigned. So the following re-definition of *x* will not change the original binding:
> (defvar *x* 400)
*X*
> *x*
100
We can assign a new value to *x* by using setq:
> (setq *x* 400)
400
> *x*
400
> (fx)
400
> (let ((*x* 500)) (fx)) ;; setq changed the binding of *x*, but
;; its dynamic property still remains.
500
> (fx)
400
DEFVAR establishes a new variable. SETQ assigns to a variable.
Most Lisp implementations I've used will issue a warning if you SETQ to a variable that doesn't yet exist.
defvar and defparameter both introduce global variables. As Ken notes, setq assigns to a variable.
In addition, defvar will not clobber something previously defvar-ed. Seibel says later in the book (Chapter 6): "Practically speaking, you should use DEFVAR to define variables that will contain data you'd want to keep even if you made a change to the source code that uses the variable."
http://www.gigamonkeys.com/book/variables.html
For instance, if you have a global *db* for the database in the Simple Database chapter:
(defvar *db* nil)
...and you start playing with it at the REPL - adding, deleting things, etc - but then you make a change to the source file which contains that defvar form, reloading that file will not wipe out *db* and all the changes you might have made... I believe that setq will, as will defparameter. A more experienced Lisper please correct me if I'm wrong though.
Related
Common Lisp is lexically scoped, but there is a possibility to create dynamic bindings with (declare (special *var*)). What I need, is a way to create a dynamically scoped structure slot, whose value is visible to all other slots. Here is an example:
(defun start-thread ()
*delay*) ;; We defer the binding of *delay*
This works for a usual lexical environment:
(let ((*delay* 1))
(declare (special *delay*))
(start-thread)) ;; returns 1
This does not work:
(defstruct table
(*delay* 0)
(thread (start-thread)))
(make-table) ;; => Error: *delay* is unbound.
My questions are
How to refer to the slot delay from other slots?
How to make the slot delay dynamically scoped, so that its value becomes visible
for the function (start-thread) ?
The first thing to realise that there's no good way to have a dynamically-scoped slot in an object (unless the implementation has some deep magic to support this): the only approach that will work is to use, essentially, explicit shallow-binding. Something like this macro, for instance (this has no error checking at all: I just typed it in):
(defmacro with-horrible-shallow-bound-slots ((&rest slotds) object &body forms)
(let ((ovar (make-symbol "OBJECT"))
(slot-vars (mapcar (lambda (slotd)
(make-symbol (symbol-name (first slotd))))
slotds)))
`(let ((,ovar ,object))
(let ,(mapcar (lambda (v slotd)
`(,v (,(first slotd) ,ovar)))
slot-vars slotds)
(unwind-protect
(progn
(setf ,#(mapcan (lambda (slotd)
`((,(first slotd) ,ovar) ,(second slotd)))
slotds))
,#forms)
(setf ,#(mapcan (lambda (slotd slot-var)
`((,(first slotd) ,ovar) ,slot-var))
slotds slot-vars)))))))
And now if we have some structure:
(defstruct foo
(x 0))
Then
(with-horrible-shallow-bound-slots ((foo-x 1)) foo
(print (foo-x foo)))
expands to
(let ((#:object foo))
(let ((#:foo-x (foo-x #:object)))
(unwind-protect
(progn (setf (foo-x #:object) 1) (print (foo-x foo)))
(setf (foo-x #:object) #:foo-x))))
where all the gensyms with the same name are in fact the same. And so:
> (let ((foo (make-foo)))
(with-horrible-shallow-bound-slots ((foo-x 1)) foo
(print (foo-x foo)))
(print (foo-x foo))
(values))
1
0
But this is a terrible approach because shallow binding is terrible in the presence of multiple threads: any other thread that wants to look at foo's slots will also see the temporary value. So this is just horrid.
A good approach is then to realise that while you can't safely dynamically-bind a slot in an object, you can dynamically bind a value which that slot indexes by using a secret special variable to hold a stack of bindings. In this approach the values of slots do not change, but the values they index do, and can do so safely in the presence of multiple threads.
A way of doing this this is Tim Bradshaw's fluids toy. The way this works is that you define the value of a slot to be a fluid, and then you can bind that fluid's value, which binding has dynamic scope.
(defstruct foo
(slot (make-fluid)))
(defun outer (v)
(let ((it (make-foo)))
(setf (fluid-value (foo-slot it) t) v) ;set global value
(values (fluid-let (((foo-slot it) (1+ (fluid-value (foo-slot it)))))
(inner it))
(fluid-value (foo-slot it)))))
(defun inner (thing)
(fluid-value (foo-slot thing)))
This often works better with CLOS objects because of the additional flexibility in things like naming and what you expose (you almost never want to be able to assign to a slot whose value is a fluid, for instance: you want to assign the value of the fluid).
The system uses a special variable behind the scenes to implement deep binding for fluids, so will work properly with threads (ie distinct threads can have different bindings for a fluid) assuming the implementation treats special variables sensibly (which I'm sure all multithreaded implementations do).
I don't think that this makes sense. Variables have scope and extent, but values just are, and slots are just parts of values. Additionally, threads do not inherit dynamic bindings.
If you want to have some kind of object that is dynamically changed (so to speak), you need to put it into a dynamic variable as a whole value, and do re-bindings with modified versions (preferably on the basis of some immutability, i. e. persistent datastructures, e. g. with FSet).
I'm doing a bit of guessing about what you need here, but I think using a class and initialize-instance will give you what you want. In the code below, I rewrote your struct as a class, and the object itself is passed to initialize-instance in a call to (make-instance 'table).
(defclass table ()
((delay :initform 5)
(thread)))
(defun start-my-thread (obj)
(print (slot-value obj 'delay)))
(defmethod initialize-instance :after ((obj table) &key)
(start-my-thread obj))
(make-instance 'table)
; above call will print 5
I want to define a function (not a macro) that can set a variable in the scope its called.
I have tried:
(defun var-set (var-str val)
(let ((var-interned
(intern (string-upcase var-str))))
(set var-interned val)
))
(let ((year "1400"))
(var-set "year" 1388)
(labeled identity year))
Which doesn't work because of the scoping rules. Any "hacks" to accomplish this?
In python, I can use
previous_frame = sys._getframe(1)
previous_frame_locals = previous_frame.f_locals
previous_frame_locals['my-var'] = some_value
Any equivalent API for lisp?
You cannot do that because after compilation the variable might not even exist in any meaningful sense.
E.g., try to figure out by looking at the output of (disassemble (lambda (x) (+ x 4))) where you
would write the new values of x.
You have to tell both the caller and the callee (at compile time!) that the variable is special:
(defun set-x (v)
(declare (special x))
(setq x v))
(defun test-set (a)
(let ((x a))
(declare (special x))
(set-x 10)
x))
(test-set 3)
==> 10
See Dynamic and Lexical variables in Common Lisp for further details on lexical vs dynamic bindings.
You can't. This is why it is called lexical scope: you have access to variable bindings if and only if you can see them. The only way to get at such a binding is to create some object for which it is visible and use that. For instance:
(defun foo (x)
(bar (lambda (&optional (v nil vp)
(if vp (setf x vp) x))))
(defun bar (a)
...
(funcall a ...))
Some languages, such as Python have both rather rudimentary implementations of variable bindings and a single implementation (or a mandated implementation) which allow you to essentially poke around inside the system to subvert lexical scoping. Some CL implementations may have rudimentary implementation of variable bindings (probably none do) but Common Lisp the language does not mandate such implementations and nor should it.
As an example of the terrible consequences of mandating that some kind of access to lexical variables must be allowed consider this:
(defun outer (n f)
(if (> n 0)
(outer (g n) f)
(funcall f)))
If f could somehow poke at the lexical bindings of outer this would mean that all those bindings would need to exist at the point f was called: tail-call elimination would thus be impossible. If the language mandated that such bindings should be accessible then the language is mandating that tail-call elimination is not possible. That would be bad.
(It is quite possible that implementations, possibly with some debugging declarations, allow such access in some circumstances. That's very different than the language mandating such a thing.)
What are you trying to achieve?
What about a closure? Write the defun inside the let, and create an accessor and a setter function if needed.
(let ((year "1400"))
(defun current-year ()
year)
(defun set-year (val)
(setf year val)))
and
CL-USER> (current-year)
"1400"
CL-USER> (set-year 1200)
1200
CL-USER> (current-year)
1200
That Python mechanism violates the encapsulation which motivates the existence of lexical scope.
Because a lexical scope is inaccessible by any external means other than invocations of function bodies which are in that scope, a compiler is free to translate a lexical scope into any representation which performs the same semantics. Variables named in the source code of the lexical scope can disappear entirely. For instance, in your example, all references to year can be replaced by copies of the pointer to the "1400" string literal object.
Separately from the encapsulation issue there is also the consideration that a function does not have any access at all to a parent lexical scope, regardless of that scope's representation. It does not exist. Functions do not implicitly pass their lexical scope to children. Your caller may not have a lexical environment at all, and there is no way to know. The essence of the lexical environment is that no aspect of it is passed down to children, other than via the explicit passage of lexical closures.
Python's feature is poorly considered because it makes programs dependent on the representation of scopes. If a compiler like PyPy is to make that code work, it has to constrain its treatment of lexical scopes to mimic the byte code interpreted version of Python.
Firstly, each function has to know who called it, so it has to receive some parameter(s) about that, including a link to the caller's environment. That's going to be a source of inefficiency even in code that doesn't take advantage of it.
The concept of a well-defined "previous frame" means that the compiler cannot merge together frames. Code which expects some variable to be in the third frame up from here will break if those frames are all inlined together due to a nested lexical scope being flattened, or due to function inlining.
As soon as you provide an interface to the parent lexical environment, and applications start using it, you no longer have lexical scoping. You have a form of dynamic scoping with lexical-like visibility rules.
The application logic can implement de facto dynamic scope on top of this API, because you can write a loop which searches for a variable across the chain of lexical scopes. Does my parent have an x variable? If not, does the grandparent, if there is one? You can search the dynamic chain of invocations for the most recent one which binds x, and that is dynamic scope.
There is nothing wrong with dynamic scope, if it is a separate discipline that is not entangled in the implementation of lexical scope.
That said, an API for tracing frames and getting at local variables is is the sort of introspection that is very useful in developing a debugger. Another angle on this is that if you work that API into an application, you're using debugging features in production.
(defvar *lexical-variables* '())
(defun get-var (name)
(let ((var (cdr (assoc name *lexical-variables*))))
(unless var (error "No lexical variable named ~S" name))
var))
(defun deref (var)
(funcall (if (symbolp var)
(or (cdr (assoc var *lexical-variables*))
(error "No lexical variable named ~S" var))
var)))
(defun (setf deref) (new-value var)
(funcall (if (symbolp var)
(or (cdr (assoc var *lexical-variables*))
(error "No lexical variable named ~S" var))
var)
new-value))
(defmacro with-named-lexical-variable ((&rest vars) &body body)
(let ((vvar (gensym))
(vnew-value (gensym))
(vsetp (gensym)))
`(let ((*lexical-variables* (list* ,#(mapcar (lambda (var)
`(cons ',var
(lambda (&optional (,vnew-value nil ,vsetp))
(if ,vsetp
(setf ,var ,vnew-value)
,var))))
vars)
*lexical-variables*)))
,#body)))
(defun var-set (var-str val)
(let ((var-interned (intern (string-upcase var-str))))
(setf (deref var-interned) val)))
(let ((x 1)
(y 2))
(with-named-lexical-variable (x y)
(var-set "x" 3)
(setf (deref 'y) 4)
(mapcar (function deref) '(x y))))
;; -> (3 4)
(let ((year "1400"))
(with-named-lexical-variable (year)
(var-set "year" 1388))
year)
;; --> 1388
I hope this isn't beating a dead horse, but I'd like an opinion about another possible strategy for writing referentially transparent code. (The previous discussion about referential transparency is at Using a Closure instead of a Global Variable). Again, the objective is to eliminate most global variables, but retain their convenience, without injecting bug-prone references or potentially non-functional behavior (ie, referential opaqueness, side-effects, and non-repeatable evaluations) into the code.
The proposal is to use local special variables to establish initial bindings, which can then be passed dynamically to the subsequent nested functions that eventually use them. The intended advantage, like globals, is that the local special variables do not need to be passed as arguments through all the intermediate functions (whose functionality has nothing to do with the local special variables). However to maintain referential transparency, they would be passed as arguments to the final consumer functions.
What I'm wondering about is whether floating a lot of dynamic variables around is prone to programming bugs. It doesn't seem particularly error prone to me, since any local rebinding of a previously bound variable should not affect the original binding, once it is released:
(defun main ()
(let ((x 0))
(declare (special x))
(fum)))
(defun fum ()
(let ((x 1)) ;inadvertant? use of x
(setf x 2))
(foo))
(defun foo ()
(declare (special x))
(bar x))
(defun bar (arg) ;final consumer of x
arg)
(main) => 0
Are there problems with this stragegy?
Now your functions are referencing a variable that is not guaranteed to be defined. Trying to execute (foo) at the repl will throw an unbound variable error. Not only is there referential opacity, but now referential context error throwing!
What you have here are globally bound routines, which can only be executed in the local context where (declare (special x)) has been hinted. You may as well put those functions in a labels so they don't get accidentally used, though at that point you are choosing between closing the variables in functions, or closing the functions in a function:
(defun main ()
(labels ((fum ()
(let ((x 1));Inadvertent use of x?
(setf x 2))
(foo))
(foo ()
(declare (special x))
(bar x))
(bar (arg) arg)) ;Final consumer of x.
(let ((x 0))
(declare (special x))
(fum))))
Wow, that is some ugly code!
After a convolution we can make x lexical! Now we can achieve the holy grail, referential transparency!
Convolute
(defun main ()
(let ((x 0))
(labels ((fum ()
(let ((x 1))
(setf x 2))
(foo))
(foo () (bar x))
(bar (arg) arg));Final consumer of x.
(fum))))
This code is much nicer, and lispy. It is essentially your code to the other question, but the functions bindings are localized. This is at least better than using explosive global naming. The inner let does nothing, same as before. Though now it is less convoluted.
CL-USER> (main) ;=> 0
Your test case is the same (main) ;=> 0 in both. The principle is to just encapsulate your variables lexially instead of with dynamic special declarations. Now we can reduce the code even more by just passing things functionally in a single environment variable, as suggested.
(defun convoluted-zero ()
(labels ((fum (x)
(let ((x 1))
(setf x 2))
(foo x))
(foo (x) (bar x))
(bar (arg) arg)).
(fum 0)))
CL-USER> (let ((x (convoluted-zero)))
(list x (convoluted-zero)))
;=> 0
□ QED your code with the special variables violates abstraction.
If you really want to go down the rabbit hole, you can read the section of chapter 6 of Doug Hoyte's Let Over Lambda on pandoric macros, where you can do something like this:
(use-package :let-over-lambda)
(let ((c 0))
(setf (symbol-function 'ludicrous+)
(plambda () (c) (incf c)))
(setf (symbol-function 'ludicrous-)
(plambda () (c)(decf c))))
You can then use pandoric-get to get c without incrementing it or defining any accessor function in that context, which is absolute bonkers. With lisp packages you can get away with a package-local "global" variable. I could see an application for this in elisp, for example, which has no packages built in.
If I have a function that accepts function argument, for optimization purposes I can declare it to be a function, let's say
(defun foo (f)
(declare (type function f))
...)
However, I can be even more specific:
(defun foo (f)
(declare (type (function (double-float) double-float) f))
...)
i.e. telling that f will accept one double-float argument and return one double-float value. SBCL, however, seem to be able to perform a better optimization on the former and for the latter it says that it doesn't know if f is fdefinition (try to compile with (optimize (speed 3)) declaration to reproduce).
So, my questions are:
Am I doing something wrong? Especially If SBCL would do exactly the same thing for just function and (function ...) I would be OK with it, but it actually does worse. Or should it be considered a bug in SBCL?
Is function type declaration in general useless in CL in terms of optimization for some reason?
SysInfo: SBCL 1.3.18
From the SBCL Manual (4.2.3 Getting Existing Programs to Run):
Some incorrect declarations can only be detected by run-time type
checking [...] because the SBCL compiler does much more
type inference than other Common Lisp compilers, so an incorrect
declaration can do more damage.
It's possible, that's why your function does worse with the variable type declarations included.
Further:
The most common problem is with variables whose constant initial value
doesn't match the type declaration. Incorrect constant initial values
will always be flagged by a compile-time type error, and they are
simple to fix once located. Consider this code fragment:
(prog (foo)
(declare (fixnum foo))
(setq foo ...)
...)
Here foo is given an initial value of nil, but is declared to be a fixnum. Even if it is never read, the initial value of a
variable must match the declared type. There are two ways to fix this
problem. Change the declaration
(prog (foo)
(declare (type (or fixnum null) foo))
(setq foo ...)
...)
or change the initial value
(prog ((foo 0))
(declare (fixnum foo))
(setq foo ...)
...)
This is from the manual for the current version of SBCL (1.4), so it may or may not apply to your situation.
EDIT: I changed the example code after the first answer because I came up with a simple version that begs the same questions.
I am currently learning Common Lisp's scoping properties. After I thought I had a solid understanding I decided to code up some examples that I could predict the outcome of, but apparently I was wrong. I have three question, each one relating to an example below:
Example 1:
(defmethod fun1 (x)
(print x)
(fun2))
(defmethod fun2 ()
(print x))
(fun1 5)
Output:
5
*** - EVAL: variable X has no value
Question: This makes sense. x is statically scoped and fun2 has no way of finding the value of x without having it passed explicitly.
Example 2:
(defvar x 100)
(defmethod fun1 (x)
(print x)
(fun2))
(defmethod fun2 ()
(print x))
(fun1 5)
Output:
5
5
Question: I don't understand why x is suddenly visible to fun2 with the value that fun1 gave it, instead of having a value of 100...
Example 3:
(setf x 100)
(defmethod fun1 (x)
(print x)
(fun2))
(defmethod fun2 ()
(print x))
(fun1 5)
Output:
5
100
Question: Should I ignore these results since calling setf on an undeclared variable is apparently undefined? This happens to be what I would expect in my second example...
Any insight would be greatly appreciated...
The effects of setting an undefined variable using setf is undefined in ANSI Common Lisp.
defvar will define a special variable. This declaration is global and also has effect on let bindings. That's the reason that by convention these variables are written as *foo*. If you have ever defined x with defvar, it is declared special and there is no way to declare it lexical later.
let by default provides local lexical variables. If the variable was already declared special (for example because of a defvar), then it just creates a new local dynamic binding.
Update
Example 1 .
Nothing to see.
Example 2
x has been declared special. All uses of the variable x now use dynamic binding.
When calling the function, you bind x to 5. Dynamically. Other functions can now access this dynamic binding and get that value.
Example 3
This is undefined behavior in Common Lisp. You are setting an undeclared variable. What happens then is implementation dependent. Your implementation (most do something similar) sets the symbol value of x to 100. In fun1, x is lexically bound. In fun2 evaluating x retrieves the symbol value (or possibly to the dynamically bound value) of x.
As an example for an implementation that did (does?) something else: the CMUCL implementation would also have declare x in example 3 by default to be special. Setting an undefined variable also declares it special.
NOTE
In portable standard compliant Common Lisp code the global variables are defined with defvar and defparameter. Both declare these variables to be special. ALL uses of these variables now involve dynamic binding.
Remember:
((lambda (x)
(sin x))
10)
is basically the same as
(let ((x 10))
(sin x))
Which means that variable bindings in let bindings and variable bindings in function calls are working the same way. If x would have been declared special in some place earlier, both would involve dynamic binding.
This is specified in the Common Lisp standard. See for example the explanation to the SPECIAL declaration.