sbcl: additional documentation for (for example) #'sb-ext:string-to-octets - documentation

I recently asked a question about sbcl for which a response mentioned two functions of which I wasn't aware: #'sb-ext:string-to-octets and #'sb-ext:octets-to-string. In addition to answering my question, this also taught me that I should browse through the external symbols of package sb-ext to see what else might be of use.
My question (sbcl-related) is this: other than browsing through the external symbols of package sb-ext, is there some other manual which describes package sb-ext, and other additions (I'm trying to avoid the word "extension" because it's a specific technical term) to sbcl? #'sb-ext:string-to-octets and #'sb-ext:octets-to-string, for example, are not discussed in the sbcl manual.

As pointed in one answer in the other question by #svante, for things like that I prefer to use another libray for postabiliyt and usually well documented like babel.
Normally for checking doc in common lisp if the symbol is in the ansi common lisp, you should check teh clhs sly and slime has an excelent facility for that.
And normally I proceed as follow:
CL-USER> (documentation 'sb-ext:octets-to-string 'function)
NIL
CL-USER> (describe 'sb-ext:octets-to-string)
SB-EXT:OCTETS-TO-STRING
[symbol]
OCTETS-TO-STRING names a compiled function:
Lambda-list: (VECTOR &KEY (EXTERNAL-FORMAT DEFAULT) (START 0) END)
Derived type: (FUNCTION
((VECTOR (UNSIGNED-BYTE 8)) &KEY (:EXTERNAL-FORMAT T)
(:START T) (:END T))
*)
Source file: SYS:SRC;CODE;OCTETS.LISP
; No values
The describe function always get you relevant info about the symbol, then you can go to teh source using sly or slime is with M-.
(defun octets-to-string (vector &key (external-format :default) (start 0) end)
(declare (type (vector (unsigned-byte 8)) vector))
(with-array-data ((vector vector)
(start start)
(end end)
:check-fill-pointer t)
(declare (type (simple-array (unsigned-byte 8) (*)) vector))
(let ((ef (maybe-defaulted-external-format external-format)))
(funcall (ef-octets-to-string-fun ef) vector start end))))
and finally you can go to teh repository for reading tests in this case going to the github SBCL repo and looking for this fucntion gives source code tests that you can read for getting an easy usage of the function:
https://github.com/sbcl/sbcl/search?utf8=%E2%9C%93&q=string-to-octets&type=
like this:
https://github.com/sbcl/sbcl/blob/622c9daf9bb41ef9ad4b8a063c62c4baf59a1c1a/tests/octets.pure.lisp

Related

How can I use types so that generic operations are inlined (or "open coded") in sbcl?

SBCL compiler optimizations are based on the idea that if a type is declared, then "open coding" allows generic operations to be replaced with specific ones.
For example
(defun add (a b)
(declare (type fixnum a b))
(+ a b))
Will allow the generic + to be replaced with a single instruction for fixnum.
However, I have found that in practice, this seems to rarely be possible because:
In order for a function to be specialized/optimized it must be inlinable. The declaration must be marked explicitly with a (declaim (inline ...)), so the author of a function must anticipate that others might want to inline it. (In theory the compiler could generate multiple versions, but this doesn't seem to be the case.)
Most standard functions do not appear inlineable.
For example, one would expect that the following declaration is sufficient for open coding to take place:
(defun max-integers (array)
(declare (optimize (speed 3) (space 0) (safety 0)))
(declare (inline reduce))
(declare (type (simple-array fixnum (*)) array))
(reduce (lambda (a b) (if (> b a) b a)) array))
However, the assembly shows it's making a function call to the generic reduce:
; Size: 22 bytes. Origin: #x1001BC8109
; 09: 488B15B0FFFFFF MOV RDX, [RIP-80] ; no-arg-parsing entry point
; #<FUNCTION (LAMBDA
; # ..)>
; 10: B904000000 MOV ECX, 4
; 15: FF7508 PUSH QWORD PTR [RBP+8]
; 18: B8781C3220 MOV EAX, #x20321C78 ; #<FDEFN REDUCE>
; 1D: FFE0 JMP RAX
The conclusion seems to be that the compiler cannot actually do much type optimization, as each usage of reduce, map, etc is a barrier to type propagation, and they are building blocks of everything else.
How can I overcome this and take advantage of optimizations by declaring types?
I really want to avoid writing type specific versions of each function or "macroifying" what should be a function.
I think one answer is that if you want to write FORTRAN-style array-bashing code, write FORTRAN-style array-bashing code. In particular using things like reduce is probably not the way to do this.
For instance if you change your function to the perfectly readable
(defun max-integers/loop (array)
(declare (optimize (speed 3) (space 0) (safety 0))
(type (simple-array fixnum (*)) array))
(loop for i of-type fixnum across array
maximizing i))
Then SBCL does a far, far better job of optimising it.
It's worth pointing out another confusion in your question: You say that for something like
(defun add (a b)
(declare (type fixnum a b))
(+ a b))
SBCL will optimize + to the machine instruction. No, it won't. The reason it won't is because the fixnum type is not closed under addition: consider what (add most-positive-fixnum 1) should do. If you want to generate very fast code for integers you need to make sure that your integer types are small enough that the compiler can be sure that the operations you're doing on them remain machine integers (or, if you want to live dangerously, cover your code with (the fixnum ...) and set safety to 0 when compiling, which seems to allow the compiler to just return the wrong answer for addition in the way people usually expect computers to do).
You can't force the implementation to open-code functions that weren't declared INLINE when they were defind -- it simply hasn't saved the information needed.
However, the overhead of calling REDUCE is probably negligible compared to the actual processing. So what you can do is declare the types of a and b, to optimize the callback function.
(reduce (lambda (a b) (declare (type fixnum a b)) (if (> b a) b a)) array)
I guess you were hoping that if it open-coded reduce it would automatically propagate this type from the declaration of array, so you wouldn't need to do this.

Optimization for accessing array in lisp

I am trying to learn how to make type declarations in lisp. I figured out that aref causes problems:
(defun getref (seq k)
(declare (optimize (speed 3) (safety 0)))
(declare (type (vector fixnum *) seq) (type fixnum k))
(aref seq k))
Compiled, it says:
; in: DEFUN GETREF
; (AREF MORE-LISP::SEQ MORE-LISP::K)
; ==>
; (SB-KERNEL:HAIRY-DATA-VECTOR-REF ARRAY SB-INT:INDEX)
;
; note: unable to
; avoid runtime dispatch on array element type
; due to type uncertainty:
; The first argument is a (VECTOR FIXNUM), not a SIMPLE-ARRAY.
;
; compilation unit finished
; printed 1 note
And so in every other function, where I want to use aref (and I do, since I need adjustable vectors), this happens too. How do I fix it?
It's not a problem and not an error. It just an information (a note) from the SBCL compiler that it can't optimize the code better. The code will work just fine. You can safely ignore it.
If you can't use a simple vector (a one-dimensional simple array), then this is the price to pay for it: aref might be slightly slower.
The optimization hint you get comes from the docstring of a deftransform defined in sbcl/src/compiler/generic/vm-tran.lisp:
(deftransform hairy-data-vector-ref ((array index) (simple-array t) *)
"avoid runtime dispatch on array element type"
...)
It has a comment which says:
This and the corresponding -SET transform work equally well on non-simple
arrays, but after benchmarking (on x86), Nikodemus didn't find any cases
where it actually helped with non-simple arrays -- to the contrary, it
only made for bigger and up to 100% slower code.
The code for arrays is quite complex and it is hard to say why and how things are designed as they are. You should probably ask SBCL developers on sbcl-help. See the mailing lists section on
Sourceforge.
Currently it seems preferable to favor simple arrays if possible.

Emacs function returns Symbol's value as variable is void:

I am fairly new to Emacs but I know enough to be dangerous. I've built my .emacs file from scratch and now have it in an org file. I am now trying to take it to the next level and make my configuration more user friendly for myself.
I mostly use Emacs for writing. Books, blogs, screenwriting, etc. I am trying to create a function that will turn on multiple modes and add the settings on the fly.
For example, I use olivetti-mode when writing. It centers the text. Each time I have to adjust the olivetti-set-width. I thought I would get fancy and enable the spell checker and turn off linum-mode as well.
However, every time I try it I get the error:
Symbol's value as variable is void: my-writing
Can anyone explain what I am doing wrong? I've google-fu'd quite a bit but I clearly have a gap in my understanding of what I am doing.
#+BEGIN_SRC emacs-lisp
(defun my-writing ()
"Start olivetti mode, set the width to 120, turn on spell-check."
((interactive)
(olivetti-mode)
(setq olivetti-set-width . 120)
(flyspell-mode)
(global-linum-mode 0)))
(add-hook 'olivetti-mode-hook
(lambda () olivetti-mode my-writing t))
#+END_SRC
To disable global-linum-mode for specific major-modes, see automatically disable a global minor mode for a specific major mode
[Inasmuch as olivetti-mode is a minor-mode that is enabled subsequent to whatever major-mode is already present in the buffer, the original poster may wish to turn off linum-mode locally in the current buffer by adding (linum-mode -1) to the tail end of the function my-writing (see below). That idea, however, assumes that the original poster wanted to have linum-mode active in the current buffer just prior to calling my-writing.]
The function my-writing in the initial question contains an extra set of parenthesis that should be omitted, and the hook setting is not in proper form.
olivetti-set-width is a function that takes one argument, so you cannot use setq -- see function beginning at line 197: https://github.com/rnkn/olivetti/blob/master/olivetti.el setq is used when setting a variable, not a function.
Although flyspell-mode is generally buffer-local, it is a good idea to get in the habit of using an argument of 1 to turn on a minor-mode or a -1 or 0 to turn it off. When an argument is omitted, calling the minor-mode works as an on/off toggle.
Unless there are other items already attached to the olivetti-mode-hook that require prioritization or special reasons for using a hook with buffer-local settings, you do not need the optional arguments for add-hook -- i.e., APPEND and LOCAL.
There is no apparent reason to call (olivetti-mode) as part of the olivetti-mode-hook that gets called automatically at the tail end of initializing the minor-mode, so there is now a check to see whether that mode has already been enabled. The olivetti-mode-hook is being included in this example to demonstrate how to format its usage. However, the original poster should consider eliminating (add-hook 'olivetti-mode-hook 'my-writing) as it appears to serve no purpose if the user will be calling M-x my-writing instead of M-x olivetti-mode. The hook would be useful in the latter circumstance -- i.e., when typing M-x olivetti-mode -- in which case, there is really no need to have (unless olivetti-mode (olivetti-mode 1)) as part of my-writing.
#+BEGIN_SRC emacs-lisp
(defun my-writing ()
"Start olivetti mode, set the width to 120, turn on spell-check."
(interactive)
(unless olivetti-mode (olivetti-mode 1))
(linum-mode -1) ;; see comments above
(olivetti-set-width 120)
(flyspell-mode 1))
;; original poster to consider eliminating this hook
(add-hook 'olivetti-mode-hook 'my-writing)
#+END_SRC
lawlist's answer describes how you can go about doing what you're actually trying to accomplish, but the particular error you're getting is because Emacs Lisp (like Common Lisp, but not Scheme) is a Lisp-2. When you associate a symbol with a function using defun, it doesn't make the value of that symbol (as a variable) that function, it makes the function value of that symbol the function. You'll get the same error in a much simplified situation:
(defun foo ()
42)
(list foo)
The symbol foo has no value here as a variable. To get something that you could later pass to funcall or apply, you need to either use the symbol foo, e.g.:
(funcall 'foo)
;=> 42
or the form (function foo):
(funcall (function foo))
;=> 42
which can be abbreviated with the shorthand #':
(funcall #'foo)
;=> 42
You're getting the error because of:
(add-hook 'olivetti-mode-hook
(lambda () olivetti-mode my-writing t))
which tries to use my-writing as a variable, but it has no variable value at that point.

Can't declare constant variable locally Error installing Artificial Intelligence a Modern Approach Code on Ubuntu Trusty Tahr - Common Lisp

Im installing the code for the book Artificial Intelligence a Modern Approach i got here http://aima.cs.berkeley.edu/lisp/doc/install.html its the lisp version im installing btw
I'm on Ubuntu Trusty using Emacs SBCL slime, I placed the code in ~/.emacs.d
so per the instructions at above link i run (load "/home/w/.emacs.d/aima/code/aima.lisp")
which loads fine i get "T" as output
i run (aima-load 'all) that works I get "T" as output
but when i run (aima-compile) I get the error
Can't declare constant variable locally special: +NO-BINDINGS+
[Condition of type SIMPLE-ERROR]
I'm not sure I understand the error I read the Hyperspec on Declare and Special but that didn't help. I'm not opposed to a hack to make this work so if someone can help me reword the code or figure out if its a emacs setting or sbcl setting i could change to get the aforementioned variable declared that would be great. The error message is referring to this file /home/w/.emacs.d/aima/code/logic/algorithms/tell-ask.lisp
this section
(defmethod ask-each ((kb literal-kb) query fn)
"For each proof of query, call fn on the substitution that
the proof ends up with."
(declare (special +no-bindings+))
(for each s in (literal-kb-sentences kb) do
(when (equal s query) (funcall fn +no-bindings+))))
I verified all the permisions of all the files in my aima folder are set to read write for my username with my username as owner as a step to correct....but as far as understanding the error I could use help figuring out the next step one would take to debug this...The code is downloadable here http://aima.cs.berkeley.edu/lisp/code.tar.gz and its an easy install for a veteran emacs/lisp user....any help is appreciated.
Using SBCL:
I tried loading the AIMA code as well, with the same problem in SBCL. Just comment out line in ./logic/algorithms/tell-ask.lisp:
(declare (special +no-bindings+))
like so
;;(declare (special +no-bindings+))
, or delete the whole line altogether. Nevertheless, this is what I did:
(defmethod ask-each ((kb literal-kb) query fn)
"For each proof of query, call fn on the substitution that
the proof ends up with."
;;(declare (special +no-bindings+))
(for each s in (literal-kb-sentences kb) do
(when (equal s query) (funcall fn +no-bindings+))))
You will still have an issue running (aims-compile) with SBCL. It will complain about some constants being redefined. Just look at the possible restarts and select every time:
0: [CONTINUE] GO ahead and change the value.
Do this as many times (about 6 times that is) as it needs to, and it will load eventually. This is probably happening because of AIMA's code non-standard build/compile system. This can be annoying, but an alternative is to trace the code and see why/where some files are being reloaded.
USING Clozure (CCL):
CCL has a different problem with ./utilities/utilities.lisp. CCL has both true and false functions predefined, therefore you have to make sure that both lines:
#-(or MCL Lispworks)
that directly precede both (defun true ...) and (defun false ...) are changed to:
#-(or MCL Lispworks CCL)
also, in the same source, modify error inside for-each macro to look like so:
(error "~a is an illegal variable in (for each ~a in ~a ...)"
var var list)
With these modifications, CCL seems load AIMA code just fine.
In general:
It's a bad idea to redefine constants or to somehow bypass debugger restarts. Best solution is to have (defconstant ...)s evaluated once only, perhaps by placing them in a separate source file and making sure that the build system picks it up only once.
Another solution found here, which entails wrapping calls to defconstantin a macro like so (borrowed from here):
(defmacro define-constant (name value &optional doc)
(if (boundp name)
(format t
"~&already defined ~A~%old value ~s~%attempted value ~s~%"
name (symbol-value name) value))
`(defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value)
,#(when doc (list doc))))
And then replacing all occurrences of defconstant like so:
(defconstant +no-bindings+ '((nil))
"Indicates unification success, with no variables.")
with:
(define-constant +no-bindings+ '((nil))
"Indicates unification success, with no variables.")
If you opt for define-constant "solution", make sure that define-constant is evaluated first.

How to know whether a racket variable is defined or not

How you can have a different behaviour if a variable is defined or not in racket language?
There are several ways to do this. But I suspect that none of these is what you want, so I'll only provide pointers to the functions (and explain the problems with each one):
namespace-variable-value is a function that retrieves the value of a toplevel variable from some namespace. This is useful only with REPL interaction and REPL code though, since code that is defined in a module is not going to use these things anyway. In other words, you can use this function (and the corresponding namespace-set-variable-value!) to get values (if any) and set them, but the only use of these values is in code that is not itself in a module. To put this differently, using this facility is as good as keeping a hash table that maps symbols to values, only it's slightly more convenient at the REPL since you just type names...
More likely, these kind of things are done in macros. The first way to do this is to use the special #%top macro. This macro gets inserted automatically for all names in a module that are not known to be bound. The usual thing that this macro does is throw an error, but you can redefine it in your code (or make up your own language that redefines it) that does something else with these unknown names.
A slightly more sophisticated way to do this is to use the identifier-binding function -- again, in a macro, not at runtime -- and use it to get information about some name that is given to the macro and decide what to expand to based on that name.
The last two options are the more useful ones, but they're not the newbie-level kind of macros, which is why I suspect that you're asking the wrong question. To clarify, you can use them to write a kind of a defined? special form that checks whether some name is defined, but that question is one that would be answered by a macro, based on the rest of the code, so it's not really useful to ask it. If you want something like that that can enable the kind of code in other dynamic languages where you use such a predicate, then the best way to go about this is to redefine #%top to do some kind of a lookup (hashtable or global namespace) instead of throwing a compilation error -- but again, the difference between that and using a hash table explicitly is mostly cosmetic (and again, this is not a newbie thing).
First, read Eli's answer. Then, based on Eli's answer, you can implement the defined? macro this way:
#lang racket
; The macro
(define-syntax (defined? stx)
(syntax-case stx ()
[(_ id)
(with-syntax ([v (identifier-binding #'id)])
#''v)]))
; Tests
(define x 3)
(if (defined? x) 'defined 'not-defined) ; -> defined
(let ([y 4])
(if (defined? y) 'defined 'not-defined)) ; -> defined
(if (defined? z) 'defined 'not-defined) ; -> not-defined
It works for this basic case, but it has a problem: if z is undefined, the branch of the if that considers that it is defined and uses its value will raise a compile-time error, because the normal if checks its condition value at run-time (dynamically):
; This doesn't work because z in `(list z)' is undefined:
(if (defined? z) (list z) 'not-defined)
So what you probably want is a if-defined macro, that tells at compile-time (instead of at run-time) what branch of the if to take:
#lang racket
; The macro
(define-syntax (if-defined stx)
(syntax-case stx ()
[(_ id iftrue iffalse)
(let ([where (identifier-binding #'id)])
(if where #'iftrue #'iffalse))]))
; Tests
(if-defined z (list z) 'not-defined) ; -> not-defined
(if-defined t (void) (define t 5))
t ; -> 5
(define x 3)
(if-defined x (void) (define x 6))
x ; -> 3