How does read-line work in Lisp when reaching eof? - file-io

Context:
I have a text file called fr.txt with 3 columns of text in it:
65 A #\A
97 a #\a
192 À #\latin_capital_letter_a_with_grave
224 à #\latin_small_letter_a_with_grave
etc...
I want to create a function to read the first (and eventually the third one too) column and write it into another text file called alphabet_code.txt.
So far I have this function:
(defun alphabets()
(setq source (open "fr.txt" :direction :input :if-does-not-exist :error))
(setq code (open "alphabet_code.txt" :direction :output :if-does-not-exist :create :if-exists :supersede))
(loop
(setq ligne (read-line source nil nil))
(cond
((equal ligne nil) (return))
(t (print (read-from-string ligne) code))
)
)
(close code)
(close source)
)
My problems:
I don't really understand how the parameters of read-line function. I have read this doc, but it's still very obscure to me. If someone would have very simple examples, that would help.
With the current code, I get this error: *** - read: input stream #<input string-input-stream> has reached its end even if I change the nil nil in (read-line source nil nil) to other values.
Thanks for your time!

Your questions
read-line optional arguments
read-line accepts 3 optional arguments:
eof-error-p: what to do on EOF (default: error)
eof-value: what to return instead of the error when you see EOF
recursive-p: are you calling it from your print-object method (forget about this for now)
E.g., when the stream is at EOF,
(read-line stream) will signal the end-of-file error
(read-line stream nil) will return nil
(read-line stream nil 42) will return 42.
Note that (read-line stream nil) is the same as (read-line stream nil nil) but people usually still pass the second optional argument explicitly.
eof-value of nil is fine for read-line because nil is not a string and read-line only returns strings.
Note also that in case of read the second optional argument is, traditionally, the stream itself: (read stream nil stream). It's quite convenient.
Error
You are getting the error from read-from-string, not read-line, because, apparently, you have an empty line in your file.
I know that because the error mentions string-input-stream, not file-stream.
Your code
Your code is correct functionally, but very wrong stylistically.
You should use with-open-file whenever possible.
You should not use print in code, it's a weird legacy function mostly for interactive use.
You can't create local variables with setq - use let or other equivalent forms (in this case, you never need let! :-)
Here is how I would re-write your function:
(defun alphabets (input-file output-file)
(with-open-stream (source input-file)
(with-open-stream (code output-file :direction :output :if-exists :supersede)
(loop for line = (read-line source nil nil)
as num = (parse-integer line :junk-allowed t)
while line do
(when num
(write num :stream code)
(write-char #\Newline code))))))
(alphabets "fr.txt" "alphabet_code.txt")
See the docs:
loop: for/as, while, do
write, write-char
parse-integer
Alternatively, instead of (when num ...) I could have use the corresponding loop conditional.
Also, instead of write+write-char I could have written (format code "~D~%" num).
Note that I do not pass those of your with-open-stream arguments that are identical to the defaults.
The defaults are set in stone, and the less code you have to write and your user has to read, the less is the chance of an error.

Related

Error running timer: (void-variable message) in Emacs init.el

Why do I get Error running timer: (void-variable message)
in the function below in my `init.el - Emacs?
(defun cypher/cowsayx-sclock (in-minutes message)
(interactive "nSet the time from now - min.: \nsWhat: ")
(run-at-time (* in-minutes 60)
nil
(lambda ()
(message "%S" message)
(shell-command (format "xcowsay %s" (shell-quote-argument
message))))))
You need to turn on lexical-binding, for that message occurrence in the lambda not to be treated as a free variable. It's a lexical variable local to function cypher/cowsayx-sclock, but within the lambda it's free.
Otherwise, you need to instead substitute the value of variable message in the lambda expression, and use that as a list. Here's a backquoted expression that gives you that list with the message value substituted.
`(lambda ()
(message "%S" ',message)
(shell-command (format "xcowsay %s" (shell-quote-argument ',message)))
But this is less performant than using lexical-binding, which produces a closure for the lambda, encapsulating the value of message.
See the Elisp manual, node Using Lexical Binding.
You can, for example, just put this at the end of a comment line as the first line of your file:
-*- lexical-binding:t -*-
For example, if your code is in file foo.el then this could be its first line:
;;; foo.el --- Code that does foo things. -*- lexical-binding:t -*-

How to properly ask an input from user in LispWorks?

I have this code:
(defvar x)
(setq x (read))
(format t "this is your input: ~a" x)
It kinda work in Common Lisp but LispWorks it is showing this error:
End of file while reading stream #<Synonym stream to
*BACKGROUND-INPUT*>.
I mean I tried this: How to read user input in Lisp of making a function. But still shows the same error.
I hope anyone can help me.
You probably wrote these three lines into Editor and then compiled it.
So, you can write this function into Editor:
(defun get-input ()
(format t "This is your input: ~a" (read)))
Compile Editor and call this function from Listener (REPL).
CL-USER 6 > (get-input)
5
This is your input: 5
NIL
You can also use *query-io* stream like this:
(format t "This is your input: ~a" (read *query-io*))
If you call this line in Listener, it behaves like read. If you call it in Editor, it shows small prompt "Enter something:".
As you can see, no global variable was needed. If you need to do something with that given value, use let, which creates local bindings:
(defun input-sum ()
(let ((x (read *query-io*))
(y (read *query-io*)))
(format t "This is x: ~a~%" x)
(format t "This is y: ~a~%" y)
(+ x y)))
Consider also using read-line, which takes input and returns it as string:
CL-USER 19 > (read-line)
some text
"some text"
NIL
CL-USER 20 > (read-line)
5
"5"
NIL

How to load non-standard characters from file using SBCL Common Lisp? [duplicate]

This question already has an answer here:
How to handle accents in Common Lisp (SBCL)?
(1 answer)
Closed 6 years ago.
Trying
loading contents of file containing one line with word: λέξη
(with-open-file (s PATH-TO-FILE :direction :input)
(let ((a (read-line s)))
(print a)))
outputs
""
T
Trying:
(with-open-file (s PATH-TO-FILE :direction :input)
(let ((buffer ""))
(do ((character (read-char s nil) (read-char s nil)))
((null character))
(setf buffer (concatenate 'string buffer (format nil "~a" character))))
(format t "~a" buffer)))
outputs
funny characters (nothing like the original contents)
T
What I would like to do is to load all lines of file containing such non-standard characters.
Then I want to be able to output these words to console or via LTK widgets (text on button for example).
You need to pass :external-format X to with-open-file, where
X is the actual encoding used in the file (:utf-8 or :ISO-8859-7 or whatever).
(with-open-file (stream PATH-TO-FILE :external-format :utf-8)
(let ((line (read-line stream)))
(loop :for char :across line :do
(print (list (char-name char) (char-code char))))
line))
Since it prints
("ZERO_WIDTH_NO-BREAK_SPACE" 65279)
("GREEK_SMALL_LETTER_LAMDA" 955)
("GREEK_SMALL_LETTER_EPSILON_WITH_TONOS" 941)
("GREEK_SMALL_LETTER_XI" 958)
("GREEK_SMALL_LETTER_ETA" 951)
you can see that you are, indeed, reading the file correctly.
Your problem now if how to print those non-ASCII characters to the screen, and that is an entirely different question.

Pop3 Over SSL/TLS in Common Lisp

Can anyone point me to a Common Lisp library (specifically for SBCL on Linux) for pulling pop3 email over SSL/TLS? Cl-pop seems fine, but it doesn't seem to support SSL and I'm not sure how to wrap it into CL+SSL (assuming it's possible). Does anyone have any suggestions short of rolling your own?
You can redefine the usocket-connect function to yield the stream type returned by the SSL library. Then you can define methods to send and receive data over this stream using regular strings (the SSL library only supports binary by default, but CL-POP assumes that strings can be sent). You'll need to depend on the FLEXI-STREAMS library to convert between text and binary. (ql:quickload :flexi-streams)
The following is code to make the change and define the needed methods. Since usocket-connect is replaced, I provide the :unencrypted keyword to create a regular socket.
The code could probably be made more efficient.
The string-to-octets and octets-to-string functions support an :external-format argument which allows them to encode/decode many character encoding schemes, including UTF-8, ISO-8859-*, and others. The full list of supported encodings is documented here. I didn't use :external-format in this answer, so it defaults to :latin-1.
The code is written against an old version of CL+SSL that seems to have been installed on my system by the Debian package manager. The current versions of make-ssl-client-stream and make-ssl-server-stream support several more keyword arguments than are supported by the version on my machine. It doesn't matter, however, because CL-POP will use none of these keyword arguments.
(defpackage :ssl-pop
(:use :common-lisp :cl+ssl :usocket :flexi-streams))
(in-package :ssl-pop)
(let ((old-connect (symbol-function 'socket-connect)))
(defun socket-connect (host port &key (protocol :stream)
external-format certificate key crypto-password
(clientp t) close-callback unencrypted
(unwrap-streams-p t) crypto-hostname
(element-type '(unsigned-byte 8)) timeout deadline
(nodelay t nodelay-specified) local-host
local-port)
(let* ((old-connect-args
`(,host ,port :protocol ,protocol
:element-type ,element-type
:timeout ,timeout :deadline ,deadline
,#(if nodelay-specified
`(:nodelay ,nodelay))
:local-host ,local-host
:local-port ,local-port))
(plain-socket (apply old-connect old-connect-args)))
(if unencrypted
plain-socket
(let ((socket-stream (socket-stream plain-socket)))
(assert (streamp socket-stream))
(if clientp
(make-ssl-client-stream socket-stream
:external-format external-format
:certificate certificate
:key key
:close-callback close-callback)
(make-ssl-server-stream socket-stream
:external-format external-format
:certificate certificate
:key key)))))))
(defmethod socket-stream ((object cl+ssl::ssl-stream))
object)
(defmethod socket-receive ((socket cl+ssl::ssl-stream) buffer length
&key (element-type '(unsigned-byte 8)))
(let ((buffer (or buffer (make-array length
:element-type element-type))))
(loop for ix from 0 below length
do
(restart-case
(setf (aref buffer ix) (read-byte socket))
(thats-ok () :report "Return the bytes that were successfully read"
(return-from socket-receive (subseq buffer 0 ix)))))
buffer))
(defmethod socket-send ((socket cl+ssl::ssl-stream) buffer length
&key host port)
(declare (ignore host port)) ;; They're for UDP
(loop for byte across buffer
do (write-byte byte socket)))
(defmethod sb-gray:stream-read-line ((socket cl+ssl::ssl-stream))
(let ((result (make-array 0 :adjustable t :fill-pointer t
:element-type '(unsigned-byte 8))))
(loop for next-byte = (read-byte socket)
until (and (>= (length result) 1)
(= next-byte 10)
(= (aref result (- (length result) 1)) 13))
do
(vector-push-extend next-byte result))
(octets-to-string
(concatenate 'vector
(subseq result 0 (- (length result) 1))))))
(defmethod trivial-gray-streams:stream-write-sequence
((stream cl+ssl::ssl-stream) (sequence string) start end
&key &allow-other-keys)
(trivial-gray-streams:stream-write-sequence stream
(string-to-octets sequence)
start end))
(defmethod sb-gray:stream-write-char ((stream cl+ssl::ssl-stream)
(char character))
(let ((string (make-string 1 :initial-element char)))
(write-sequence (string-to-octets string) stream)))
(defmethod socket-close ((socket cl+ssl::ssl-stream))
(close socket))

Input stream ends within an object

I want to count the number of rows in a flat file, and so I wrote the code:
(defun ff-rows (dir file)
(with-open-file (str (make-pathname :name file
:directory dir)
:direction :input)
(let ((rownum 0))
(do ((line (read-line str file nil 'eof)
(read-line str file nil 'eof)))
((eql line 'eof) rownum)
(incf rownum )))))
However I get the error:
*** - READ: input stream
#<INPUT BUFFERED FILE-STREAM CHARACTER #P"/home/lambda/Documents/flatfile"
#4>
ends within an object
May I ask what the problem is here? I tried counting the rows; this operation is fine.
Note: Here is contents of the flat file that I used to test the function:
2 3 4 6 2
1 2 3 1 2
2 3 4 1 6
A bit shorter.
(defun ff-rows (dir file)
(with-open-file (stream (make-pathname :name file
:directory dir)
:direction :input)
(loop for line = (read-line stream nil nil)
while line count line)))
Note that you need to get the arguments for READ-LINE right. First is the stream. A file is not part of the parameter list.
Also generally is not a good idea to mix pathname handling into general Lisp functions.
(defun ff-rows (pathname)
(with-open-file (stream pathname :direction :input)
(loop for line = (read-line stream nil nil)
while line count line)))
Do the pathname handling in another function or some other code. Passing pathname components to functions is usually a wrong design. Pass complete pathnames.
Using a LispWorks file selector:
CL-USER 2 > (ff-rows (capi:prompt-for-file "some file"))
27955
Even better is when all the basic I/O functions work on streams, and not pathnames. Thus you you could count lines in a network stream, a serial line or some other stream.
The problem, as far as I can tell, is the "file" in your (read-line ... ) call.
Based on the hyperspec, the signature of read-line is:
read-line &optional input-stream eof-error-p eof-value recursive-p
=> line, missing-newline-p
...which means that "file" is interpreted as eof-error-p, nil as eof-value and 'eof as recursive-p. Needless to say, problems ensue. If you remove "file" from the read-line call (e.g. (read-line str nil :eof)), the code runs fine without further modifications on my machine (AllegroCL & LispWorks.)
(defun ff-rows (dir file)
(with-open-file
(str (make-pathname :name file :directory dir)
:direction :input)
(let ((result 0))
(handler-case
(loop (progn (incf result) (read-line str)))
(end-of-file () (1- result))
(error () result)))))
Now, of course if you were more pedantic then I am, you could've specified what kind of error you want to handle exactly, but for the simple example this will do.
EDIT: I think #Moritz answered the question better, still this may be an example of how to use the error thrown by read-line to your advantage instead of trying to avoid it.